Mercurial > hg
annotate hgext/clonebundles.py @ 51656:21442f87f7fa stable
mmap: only use mmap to read revlog persistent nodemap if it is safe
Cf `is_mmap_safe` docstring.
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Wed, 03 Jul 2024 12:32:57 +0200 |
parents | d718eddf01d9 |
children |
rev | line source |
---|---|
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # This software may be used and distributed according to the terms of the |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # GNU General Public License version 2 or any later version. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 |
27738
a0e783d26e81
exchange: make clone bundles non-experimental and enabled by default
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27737
diff
changeset
|
4 """advertise pre-generated bundles to seed clones |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
5 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
6 "clonebundles" is a server-side extension used to advertise the existence |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
7 of pre-generated, externally hosted bundle files to clients that are |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
8 cloning so that cloning can be faster, more reliable, and require less |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
9 resources on the server. "pullbundles" is a related feature for sending |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
10 pre-generated bundle files to clients as part of pull operations. |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
11 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
12 Cloning can be a CPU and I/O intensive operation on servers. Traditionally, |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
13 the server, in response to a client's request to clone, dynamically generates |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
14 a bundle containing the entire repository content and sends it to the client. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
15 There is no caching on the server and the server will have to redundantly |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
16 generate the same outgoing bundle in response to each clone request. For |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
17 servers with large repositories or with high clone volume, the load from |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
18 clones can make scaling the server challenging and costly. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
19 |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
20 This extension provides server operators the ability to offload |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
21 potentially expensive clone load to an external service. Pre-generated |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
22 bundles also allow using more CPU intensive compression, reducing the |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
23 effective bandwidth requirements. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
24 |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
25 Here's how clone bundles work: |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
26 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
27 1. A server operator establishes a mechanism for making bundle files available |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
28 on a hosting service where Mercurial clients can fetch them. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
29 2. A manifest file listing available bundle URLs and some optional metadata |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
30 is added to the Mercurial repository on the server. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
31 3. A client initiates a clone against a clone bundles aware server. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
32 4. The client sees the server is advertising clone bundles and fetches the |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
33 manifest listing available bundles. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
34 5. The client filters and sorts the available bundles based on what it |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
35 supports and prefers. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
36 6. The client downloads and applies an available bundle from the |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
37 server-specified URL. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
38 7. The client reconnects to the original server and performs the equivalent |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
39 of :hg:`pull` to retrieve all repository data not in the bundle. (The |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
40 repository could have been updated between when the bundle was created |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
41 and when the client started the clone.) This may use "pullbundles". |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
42 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
43 Instead of the server generating full repository bundles for every clone |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
44 request, it generates full bundles once and they are subsequently reused to |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
45 bootstrap new clones. The server may still transfer data at clone time. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
46 However, this is only data that has been added/changed since the bundle was |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
47 created. For large, established repositories, this can reduce server load for |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
48 clones to less than 1% of original. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
49 |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
50 Here's how pullbundles work: |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
51 |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
52 1. A manifest file listing available bundles and describing the revisions |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
53 is added to the Mercurial repository on the server. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
54 2. A new-enough client informs the server that it supports partial pulls |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
55 and initiates a pull. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
56 3. If the server has pull bundles enabled and sees the client advertising |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
57 partial pulls, it checks for a matching pull bundle in the manifest. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
58 A bundle matches if the format is supported by the client, the client |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
59 has the required revisions already and needs something from the bundle. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
60 4. If there is at least one matching bundle, the server sends it to the client. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
61 5. The client applies the bundle and notices that the server reply was |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
62 incomplete. It initiates another pull. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
63 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
64 To work, this extension requires the following of server operators: |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
65 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
66 * Generating bundle files of repository content (typically periodically, |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
67 such as once per day). |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
68 * Clone bundles: A file server that clients have network access to and that |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
69 Python knows how to talk to through its normal URL handling facility |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
70 (typically an HTTP/HTTPS server). |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
71 * A process for keeping the bundles manifest in sync with available bundle |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
72 files. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
73 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
74 Strictly speaking, using a static file hosting server isn't required: a server |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
75 operator could use a dynamic service for retrieving bundle data. However, |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
76 static file hosting services are simple and scalable and should be sufficient |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
77 for most needs. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
78 |
26884
762bf510b42c
clonebundles: fix typo s/comand/command/
Javi Merino <merino.jav@gmail.com>
parents:
26857
diff
changeset
|
79 Bundle files can be generated with the :hg:`bundle` command. Typically |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
80 :hg:`bundle --all` is used to produce a bundle of the entire repository. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
81 |
50645
bf7404f2e22d
bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
48875
diff
changeset
|
82 The bundlespec option `stream` (see :hg:`help bundlespec`) |
bf7404f2e22d
bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
48875
diff
changeset
|
83 can be used to produce a special *streaming clonebundle*, typically using |
bf7404f2e22d
bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
48875
diff
changeset
|
84 :hg:`bundle --all --type="none-streamv2"`. |
bf7404f2e22d
bundles: clarify streaming v2 bundle usage
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
48875
diff
changeset
|
85 These are bundle files that are extremely efficient |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
86 to produce and consume (read: fast). However, they are larger than |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
87 traditional bundle formats and require that clients support the exact set |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
88 of repository data store formats in use by the repository that created them. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
89 Typically, a newer server can serve data that is compatible with older clients. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
90 However, *streaming clone bundles* don't have this guarantee. **Server |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
91 operators need to be aware that newer versions of Mercurial may produce |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
92 streaming clone bundles incompatible with older Mercurial versions.** |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
93 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
94 A server operator is responsible for creating a ``.hg/clonebundles.manifest`` |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
95 file containing the list of available bundle files suitable for seeding |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
96 clones. If this file does not exist, the repository will not advertise the |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
97 existence of clone bundles when clients connect. For pull bundles, |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
98 ``.hg/pullbundles.manifest`` is used. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
99 |
32454
702af1ad3b18
clonebundles: fix missing newline character
Matt Harbison <matt_harbison@yahoo.com>
parents:
31146
diff
changeset
|
100 The manifest file contains a newline (\\n) delimited list of entries. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
101 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
102 Each line in this file defines an available bundle. Lines have the format: |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
103 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
104 <URL> [<key>=<value>[ <key>=<value>]] |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
105 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
106 That is, a URL followed by an optional, space-delimited list of key=value |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
107 pairs describing additional properties of this bundle. Both keys and values |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
108 are URI encoded. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
109 |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
110 For pull bundles, the URL is a path under the ``.hg`` directory of the |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
111 repository. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
112 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
113 Keys in UPPERCASE are reserved for use by Mercurial and are defined below. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
114 All non-uppercase keys can be used by site installations. An example use |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
115 for custom properties is to use the *datacenter* attribute to define which |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
116 data center a file is hosted in. Clients could then prefer a server in the |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
117 data center closest to them. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
118 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
119 The following reserved keys are currently defined: |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
120 |
26644
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
121 BUNDLESPEC |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
122 A "bundle specification" string that describes the type of the bundle. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
123 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
124 These are string values that are accepted by the "--type" argument of |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
125 :hg:`bundle`. |
26644
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
126 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
127 The values are parsed in strict mode, which means they must be of the |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
128 "<compression>-<type>" form. See |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
129 mercurial.exchange.parsebundlespec() for more details. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
130 |
27886
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
131 :hg:`debugbundle --spec` can be used to print the bundle specification |
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
132 string for a bundle file. The output of this command can be used verbatim |
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
133 for the value of ``BUNDLESPEC`` (it is already escaped). |
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
134 |
26644
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
135 Clients will automatically filter out specifications that are unknown or |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
136 unsupported so they won't attempt to download something that likely won't |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
137 apply. |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
138 |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
139 The actual value doesn't impact client behavior beyond filtering: |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
140 clients will still sniff the bundle type from the header of downloaded |
74de1c59f71c
clonebundles: filter on bundle specification
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26623
diff
changeset
|
141 files. |
26645
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
142 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
143 **Use of this key is highly recommended**, as it allows clients to |
27886
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
144 easily skip unsupported bundles. If this key is not defined, an old |
0288e63ea3be
clonebundles: improve BUNDLESPEC documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27738
diff
changeset
|
145 client may attempt to apply a bundle that it is incapable of reading. |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
146 |
26645
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
147 REQUIRESNI |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
148 Whether Server Name Indication (SNI) is required to connect to the URL. |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
149 SNI allows servers to use multiple certificates on the same IP. It is |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
150 somewhat common in CDNs and other hosting providers. Older Python |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
151 versions do not support SNI. Defining this attribute enables clients |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
152 with older Python versions to filter this entry without experiencing |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
153 an opaque SSL failure at connection time. |
26645
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
154 |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
155 If this is defined, it is important to advertise a non-SNI fallback |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
156 URL or clients running old Python releases may not be able to clone |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
157 with the clonebundles facility. |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
158 |
2faa7671a4b3
clonebundles: filter on SNI requirement
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26644
diff
changeset
|
159 Value should be "true". |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
160 |
45252
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
161 REQUIREDRAM |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
162 Value specifies expected memory requirements to decode the payload. |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
163 Values can have suffixes for common bytes sizes. e.g. "64MB". |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
164 |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
165 This key is often used with zstd-compressed bundles using a high |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
166 compression level / window size, which can require 100+ MB of memory |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
167 to decode. |
20d110e6eea6
clonebundles: document REQUIREDRAM key
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43077
diff
changeset
|
168 |
37498
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
169 heads |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
170 Used for pull bundles. This contains the ``;`` separated changeset |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
171 hashes of the heads of the bundle content. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
172 |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
173 bases |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
174 Used for pull bundles. This contains the ``;`` separated changeset |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
175 hashes of the roots of the bundle content. This can be skipped if |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
176 the bundle was created without ``--base``. |
aacfca6f9767
wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents:
32773
diff
changeset
|
177 |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
178 Manifests can contain multiple entries. Assuming metadata is defined, clients |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
179 will filter entries from the manifest that they don't support. The remaining |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
180 entries are optionally sorted by client preferences |
32773
d25802b0eef5
clonebundles: reference correct config option
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32454
diff
changeset
|
181 (``ui.clonebundleprefers`` config option). The client then attempts |
26762
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
182 to fetch the bundle at the first URL in the remaining list. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
183 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
184 **Errors when downloading a bundle will fail the entire clone operation: |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
185 clients do not automatically fall back to a traditional clone.** The reason |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
186 for this is that if a server is using clone bundles, it is probably doing so |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
187 because the feature is necessary to help it scale. In other words, there |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
188 is an assumption that clone load will be offloaded to another service and |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
189 that the Mercurial server isn't responsible for serving this clone load. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
190 If that other service experiences issues and clients start mass falling back to |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
191 the original Mercurial server, the added clone load could overwhelm the server |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
192 due to unexpected load and effectively take it offline. Not having clients |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
193 automatically fall back to cloning from the original server mitigates this |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
194 scenario. |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
195 |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
196 Because there is no automatic Mercurial server fallback on failure of the |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
197 bundle hosting service, it is important for server operators to view the bundle |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
198 hosting service as an extension of the Mercurial server in terms of |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
199 availability and service level agreements: if the bundle hosting service goes |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
200 down, so does the ability for clients to clone. Note: clients will see a |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
201 message informing them how to bypass the clone bundles facility when a failure |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
202 occurs. So server operators should prepare for some people to follow these |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
203 instructions when a failure occurs, thus driving more load to the original |
26f622859288
clonebundles: rewrite documentation
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26691
diff
changeset
|
204 Mercurial server when the bundle hosting service fails. |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
205 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
206 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
207 inline clonebundles |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
208 ------------------- |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
209 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
210 It is possible to transmit clonebundles inline in case repositories are |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
211 accessed over SSH. This avoids having to setup an external HTTPS server |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
212 and results in the same access control as already present for the SSH setup. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
213 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
214 Inline clonebundles should be placed into the `.hg/bundle-cache` directory. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
215 A clonebundle at `.hg/bundle-cache/mybundle.bundle` is referred to |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
216 in the `clonebundles.manifest` file as `peer-bundle-cache://mybundle.bundle`. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
217 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
218 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
219 auto-generation of clone bundles |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
220 -------------------------------- |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
221 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
222 It is possible to set Mercurial to automatically re-generate clone bundles when |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
223 enough new content is available. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
224 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
225 Mercurial will take care of the process asynchronously. The defined list of |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
226 bundle-type will be generated, uploaded, and advertised. Older bundles will get |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
227 decommissioned as newer ones replace them. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
228 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
229 Bundles Generation: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
230 ................... |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
231 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
232 The extension can generate multiple variants of the clone bundle. Each |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
233 different variant will be defined by the "bundle-spec" they use:: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
234 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
235 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
236 auto-generate.formats= zstd-v2, gzip-v2 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
237 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
238 See `hg help bundlespec` for details about available options. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
239 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
240 By default, new bundles are generated when 5% of the repository contents or at |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
241 least 1000 revisions are not contained in the cached bundles. This option can |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
242 be controlled by the `clone-bundles.trigger.below-bundled-ratio` option |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
243 (default 0.95) and the `clone-bundles.trigger.revs` option (default 1000):: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
244 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
245 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
246 trigger.below-bundled-ratio=0.95 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
247 trigger.revs=1000 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
248 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
249 This logic can be manually triggered using the `admin::clone-bundles-refresh` |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
250 command, or automatically on each repository change if |
50735
2b0598121a71
clonebundles: fix display of auto-generate.on-change lines
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50713
diff
changeset
|
251 `clone-bundles.auto-generate.on-change` is set to `yes`:: |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
252 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
253 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
254 auto-generate.on-change=yes |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
255 auto-generate.formats= zstd-v2, gzip-v2 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
256 |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
257 Automatic Inline serving |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
258 ........................ |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
259 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
260 The simplest way to serve the generated bundle is through the Mercurial |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
261 protocol. However it is not the most efficient as request will still be served |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
262 by that main server. It is useful in case where authentication is complexe or |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
263 when an efficient mirror system is already in use anyway. See the `inline |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
264 clonebundles` section above for details about inline clonebundles |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
265 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
266 To automatically serve generated bundle through inline clonebundle, simply set |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
267 the following option:: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
268 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
269 auto-generate.serve-inline=yes |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
270 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
271 Enabling this option disable the managed upload and serving explained below. |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
272 |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
273 Bundles Upload and Serving: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
274 ........................... |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
275 |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
276 This is the most efficient way to serve automatically generated clone bundles, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
277 but requires some setup. |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
278 |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
279 The generated bundles need to be made available to users through a "public" URL. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
280 This should be donne through `clone-bundles.upload-command` configuration. The |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
281 value of this command should be a shell command. It will have access to the |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
282 bundle file path through the `$HGCB_BUNDLE_PATH` variable. And the expected |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
283 basename in the "public" URL is accessible at:: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
284 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
285 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
286 upload-command=sftp put $HGCB_BUNDLE_PATH \ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
287 sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
288 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
289 If the file was already uploaded, the command must still succeed. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
290 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
291 After upload, the file should be available at an url defined by |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
292 `clone-bundles.url-template`. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
293 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
294 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
295 url-template=https://bundles.host/cache/clone-bundles/{basename} |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
296 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
297 Old bundles cleanup: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
298 .................... |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
299 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
300 When new bundles are generated, the older ones are no longer necessary and can |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
301 be removed from storage. This is done through the `clone-bundles.delete-command` |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
302 configuration. The command is given the url of the artifact to delete through |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
303 the `$HGCB_BUNDLE_URL` environment variable. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
304 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
305 [clone-bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
306 delete-command=sftp rm sftp://bundles.host/clone-bundles/$HGCB_BUNDLE_BASENAME |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
307 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
308 If the file was already deleted, the command must still succeed. |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
309 """ |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
310 |
28095
7fa139eaebb4
clonebundles: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27886
diff
changeset
|
311 |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
312 import os |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
313 import weakref |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
314 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
315 from mercurial.i18n import _ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
316 |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
317 from mercurial import ( |
45785
80f32ec8653a
clonebundle: move the manifest filename to a constant
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45252
diff
changeset
|
318 bundlecaches, |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
319 commands, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
320 error, |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
321 extensions, |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
322 localrepo, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
323 lock, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
324 node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
325 registrar, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
326 util, |
37785
b4d85bc122bd
wireproto: rename wireproto to wireprotov1server (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37498
diff
changeset
|
327 wireprotov1server, |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
328 ) |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
329 |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
330 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
331 from mercurial.utils import ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
332 procutil, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
333 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
334 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
335 testedwith = b'ships-with-hg-core' |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
336 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
37785
diff
changeset
|
337 |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
338 def capabilities(orig, repo, proto): |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
339 caps = orig(repo, proto) |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
340 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
341 # Only advertise if a manifest exists. This does add some I/O to requests. |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
342 # But this should be cheaper than a wasted network round trip due to |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
343 # missing file. |
45785
80f32ec8653a
clonebundle: move the manifest filename to a constant
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45252
diff
changeset
|
344 if repo.vfs.exists(bundlecaches.CB_MANIFEST_FILE): |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
345 caps.append(b'clonebundles') |
50701
4238e6b22fc4
clonebundles: introduce a new write protocol command
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50686
diff
changeset
|
346 caps.append(b'clonebundles_manifest') |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
347 |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
348 return caps |
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
349 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
37785
diff
changeset
|
350 |
26623
5a95fe44121d
clonebundles: support for seeding clones from pre-generated bundles
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
351 def extsetup(ui): |
50776
335aeb3d670b
wrapfunction: use sysstr instead of bytes as argument in "clonebundles"
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50736
diff
changeset
|
352 extensions.wrapfunction(wireprotov1server, '_capabilities', capabilities) |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
353 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
354 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
355 # logic for bundle auto-generation |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
356 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
357 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
358 configtable = {} |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
359 configitem = registrar.configitem(configtable) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
360 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
361 cmdtable = {} |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
362 command = registrar.command(cmdtable) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
363 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
364 configitem(b'clone-bundles', b'auto-generate.on-change', default=False) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
365 configitem(b'clone-bundles', b'auto-generate.formats', default=list) |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
366 configitem(b'clone-bundles', b'auto-generate.serve-inline', default=False) |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
367 configitem(b'clone-bundles', b'trigger.below-bundled-ratio', default=0.95) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
368 configitem(b'clone-bundles', b'trigger.revs', default=1000) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
369 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
370 configitem(b'clone-bundles', b'upload-command', default=None) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
371 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
372 configitem(b'clone-bundles', b'delete-command', default=None) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
373 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
374 configitem(b'clone-bundles', b'url-template', default=None) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
375 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
376 configitem(b'devel', b'debug.clonebundles', default=False) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
377 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
378 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
379 # category for the post-close transaction hooks |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
380 CAT_POSTCLOSE = b"clonebundles-autobundles" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
381 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
382 # template for bundle file names |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
383 BUNDLE_MASK = ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
384 b"full-%(bundle_type)s-%(revs)d_revs-%(tip_short)s_tip-%(op_id)s.hg" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
385 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
386 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
387 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
388 # file in .hg/ use to track clonebundles being auto-generated |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
389 AUTO_GEN_FILE = b'clonebundles.auto-gen' |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
390 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
391 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
392 class BundleBase(object): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
393 """represents the core of properties that matters for us in a bundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
394 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
395 :bundle_type: the bundlespec (see hg help bundlespec) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
396 :revs: the number of revisions in the repo at bundle creation time |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
397 :tip_rev: the rev-num of the tip revision |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
398 :tip_node: the node id of the tip-most revision in the bundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
399 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
400 :ready: True if the bundle is ready to be served |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
401 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
402 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
403 ready = False |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
404 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
405 def __init__(self, bundle_type, revs, tip_rev, tip_node): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
406 self.bundle_type = bundle_type |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
407 self.revs = revs |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
408 self.tip_rev = tip_rev |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
409 self.tip_node = tip_node |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
410 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
411 def valid_for(self, repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
412 """is this bundle applicable to the current repository |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
413 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
414 This is useful for detecting bundles made irrelevant by stripping. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
415 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
416 tip_node = node.bin(self.tip_node) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
417 return repo.changelog.index.get_rev(tip_node) == self.tip_rev |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
418 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
419 def __eq__(self, other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
420 left = (self.ready, self.bundle_type, self.tip_rev, self.tip_node) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
421 right = (other.ready, other.bundle_type, other.tip_rev, other.tip_node) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
422 return left == right |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
423 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
424 def __neq__(self, other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
425 return not self == other |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
426 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
427 def __cmp__(self, other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
428 if self == other: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
429 return 0 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
430 return -1 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
431 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
432 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
433 class RequestedBundle(BundleBase): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
434 """A bundle that should be generated. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
435 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
436 Additional attributes compared to BundleBase |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
437 :heads: list of head revisions (as rev-num) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
438 :op_id: a "unique" identifier for the operation triggering the change |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
439 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
440 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
441 def __init__(self, bundle_type, revs, tip_rev, tip_node, head_revs, op_id): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
442 self.head_revs = head_revs |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
443 self.op_id = op_id |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
444 super(RequestedBundle, self).__init__( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
445 bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
446 revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
447 tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
448 tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
449 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
450 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
451 @property |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
452 def suggested_filename(self): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
453 """A filename that can be used for the generated bundle""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
454 data = { |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
455 b'bundle_type': self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
456 b'revs': self.revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
457 b'heads': self.head_revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
458 b'tip_rev': self.tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
459 b'tip_node': self.tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
460 b'tip_short': self.tip_node[:12], |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
461 b'op_id': self.op_id, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
462 } |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
463 return BUNDLE_MASK % data |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
464 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
465 def generate_bundle(self, repo, file_path): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
466 """generate the bundle at `filepath`""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
467 commands.bundle( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
468 repo.ui, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
469 repo, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
470 file_path, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
471 base=[b"null"], |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
472 rev=self.head_revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
473 type=self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
474 quiet=True, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
475 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
476 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
477 def generating(self, file_path, hostname=None, pid=None): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
478 """return a GeneratingBundle object from this object""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
479 if pid is None: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
480 pid = os.getpid() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
481 if hostname is None: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
482 hostname = lock._getlockprefix() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
483 return GeneratingBundle( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
484 self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
485 self.revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
486 self.tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
487 self.tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
488 hostname, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
489 pid, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
490 file_path, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
491 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
492 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
493 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
494 class GeneratingBundle(BundleBase): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
495 """A bundle being generated |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
496 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
497 extra attributes compared to BundleBase: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
498 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
499 :hostname: the hostname of the machine generating the bundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
500 :pid: the pid of the process generating the bundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
501 :filepath: the target filename of the bundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
502 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
503 These attributes exist to help detect stalled generation processes. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
504 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
505 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
506 ready = False |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
507 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
508 def __init__( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
509 self, bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
510 ): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
511 self.hostname = hostname |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
512 self.pid = pid |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
513 self.filepath = filepath |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
514 super(GeneratingBundle, self).__init__( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
515 bundle_type, revs, tip_rev, tip_node |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
516 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
517 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
518 @classmethod |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
519 def from_line(cls, line): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
520 """create an object by deserializing a line from AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
521 assert line.startswith(b'PENDING-v1 ') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
522 ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
523 __, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
524 bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
525 revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
526 tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
527 tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
528 hostname, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
529 pid, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
530 filepath, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
531 ) = line.split() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
532 hostname = util.urlreq.unquote(hostname) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
533 filepath = util.urlreq.unquote(filepath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
534 revs = int(revs) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
535 tip_rev = int(tip_rev) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
536 pid = int(pid) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
537 return cls( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
538 bundle_type, revs, tip_rev, tip_node, hostname, pid, filepath |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
539 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
540 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
541 def to_line(self): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
542 """serialize the object to include as a line in AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
543 templ = b"PENDING-v1 %s %d %d %s %s %d %s" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
544 data = ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
545 self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
546 self.revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
547 self.tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
548 self.tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
549 util.urlreq.quote(self.hostname), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
550 self.pid, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
551 util.urlreq.quote(self.filepath), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
552 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
553 return templ % data |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
554 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
555 def __eq__(self, other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
556 if not super(GeneratingBundle, self).__eq__(other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
557 return False |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
558 left = (self.hostname, self.pid, self.filepath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
559 right = (other.hostname, other.pid, other.filepath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
560 return left == right |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
561 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
562 def uploaded(self, url, basename): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
563 """return a GeneratedBundle from this object""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
564 return GeneratedBundle( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
565 self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
566 self.revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
567 self.tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
568 self.tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
569 url, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
570 basename, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
571 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
572 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
573 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
574 class GeneratedBundle(BundleBase): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
575 """A bundle that is done being generated and can be served |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
576 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
577 extra attributes compared to BundleBase: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
578 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
579 :file_url: the url where the bundle is available. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
580 :basename: the "basename" used to upload (useful for deletion) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
581 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
582 These attributes exist to generate a bundle manifest |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
583 (.hg/pullbundles.manifest) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
584 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
585 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
586 ready = True |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
587 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
588 def __init__( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
589 self, bundle_type, revs, tip_rev, tip_node, file_url, basename |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
590 ): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
591 self.file_url = file_url |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
592 self.basename = basename |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
593 super(GeneratedBundle, self).__init__( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
594 bundle_type, revs, tip_rev, tip_node |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
595 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
596 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
597 @classmethod |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
598 def from_line(cls, line): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
599 """create an object by deserializing a line from AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
600 assert line.startswith(b'DONE-v1 ') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
601 ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
602 __, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
603 bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
604 revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
605 tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
606 tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
607 file_url, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
608 basename, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
609 ) = line.split() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
610 revs = int(revs) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
611 tip_rev = int(tip_rev) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
612 file_url = util.urlreq.unquote(file_url) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
613 return cls(bundle_type, revs, tip_rev, tip_node, file_url, basename) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
614 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
615 def to_line(self): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
616 """serialize the object to include as a line in AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
617 templ = b"DONE-v1 %s %d %d %s %s %s" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
618 data = ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
619 self.bundle_type, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
620 self.revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
621 self.tip_rev, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
622 self.tip_node, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
623 util.urlreq.quote(self.file_url), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
624 self.basename, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
625 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
626 return templ % data |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
627 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
628 def manifest_line(self): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
629 """serialize the object to include as a line in pullbundles.manifest""" |
50710
1299525832d0
clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50701
diff
changeset
|
630 templ = b"%s BUNDLESPEC=%s" |
1299525832d0
clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50701
diff
changeset
|
631 if self.file_url.startswith(b'http'): |
1299525832d0
clone-bundle: only add the REQUIRESNI bit for http(s)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50701
diff
changeset
|
632 templ += b" REQUIRESNI=true" |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
633 return templ % (self.file_url, self.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
634 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
635 def __eq__(self, other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
636 if not super(GeneratedBundle, self).__eq__(other): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
637 return False |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
638 return self.file_url == other.file_url |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
639 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
640 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
641 def parse_auto_gen(content): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
642 """parse the AUTO_GEN_FILE to return a list of Bundle object""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
643 bundles = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
644 for line in content.splitlines(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
645 if line.startswith(b'PENDING-v1 '): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
646 bundles.append(GeneratingBundle.from_line(line)) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
647 elif line.startswith(b'DONE-v1 '): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
648 bundles.append(GeneratedBundle.from_line(line)) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
649 return bundles |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
650 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
651 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
652 def dumps_auto_gen(bundles): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
653 """serialize a list of Bundle as a AUTO_GEN_FILE content""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
654 lines = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
655 for b in bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
656 lines.append(b"%s\n" % b.to_line()) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
657 lines.sort() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
658 return b"".join(lines) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
659 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
660 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
661 def read_auto_gen(repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
662 """read the AUTO_GEN_FILE for the <repo> a list of Bundle object""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
663 data = repo.vfs.tryread(AUTO_GEN_FILE) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
664 if not data: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
665 return [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
666 return parse_auto_gen(data) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
667 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
668 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
669 def write_auto_gen(repo, bundles): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
670 """write a list of Bundle objects into the repo's AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
671 assert repo._cb_lock_ref is not None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
672 data = dumps_auto_gen(bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
673 with repo.vfs(AUTO_GEN_FILE, mode=b'wb', atomictemp=True) as f: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
674 f.write(data) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
675 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
676 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
677 def generate_manifest(bundles): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
678 """write a list of Bundle objects into the repo's AUTO_GEN_FILE""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
679 bundles = list(bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
680 bundles.sort(key=lambda b: b.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
681 lines = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
682 for b in bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
683 lines.append(b"%s\n" % b.manifest_line()) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
684 return b"".join(lines) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
685 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
686 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
687 def update_ondisk_manifest(repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
688 """update the clonebundle manifest with latest url""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
689 with repo.clonebundles_lock(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
690 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
691 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
692 per_types = {} |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
693 for b in bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
694 if not (b.ready and b.valid_for(repo)): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
695 continue |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
696 current = per_types.get(b.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
697 if current is not None and current.revs >= b.revs: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
698 continue |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
699 per_types[b.bundle_type] = b |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
700 manifest = generate_manifest(per_types.values()) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
701 with repo.vfs( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
702 bundlecaches.CB_MANIFEST_FILE, mode=b"wb", atomictemp=True |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
703 ) as f: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
704 f.write(manifest) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
705 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
706 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
707 def update_bundle_list(repo, new_bundles=(), del_bundles=()): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
708 """modify the repo's AUTO_GEN_FILE |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
709 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
710 This method also regenerates the clone bundle manifest when needed""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
711 with repo.clonebundles_lock(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
712 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
713 if del_bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
714 bundles = [b for b in bundles if b not in del_bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
715 new_bundles = [b for b in new_bundles if b not in bundles] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
716 bundles.extend(new_bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
717 write_auto_gen(repo, bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
718 all_changed = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
719 all_changed.extend(new_bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
720 all_changed.extend(del_bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
721 if any(b.ready for b in all_changed): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
722 update_ondisk_manifest(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
723 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
724 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
725 def cleanup_tmp_bundle(repo, target): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
726 """remove a GeneratingBundle file and entry""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
727 assert not target.ready |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
728 with repo.clonebundles_lock(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
729 repo.vfs.tryunlink(target.filepath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
730 update_bundle_list(repo, del_bundles=[target]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
731 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
732 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
733 def finalize_one_bundle(repo, target): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
734 """upload a generated bundle and advertise it in the clonebundles.manifest""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
735 with repo.clonebundles_lock(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
736 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
737 if target in bundles and target.valid_for(repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
738 result = upload_bundle(repo, target) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
739 update_bundle_list(repo, new_bundles=[result]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
740 cleanup_tmp_bundle(repo, target) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
741 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
742 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
743 def find_outdated_bundles(repo, bundles): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
744 """finds outdated bundles""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
745 olds = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
746 per_types = {} |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
747 for b in bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
748 if not b.valid_for(repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
749 olds.append(b) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
750 continue |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
751 l = per_types.setdefault(b.bundle_type, []) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
752 l.append(b) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
753 for key in sorted(per_types): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
754 all = per_types[key] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
755 if len(all) > 1: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
756 all.sort(key=lambda b: b.revs, reverse=True) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
757 olds.extend(all[1:]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
758 return olds |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
759 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
760 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
761 def collect_garbage(repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
762 """finds outdated bundles and get them deleted""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
763 with repo.clonebundles_lock(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
764 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
765 olds = find_outdated_bundles(repo, bundles) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
766 for o in olds: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
767 delete_bundle(repo, o) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
768 update_bundle_list(repo, del_bundles=olds) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
769 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
770 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
771 def upload_bundle(repo, bundle): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
772 """upload the result of a GeneratingBundle and return a GeneratedBundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
773 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
774 The upload is done using the `clone-bundles.upload-command` |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
775 """ |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
776 inline = repo.ui.config(b'clone-bundles', b'auto-generate.serve-inline') |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
777 basename = repo.vfs.basename(bundle.filepath) |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
778 if inline: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
779 dest_dir = repo.vfs.join(bundlecaches.BUNDLE_CACHE_DIR) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
780 repo.vfs.makedirs(dest_dir) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
781 dest = repo.vfs.join(dest_dir, basename) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
782 util.copyfiles(bundle.filepath, dest, hardlink=True) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
783 url = bundlecaches.CLONEBUNDLESCHEME + basename |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
784 return bundle.uploaded(url, basename) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
785 else: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
786 cmd = repo.ui.config(b'clone-bundles', b'upload-command') |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
787 url = repo.ui.config(b'clone-bundles', b'url-template') |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
788 filepath = procutil.shellquote(bundle.filepath) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
789 variables = { |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
790 b'HGCB_BUNDLE_PATH': filepath, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
791 b'HGCB_BUNDLE_BASENAME': basename, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
792 } |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
793 env = procutil.shellenviron(environ=variables) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
794 ret = repo.ui.system(cmd, environ=env) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
795 if ret: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
796 raise error.Abort(b"command returned status %d: %s" % (ret, cmd)) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
797 url = ( |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
798 url.decode('utf8') |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
799 .format(basename=basename.decode('utf8')) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
800 .encode('utf8') |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
801 ) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
802 return bundle.uploaded(url, basename) |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
803 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
804 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
805 def delete_bundle(repo, bundle): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
806 """delete a bundle from storage""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
807 assert bundle.ready |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
808 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
809 inline = bundle.file_url.startswith(bundlecaches.CLONEBUNDLESCHEME) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
810 |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
811 if inline: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
812 msg = b'clone-bundles: deleting inline bundle %s\n' |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
813 else: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
814 msg = b'clone-bundles: deleting bundle %s\n' |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
815 msg %= bundle.basename |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
816 if repo.ui.configbool(b'devel', b'debug.clonebundles'): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
817 repo.ui.write(msg) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
818 else: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
819 repo.ui.debug(msg) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
820 |
50711
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
821 if inline: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
822 inline_path = repo.vfs.join( |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
823 bundlecaches.BUNDLE_CACHE_DIR, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
824 bundle.basename, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
825 ) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
826 util.tryunlink(inline_path) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
827 else: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
828 cmd = repo.ui.config(b'clone-bundles', b'delete-command') |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
829 variables = { |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
830 b'HGCB_BUNDLE_URL': bundle.file_url, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
831 b'HGCB_BASENAME': bundle.basename, |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
832 } |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
833 env = procutil.shellenviron(environ=variables) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
834 ret = repo.ui.system(cmd, environ=env) |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
835 if ret: |
40638610c6ee
clonebundles: adds a auto-generate.serve-inline option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50710
diff
changeset
|
836 raise error.Abort(b"command returned status %d: %s" % (ret, cmd)) |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
837 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
838 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
839 def auto_bundle_needed_actions(repo, bundles, op_id): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
840 """find the list of bundles that need action |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
841 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
842 returns a list of RequestedBundle objects that need to be generated and |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
843 uploaded.""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
844 create_bundles = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
845 delete_bundles = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
846 repo = repo.filtered(b"immutable") |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
847 targets = repo.ui.configlist(b'clone-bundles', b'auto-generate.formats') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
848 ratio = float( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
849 repo.ui.config(b'clone-bundles', b'trigger.below-bundled-ratio') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
850 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
851 abs_revs = repo.ui.configint(b'clone-bundles', b'trigger.revs') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
852 revs = len(repo.changelog) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
853 generic_data = { |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
854 'revs': revs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
855 'head_revs': repo.changelog.headrevs(), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
856 'tip_rev': repo.changelog.tiprev(), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
857 'tip_node': node.hex(repo.changelog.tip()), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
858 'op_id': op_id, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
859 } |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
860 for t in targets: |
50713
ddc55fb220ba
clone-bundles: check we generate a valid specification
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50711
diff
changeset
|
861 t = bundlecaches.parsebundlespec(repo, t, strict=False).as_spec() |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
862 if new_bundle_needed(repo, bundles, ratio, abs_revs, t, revs): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
863 data = generic_data.copy() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
864 data['bundle_type'] = t |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
865 b = RequestedBundle(**data) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
866 create_bundles.append(b) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
867 delete_bundles.extend(find_outdated_bundles(repo, bundles)) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
868 return create_bundles, delete_bundles |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
869 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
870 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
871 def new_bundle_needed(repo, bundles, ratio, abs_revs, bundle_type, revs): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
872 """consider the current cached content and trigger new bundles if needed""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
873 threshold = max((revs * ratio), (revs - abs_revs)) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
874 for b in bundles: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
875 if not b.valid_for(repo) or b.bundle_type != bundle_type: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
876 continue |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
877 if b.revs > threshold: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
878 return False |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
879 return True |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
880 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
881 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
882 def start_one_bundle(repo, bundle): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
883 """start the generation of a single bundle file |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
884 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
885 the `bundle` argument should be a RequestedBundle object. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
886 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
887 This data is passed to the `debugmakeclonebundles` "as is". |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
888 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
889 data = util.pickle.dumps(bundle) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
890 cmd = [procutil.hgexecutable(), b'--cwd', repo.path, INTERNAL_CMD] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
891 env = procutil.shellenviron() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
892 msg = b'clone-bundles: starting bundle generation: %s\n' |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
893 stdout = None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
894 stderr = None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
895 waits = [] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
896 record_wait = None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
897 if repo.ui.configbool(b'devel', b'debug.clonebundles'): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
898 stdout = procutil.stdout |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
899 stderr = procutil.stderr |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
900 repo.ui.write(msg % bundle.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
901 record_wait = waits.append |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
902 else: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
903 repo.ui.debug(msg % bundle.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
904 bg = procutil.runbgcommand |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
905 bg( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
906 cmd, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
907 env, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
908 stdin_bytes=data, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
909 stdout=stdout, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
910 stderr=stderr, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
911 record_wait=record_wait, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
912 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
913 for f in waits: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
914 f() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
915 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
916 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
917 INTERNAL_CMD = b'debug::internal-make-clone-bundles' |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
918 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
919 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
920 @command(INTERNAL_CMD, [], b'') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
921 def debugmakeclonebundles(ui, repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
922 """Internal command to auto-generate debug bundles""" |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
923 requested_bundle = util.pickle.load(procutil.stdin) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
924 procutil.stdin.close() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
925 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
926 collect_garbage(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
927 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
928 fname = requested_bundle.suggested_filename |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
929 fpath = repo.vfs.makedirs(b'tmp-bundles') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
930 fpath = repo.vfs.join(b'tmp-bundles', fname) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
931 bundle = requested_bundle.generating(fpath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
932 update_bundle_list(repo, new_bundles=[bundle]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
933 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
934 requested_bundle.generate_bundle(repo, fpath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
935 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
936 repo.invalidate() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
937 finalize_one_bundle(repo, bundle) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
938 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
939 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
940 def make_auto_bundler(source_repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
941 reporef = weakref.ref(source_repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
942 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
943 def autobundle(tr): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
944 repo = reporef() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
945 assert repo is not None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
946 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
947 new, __ = auto_bundle_needed_actions(repo, bundles, b"%d_txn" % id(tr)) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
948 for data in new: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
949 start_one_bundle(repo, data) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
950 return None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
951 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
952 return autobundle |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
953 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
954 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
955 def reposetup(ui, repo): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
956 """install the two pieces needed for automatic clonebundle generation |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
957 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
958 - add a "post-close" hook that fires bundling when needed |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
959 - introduce a clone-bundle lock to let multiple processes meddle with the |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
960 state files. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
961 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
962 if not repo.local(): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
963 return |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
964 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
965 class autobundlesrepo(repo.__class__): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
966 def transaction(self, *args, **kwargs): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
967 tr = super(autobundlesrepo, self).transaction(*args, **kwargs) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
968 enabled = repo.ui.configbool( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
969 b'clone-bundles', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
970 b'auto-generate.on-change', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
971 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
972 targets = repo.ui.configlist( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
973 b'clone-bundles', b'auto-generate.formats' |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
974 ) |
50736
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
975 if enabled: |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
976 if not targets: |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
977 repo.ui.warn( |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
978 _( |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
979 b'clone-bundle auto-generate enabled, ' |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
980 b'but no formats specified: disabling generation\n' |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
981 ) |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
982 ) |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
983 else: |
b852c34dc2b8
clonebundles: add warning if auto-generate is enabled without formats
Mathias De Mare <mathias.de_mare@nokia.com>
parents:
50735
diff
changeset
|
984 tr.addpostclose(CAT_POSTCLOSE, make_auto_bundler(self)) |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
985 return tr |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
986 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
987 @localrepo.unfilteredmethod |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
988 def clonebundles_lock(self, wait=True): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
989 '''Lock the repository file related to clone bundles''' |
50928
d718eddf01d9
safehasattr: drop usage in favor of hasattr
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50776
diff
changeset
|
990 if not hasattr(self, '_cb_lock_ref'): |
50686
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
991 self._cb_lock_ref = None |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
992 l = self._currentlock(self._cb_lock_ref) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
993 if l is not None: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
994 l.lock() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
995 return l |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
996 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
997 l = self._lock( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
998 vfs=self.vfs, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
999 lockname=b"clonebundleslock", |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1000 wait=wait, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1001 releasefn=None, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1002 acquirefn=None, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1003 desc=_(b'repository %s') % self.origroot, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1004 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1005 self._cb_lock_ref = weakref.ref(l) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1006 return l |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1007 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1008 repo._wlockfreeprefix.add(AUTO_GEN_FILE) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1009 repo._wlockfreeprefix.add(bundlecaches.CB_MANIFEST_FILE) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1010 repo.__class__ = autobundlesrepo |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1011 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1012 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1013 @command( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1014 b'admin::clone-bundles-refresh', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1015 [ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1016 ( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1017 b'', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1018 b'background', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1019 False, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1020 _(b'start bundle generation in the background'), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1021 ), |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1022 ], |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1023 b'', |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1024 ) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1025 def cmd_admin_clone_bundles_refresh( |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1026 ui, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1027 repo: localrepo.localrepository, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1028 background=False, |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1029 ): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1030 """generate clone bundles according to the configuration |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1031 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1032 This runs the logic for automatic generation, removing outdated bundles and |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1033 generating new ones if necessary. See :hg:`help -e clone-bundles` for |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1034 details about how to configure this feature. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1035 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1036 debug = repo.ui.configbool(b'devel', b'debug.clonebundles') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1037 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1038 op_id = b"%d_acbr" % os.getpid() |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1039 create, delete = auto_bundle_needed_actions(repo, bundles, op_id) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1040 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1041 # if some bundles are scheduled for creation in the background, they will |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1042 # deal with garbage collection too, so no need to synchroniously do it. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1043 # |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1044 # However if no bundles are scheduled for creation, we need to explicitly do |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1045 # it here. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1046 if not (background and create): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1047 # we clean up outdated bundles before generating new ones to keep the |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1048 # last two versions of the bundle around for a while and avoid having to |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1049 # deal with clients that just got served a manifest. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1050 for o in delete: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1051 delete_bundle(repo, o) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1052 update_bundle_list(repo, del_bundles=delete) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1053 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1054 if create: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1055 fpath = repo.vfs.makedirs(b'tmp-bundles') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1056 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1057 if background: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1058 for requested_bundle in create: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1059 start_one_bundle(repo, requested_bundle) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1060 else: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1061 for requested_bundle in create: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1062 if debug: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1063 msg = b'clone-bundles: starting bundle generation: %s\n' |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1064 repo.ui.write(msg % requested_bundle.bundle_type) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1065 fname = requested_bundle.suggested_filename |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1066 fpath = repo.vfs.join(b'tmp-bundles', fname) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1067 generating_bundle = requested_bundle.generating(fpath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1068 update_bundle_list(repo, new_bundles=[generating_bundle]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1069 requested_bundle.generate_bundle(repo, fpath) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1070 result = upload_bundle(repo, generating_bundle) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1071 update_bundle_list(repo, new_bundles=[result]) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1072 update_ondisk_manifest(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1073 cleanup_tmp_bundle(repo, generating_bundle) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1074 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1075 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1076 @command(b'admin::clone-bundles-clear', [], b'') |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1077 def cmd_admin_clone_bundles_clear(ui, repo: localrepo.localrepository): |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1078 """remove existing clone bundle caches |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1079 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1080 See `hg help admin::clone-bundles-refresh` for details on how to regenerate |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1081 them. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1082 |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1083 This command will only affect bundles currently available, it will not |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1084 affect bundles being asynchronously generated. |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1085 """ |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1086 bundles = read_auto_gen(repo) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1087 delete = [b for b in bundles if b.ready] |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1088 for o in delete: |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1089 delete_bundle(repo, o) |
a41eeb877d07
branching: merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
50645
diff
changeset
|
1090 update_bundle_list(repo, del_bundles=delete) |