annotate hgext/largefiles/__init__.py @ 33408:e3867c712d51

bundle2: automatically add 'targetphase' parameter in writenewbundle If we are bundling secret changeset and the bundle will contain phase, we request the changegroup to be applied as secret. It will be useful for next patch as we are now sure that secrets changesets are applied as secret and not applied as draft then forced to secret.
author Boris Feld <boris.feld@octobus.net>
date Tue, 11 Jul 2017 05:12:03 +0200
parents bd872f64a8ba
children 1775f93da25c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
1 # Copyright 2009-2010 Gregory P. Ward
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
3 # Copyright 2010-2011 Fog Creek Software
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
4 # Copyright 2010-2011 Unity Technologies
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
5 #
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
8
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
9 '''track large binary files
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
10
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
11 Large binary files tend to be not very compressible, not very
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
12 diffable, and not at all mergeable. Such files are not handled
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
13 efficiently by Mercurial's storage format (revlog), which is based on
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
14 compressed binary deltas; storing large binary files as regular
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
15 Mercurial files wastes bandwidth and disk space and increases
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
16 Mercurial's memory usage. The largefiles extension addresses these
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
17 problems by adding a centralized client-server layer on top of
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
18 Mercurial: largefiles live in a *central store* out on the network
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
19 somewhere, and you only fetch the revisions that you need when you
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
20 need them.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
21
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
22 largefiles works by maintaining a "standin file" in .hglf/ for each
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
23 largefile. The standins are small (41 bytes: an SHA-1 hash plus
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
24 newline) and are tracked by Mercurial. Largefile revisions are
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
25 identified by the SHA-1 hash of their contents, which is written to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
26 the standin. largefiles uses that revision ID to get/put largefile
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
27 revisions from/to the central store. This saves both disk space and
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
28 bandwidth, since you don't need to retrieve all historical revisions
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
29 of large files when you clone or pull.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
30
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
31 To start a new repository or add new large binary files, just add
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
32 --large to your :hg:`add` command. For example::
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
33
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
34 $ dd if=/dev/urandom of=randomdata count=2000
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
35 $ hg add --large randomdata
28798
c065e0ec13d1 largefiles: use double quotes for arguments
timeless <timeless@mozdev.org>
parents: 28394
diff changeset
36 $ hg commit -m "add randomdata as a largefile"
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
37
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
38 When you push a changeset that adds/modifies largefiles to a remote
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
39 repository, its largefile revisions will be uploaded along with it.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
40 Note that the remote Mercurial must also have the largefiles extension
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
41 enabled for this to work.
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
42
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
43 When you pull a changeset that affects largefiles from a remote
18975
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
44 repository, the largefiles for the changeset will by default not be
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
45 pulled down. However, when you update to such a revision, any
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
46 largefiles needed by that revision are downloaded and cached (if
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
47 they have never been downloaded before). One way to pull largefiles
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
48 when pulling is thus to use --update, which will update your working
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
49 copy to the latest pulled revision (and thereby downloading any new
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
50 largefiles).
18704
d69585a5c5c0 largefiles: don't cache largefiles for pulled heads by default
Na'Tosha Bard <natosha@unity3d.com>
parents: 18599
diff changeset
51
18976
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18975
diff changeset
52 If you want to pull largefiles you don't need for update yet, then
18978
8abaadab9abb largefiles: introduce pull --lfrev option
Mads Kiilerich <madski@unity3d.com>
parents: 18976
diff changeset
53 you can use pull with the `--lfrev` option or the :hg:`lfpull` command.
18976
6734951e2d24 largefiles: introduce lfpull command for pulling missing largefiles
Mads Kiilerich <madski@unity3d.com>
parents: 18975
diff changeset
54
19071
64ea454e7d76 largefiles: fix typos in documentation
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 18980
diff changeset
55 If you know you are pulling from a non-default location and want to
64ea454e7d76 largefiles: fix typos in documentation
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 18980
diff changeset
56 download all the largefiles that correspond to the new changesets at
18979
1176832fc757 largefiles: introduce pulled() revset expression for use in --lfrev
Mads Kiilerich <madski@unity3d.com>
parents: 18978
diff changeset
57 the same time, then you can pull with `--lfrev "pulled()"`.
1176832fc757 largefiles: introduce pulled() revset expression for use in --lfrev
Mads Kiilerich <madski@unity3d.com>
parents: 18978
diff changeset
58
18975
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
59 If you just want to ensure that you will have the largefiles needed to
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
60 merge or rebase with new heads that you are pulling, then you can pull
18979
1176832fc757 largefiles: introduce pulled() revset expression for use in --lfrev
Mads Kiilerich <madski@unity3d.com>
parents: 18978
diff changeset
61 with `--lfrev "head(pulled())"` flag to pre-emptively download any largefiles
18704
d69585a5c5c0 largefiles: don't cache largefiles for pulled heads by default
Na'Tosha Bard <natosha@unity3d.com>
parents: 18599
diff changeset
62 that are new in the heads you are pulling.
18599
5cd1dbf4c5d2 largefiles: document behavior of caching largefiles for new heads
Na'Tosha Bard <natosha@unity3d.com>
parents: 17233
diff changeset
63
18975
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
64 Keep in mind that network access may now be required to update to
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
65 changesets that you have not previously updated to. The nature of the
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
66 largefiles extension means that updating is no longer guaranteed to
aa8205a9f51a largefiles: update help
Mads Kiilerich <madski@unity3d.com>
parents: 18704
diff changeset
67 be a local-only operation.
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
68
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
69 If you already have large files tracked by Mercurial without the
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
70 largefiles extension, you will need to convert your repository in
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
71 order to benefit from largefiles. This is done with the
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
72 :hg:`lfconvert` command::
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
73
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
74 $ hg lfconvert --size 10 oldrepo newrepo
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
75
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
76 In repositories that already have largefiles in them, any new file
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
77 over 10MB will automatically be added as a largefile. To change this
15304
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
78 threshold, set ``largefiles.minsize`` in your Mercurial config file
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
79 to the minimum size in megabytes to track as a largefile, or use the
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
80 --lfsize option to the add command (also in megabytes)::
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
81
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
82 [largefiles]
15304
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
83 minsize = 2
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
84
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
85 $ hg add --lfsize 2
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
86
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
87 The ``largefiles.patterns`` config option allows you to specify a list
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
88 of filename patterns (see :hg:`help patterns`) that should always be
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
89 tracked as largefiles::
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
90
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
91 [largefiles]
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
92 patterns =
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
93 *.jpg
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
94 re:.*\.(png|bmp)$
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
95 library.zip
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
96 content/audio/*
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
97
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
98 Files that match one of these patterns will be added as largefiles
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
99 regardless of their size.
15743
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
100
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
101 The ``largefiles.minsize`` and ``largefiles.patterns`` config options
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
102 will be ignored for any repositories not already containing a
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
103 largefile. To add the first largefile to a repository, you must
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
104 explicitly do so with the --large flag passed to the :hg:`add`
6266b1b970a5 largefiles: clarify help when options are ignored until first add is done
Michal Sznajder <michalsznajder@gmail.com>
parents: 15352
diff changeset
105 command.
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
106 '''
29306
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
107 from __future__ import absolute_import
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
108
29306
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
109 from mercurial import (
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
110 hg,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
111 localrepo,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
112 )
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
113
29306
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
114 from . import (
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
115 lfcommands,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
116 overrides,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
117 proto,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
118 reposetup,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
119 uisetup as uisetupmod,
83cecc0b991f py3: make largefiles/__init__.py use absolute_import
liscju <piotr.listkiewicz@gmail.com>
parents: 28798
diff changeset
120 )
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
121
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29306
diff changeset
122 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21770
diff changeset
123 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21770
diff changeset
124 # be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21770
diff changeset
125 # leave the attribute unspecified.
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29306
diff changeset
126 testedwith = 'ships-with-hg-core'
17233
acea82757d8a largefiles: mark as a first party extension
Matt Harbison <matt_harbison@yahoo.com>
parents: 15743
diff changeset
127
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
128 reposetup = reposetup.reposetup
19779
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
129
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
130 def featuresetup(ui, supported):
19928
d1ac3790e10a localrepo: invoke only feature setup functions for enabled extensions
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19779
diff changeset
131 # don't die on seeing a repo with the largefiles requirement
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
132 supported |= {'largefiles'}
19779
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
133
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
134 def uisetup(ui):
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
135 localrepo.localrepository.featuresetupfuncs.add(featuresetup)
20858
bc56ec9e64df hg: introduce "wirepeersetupfuncs" to setup wire peer by extensions (issue4109)
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19928
diff changeset
136 hg.wirepeersetupfuncs.append(proto.wirereposetup)
19779
fb6e87d93948 largefiles: setup "largefiles" feature in each repositories individually
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 19071
diff changeset
137 uisetupmod.uisetup(ui)
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
138
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
139 cmdtable = lfcommands.cmdtable
28394
dcb4209bd30d revset: replace extpredicate by revsetpredicate of registrar
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25186
diff changeset
140 revsetpredicate = overrides.revsetpredicate