annotate hgext/largefiles/__init__.py @ 15371:f26ed4ea46d8 stable

largefiles: remove lfutil.createdir, replace calls with util.makedirs
author Hao Lian <hao@fogcreek.com>
date Tue, 25 Oct 2011 11:45:28 -0400
parents b74f74b482d8
children 6266b1b970a5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
1 # Copyright 2009-2010 Gregory P. Ward
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
3 # Copyright 2010-2011 Fog Creek Software
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
4 # Copyright 2010-2011 Unity Technologies
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
5 #
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
8
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
9 '''track large binary files
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
10
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
11 Large binary files tend to be not very compressible, not very
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
12 diffable, and not at all mergeable. Such files are not handled
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
13 efficiently by Mercurial's storage format (revlog), which is based on
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
14 compressed binary deltas; storing large binary files as regular
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
15 Mercurial files wastes bandwidth and disk space and increases
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
16 Mercurial's memory usage. The largefiles extension addresses these
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
17 problems by adding a centralized client-server layer on top of
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
18 Mercurial: largefiles live in a *central store* out on the network
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
19 somewhere, and you only fetch the revisions that you need when you
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
20 need them.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
21
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
22 largefiles works by maintaining a "standin file" in .hglf/ for each
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
23 largefile. The standins are small (41 bytes: an SHA-1 hash plus
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
24 newline) and are tracked by Mercurial. Largefile revisions are
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
25 identified by the SHA-1 hash of their contents, which is written to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
26 the standin. largefiles uses that revision ID to get/put largefile
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
27 revisions from/to the central store. This saves both disk space and
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
28 bandwidth, since you don't need to retrieve all historical revisions
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
29 of large files when you clone or pull.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
30
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
31 To start a new repository or add new large binary files, just add
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
32 --large to your :hg:`add` command. For example::
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
33
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
34 $ dd if=/dev/urandom of=randomdata count=2000
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
35 $ hg add --large randomdata
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
36 $ hg commit -m 'add randomdata as a largefile'
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
37
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
38 When you push a changeset that adds/modifies largefiles to a remote
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
39 repository, its largefile revisions will be uploaded along with it.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
40 Note that the remote Mercurial must also have the largefiles extension
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
41 enabled for this to work.
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
42
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
43 When you pull a changeset that affects largefiles from a remote
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
44 repository, Mercurial behaves as normal. However, when you update to
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
45 such a revision, any largefiles needed by that revision are downloaded
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
46 and cached (if they have never been downloaded before). This means
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
47 that network access may be required to update to changesets you have
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
48 not previously updated to.
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
49
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
50 If you already have large files tracked by Mercurial without the
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
51 largefiles extension, you will need to convert your repository in
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
52 order to benefit from largefiles. This is done with the
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
53 :hg:`lfconvert` command::
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
54
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
55 $ hg lfconvert --size 10 oldrepo newrepo
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
56
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
57 In repositories that already have largefiles in them, any new file
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
58 over 10MB will automatically be added as a largefile. To change this
15304
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
59 threshold, set ``largefiles.minsize`` in your Mercurial config file
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
60 to the minimum size in megabytes to track as a largefile, or use the
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
61 --lfsize option to the add command (also in megabytes)::
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
62
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
63 [largefiles]
15304
9aa9d4bb3d88 largefiles: rename config setting 'size' to 'minsize'
Greg Ward <greg@gerg.ca>
parents: 15291
diff changeset
64 minsize = 2
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
65
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
66 $ hg add --lfsize 2
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
67
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
68 The ``largefiles.patterns`` config option allows you to specify a list
15352
b74f74b482d8 largefiles: improve markup in module help text
Martin Geisler <mg@aragost.com>
parents: 15304
diff changeset
69 of filename patterns (see :hg:`help patterns`) that should always be
15230
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
70 tracked as largefiles::
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
71
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
72 [largefiles]
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
73 patterns =
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
74 *.jpg
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
75 re:.*\.(png|bmp)$
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
76 library.zip
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
77 content/audio/*
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
78
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
79 Files that match one of these patterns will be added as largefiles
697289c5d415 largefiles: improve help
Greg Ward <greg@gerg.ca>
parents: 15168
diff changeset
80 regardless of their size.
15168
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
81 '''
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
82
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
83 from mercurial import commands
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
84
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
85 import lfcommands
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
86 import reposetup
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
87 import uisetup
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
88
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
89 reposetup = reposetup.reposetup
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
90 uisetup = uisetup.uisetup
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
91
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
92 commands.norepo += " lfconvert"
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
93
cfccd3bee7b3 hgext: add largefiles extension
various
parents:
diff changeset
94 cmdtable = lfcommands.cmdtable