author | Matt Mackall <mpm@selenic.com> |
Fri, 21 Oct 2011 16:52:16 -0500 | |
branch | stable |
changeset 15333 | f37b71fec602 |
parent 15315 | ca51a5dd5d0b |
permissions | -rw-r--r-- |
15168 | 1 |
= largefiles - manage large binary files = |
2 |
This extension is based off of Greg Ward's bfiles extension which can be found |
|
3 |
at http://mercurial.selenic.com/wiki/BfilesExtension. |
|
4 |
||
5 |
== The largefile store == |
|
6 |
||
7 |
largefile stores are, in the typical use case, centralized servers that have |
|
8 |
every past revision of a given binary file. Each largefile is identified by |
|
9 |
its sha1 hash, and all interactions with the store take one of the following |
|
10 |
forms. |
|
11 |
||
12 |
-Download a bfile with this hash |
|
13 |
-Upload a bfile with this hash |
|
14 |
-Check if the store has a bfile with this hash |
|
15 |
||
16 |
largefiles stores can take one of two forms: |
|
17 |
||
18 |
-Directories on a network file share |
|
19 |
-Mercurial wireproto servers, either via ssh or http (hgweb) |
|
20 |
||
21 |
== The Local Repository == |
|
22 |
||
15315
ca51a5dd5d0b
largefiles: fix documentation to match desired behavior
Benjamin Pollack <benjamin@bitquabit.com>
parents:
15168
diff
changeset
|
23 |
The local repository has a largefile store in .hg/largefiles which holds a |
15168 | 24 |
subset of the largefiles needed. On a clone only the largefiles at tip are |
25 |
downloaded. When largefiles are downloaded from the central store, a copy is |
|
26 |
saved in this store. |
|
27 |
||
15315
ca51a5dd5d0b
largefiles: fix documentation to match desired behavior
Benjamin Pollack <benjamin@bitquabit.com>
parents:
15168
diff
changeset
|
28 |
== The User Cache == |
15168 | 29 |
|
15315
ca51a5dd5d0b
largefiles: fix documentation to match desired behavior
Benjamin Pollack <benjamin@bitquabit.com>
parents:
15168
diff
changeset
|
30 |
largefiles in a local repository store are hardlinked to files in the user |
ca51a5dd5d0b
largefiles: fix documentation to match desired behavior
Benjamin Pollack <benjamin@bitquabit.com>
parents:
15168
diff
changeset
|
31 |
cache. Before a file is downloaded we check if it is in the global cache, |
ca51a5dd5d0b
largefiles: fix documentation to match desired behavior
Benjamin Pollack <benjamin@bitquabit.com>
parents:
15168
diff
changeset
|
32 |
hard-linking to the local store if we find it. |
15168 | 33 |
|
34 |
== Implementation Details == |
|
35 |
||
36 |
Each largefile has a standin which is in .hglf. The standin is tracked by |
|
37 |
Mercurial. The standin contains the SHA1 hash of the largefile. When a |
|
38 |
largefile is added/removed/copied/renamed/etc the same operation is applied to |
|
39 |
the standin. Thus the history of the standin is the history of the largefile. |
|
40 |
||
41 |
For performance reasons, the contents of a standin are only updated before a |
|
42 |
commit. Standins are added/removed/copied/renamed from add/remove/copy/rename |
|
43 |
Mercurial commands but their contents will not be updated. The contents of a |
|
44 |
standin will always be the hash of the largefile as of the last commit. To |
|
45 |
support some commands (revert) some standins are temporarily updated but will |
|
46 |
be changed back after the command is finished. |
|
47 |
||
48 |
A Mercurial dirstate object tracks the state of the largefiles. The dirstate |
|
49 |
uses the last modified time and current size to detect if a file has changed |
|
50 |
(without reading the entire contents of the file). |