largefiles: use the share source as the primary local store (issue4471)
authorMatt Harbison <matt_harbison@yahoo.com>
Sat, 04 Apr 2015 19:06:43 -0400
changeset 24631 2a3f24786d09
parent 24630 c082a4756ed7
child 24632 b2fb1403994e
largefiles: use the share source as the primary local store (issue4471) The benefit of retargeting the local store to the share source is that all shares will always have access to the largefiles any one of them commit, even if the user cache is deleted (which is documented to be OK to do). Further, any push into the source (and now any shares), will likewise make the largefile(s) visible to all related repositories. In order to maintain compatibility with existing repos, where the largefiles would be cached only in the local share, fallback to searching the local share if it isn't found at the share source. The unshare command should probably be taught to copy the source store into the store for the repo being unshared to complete the loop. This patch changes the test like this: @@ -159,6 +159,5 @@ $ hg share -q src share_dst --config extensions.share= $ hg -R share_dst update -r0 getting changed largefiles - large: largefile $HASH not available from file:///$TESTTMP\share_dst - 0 largefiles updated, 0 removed + 1 largefiles updated, 0 removed 1 files updated, 0 files merged, 0 files removed, 0 files unresolved The issue writeup mentions pushing a largefile from a remote repo to the main local repo, and the largefile is then not available in any shares. Since the push doesn't cache the largefile in $USERCACHE, the trashed $USERCACHE in this test is equivalent.
hgext/largefiles/lfutil.py
tests/test-largefiles-cache.t
--- a/hgext/largefiles/lfutil.py	Sat Apr 04 19:34:36 2015 -0400
+++ b/hgext/largefiles/lfutil.py	Sat Apr 04 19:06:43 2015 -0400
@@ -82,9 +82,10 @@
     return path and os.path.exists(path)
 
 def findfile(repo, hash):
-    if instore(repo, hash):
+    path, exists = findstorepath(repo, hash)
+    if exists:
         repo.ui.note(_('found %s in store\n') % hash)
-        return storepath(repo, hash)
+        return path
     elif inusercache(repo.ui, hash):
         repo.ui.note(_('found %s in system cache\n') % hash)
         path = storepath(repo, hash)
@@ -164,10 +165,12 @@
             for f in repo[rev].walk(matcher)
             if rev is not None or repo.dirstate[f] != '?']
 
-def instore(repo, hash):
-    return os.path.exists(storepath(repo, hash))
+def instore(repo, hash, forcelocal=False):
+    return os.path.exists(storepath(repo, hash, forcelocal))
 
-def storepath(repo, hash):
+def storepath(repo, hash, forcelocal=False):
+    if not forcelocal and repo.shared():
+        return repo.vfs.reljoin(repo.sharedpath, longname, hash)
     return repo.join(longname, hash)
 
 def findstorepath(repo, hash):
@@ -175,7 +178,17 @@
     hash.  If the file is not found, its path in the primary store is returned.
     The return value is a tuple of (path, exists(path)).
     '''
-    return (storepath(repo, hash), instore(repo, hash))
+    # For shared repos, the primary store is in the share source.  But for
+    # backward compatibility, force a lookup in the local store if it wasn't
+    # found in the share source.
+    path = storepath(repo, hash, False)
+
+    if instore(repo, hash):
+        return (path, True)
+    elif repo.shared() and instore(repo, hash, True):
+        return storepath(repo, hash, True)
+
+    return (path, False)
 
 def copyfromcache(repo, hash, filename):
     '''Copy the specified largefile from the repo or system cache to
--- a/tests/test-largefiles-cache.t	Sat Apr 04 19:34:36 2015 -0400
+++ b/tests/test-largefiles-cache.t	Sat Apr 04 19:06:43 2015 -0400
@@ -153,3 +153,29 @@
   ENOENT: * (glob)
   not removing z: file is already untracked
   [1]
+
+Largefiles are accessible from the share's store
+  $ cd ..
+  $ hg share -q src share_dst --config extensions.share=
+  $ hg -R share_dst update -r0
+  getting changed largefiles
+  1 largefiles updated, 0 removed
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+  $ echo modified > share_dst/large
+  $ hg -R share_dst ci -m modified
+  created new head
+
+Only dirstate is in the local store for the share, and the largefile is in the
+share source's local store.  Avoid the extra largefiles added in the unix
+conditional above.
+  $ hash=`hg -R share_dst cat share_dst/.hglf/large`
+  $ echo $hash
+  e2fb5f2139d086ded2cb600d5a91a196e76bf020
+
+  $ find share_dst/.hg/largefiles/* | sort
+  share_dst/.hg/largefiles/dirstate
+
+  $ find src/.hg/largefiles/* | egrep "(dirstate|$hash)" | sort
+  src/.hg/largefiles/dirstate
+  src/.hg/largefiles/e2fb5f2139d086ded2cb600d5a91a196e76bf020