changeset 24631:2a3f24786d09

largefiles: use the share source as the primary local store (issue4471)

The benefit of retargeting the local store to the share source is that all
shares will always have access to the largefiles any one of them commit, even if
the user cache is deleted (which is documented to be OK to do).  Further, any
push into the source (and now any shares), will likewise make the largefile(s)
visible to all related repositories.

In order to maintain compatibility with existing repos, where the largefiles
would be cached only in the local share, fallback to searching the local share
if it isn't found at the share source.

The unshare command should probably be taught to copy the source store into the
store for the repo being unshared to complete the loop.


This patch changes the test like this:

  @@ -159,6 +159,5 @@
     $ hg share -q src share_dst --config extensions.share=
     $ hg -R share_dst update -r0
     getting changed largefiles
  -  large: largefile $HASH not available from file:///$TESTTMP\share_dst
  -  0 largefiles updated, 0 removed
  +  1 largefiles updated, 0 removed
     1 files updated, 0 files merged, 0 files removed, 0 files unresolved


The issue writeup mentions pushing a largefile from a remote repo to the main
local repo, and the largefile is then not available in any shares.  Since the
push doesn't cache the largefile in $USERCACHE, the trashed $USERCACHE in this
test is equivalent.

--- a/hgext/largefiles/lfutil.py	Sat Apr 04 19:34:36 2015 -0400
+++ b/hgext/largefiles/lfutil.py	Sat Apr 04 19:06:43 2015 -0400
@@ -82,9 +82,10 @@
     return path and os.path.exists(path)
 
 def findfile(repo, hash):
-    if instore(repo, hash):
+    path, exists = findstorepath(repo, hash)
+    if exists:
         repo.ui.note(_('found %s in store\n') % hash)
-        return storepath(repo, hash)
+        return path
     elif inusercache(repo.ui, hash):
         repo.ui.note(_('found %s in system cache\n') % hash)
         path = storepath(repo, hash)
@@ -164,10 +165,12 @@
             for f in repo[rev].walk(matcher)
             if rev is not None or repo.dirstate[f] != '?']
 
-def instore(repo, hash):
-    return os.path.exists(storepath(repo, hash))
+def instore(repo, hash, forcelocal=False):
+    return os.path.exists(storepath(repo, hash, forcelocal))
 
-def storepath(repo, hash):
+def storepath(repo, hash, forcelocal=False):
+    if not forcelocal and repo.shared():
+        return repo.vfs.reljoin(repo.sharedpath, longname, hash)
     return repo.join(longname, hash)
 
 def findstorepath(repo, hash):
@@ -175,7 +178,17 @@
     hash.  If the file is not found, its path in the primary store is returned.
     The return value is a tuple of (path, exists(path)).
     '''
-    return (storepath(repo, hash), instore(repo, hash))
+    # For shared repos, the primary store is in the share source.  But for
+    # backward compatibility, force a lookup in the local store if it wasn't
+    # found in the share source.
+    path = storepath(repo, hash, False)
+
+    if instore(repo, hash):
+        return (path, True)
+    elif repo.shared() and instore(repo, hash, True):
+        return storepath(repo, hash, True)
+
+    return (path, False)
 
 def copyfromcache(repo, hash, filename):
     '''Copy the specified largefile from the repo or system cache to

--- a/tests/test-largefiles-cache.t	Sat Apr 04 19:34:36 2015 -0400
+++ b/tests/test-largefiles-cache.t	Sat Apr 04 19:06:43 2015 -0400
@@ -153,3 +153,29 @@
   ENOENT: * (glob)
   not removing z: file is already untracked
   [1]
+
+Largefiles are accessible from the share's store
+  $ cd ..
+  $ hg share -q src share_dst --config extensions.share=
+  $ hg -R share_dst update -r0
+  getting changed largefiles
+  1 largefiles updated, 0 removed
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+  $ echo modified > share_dst/large
+  $ hg -R share_dst ci -m modified
+  created new head
+
+Only dirstate is in the local store for the share, and the largefile is in the
+share source's local store.  Avoid the extra largefiles added in the unix
+conditional above.
+  $ hash=`hg -R share_dst cat share_dst/.hglf/large`
+  $ echo $hash
+  e2fb5f2139d086ded2cb600d5a91a196e76bf020
+
+  $ find share_dst/.hg/largefiles/* | sort
+  share_dst/.hg/largefiles/dirstate
+
+  $ find src/.hg/largefiles/* | egrep "(dirstate|$hash)" | sort
+  src/.hg/largefiles/dirstate
+  src/.hg/largefiles/e2fb5f2139d086ded2cb600d5a91a196e76bf020