changeset 33373:fb320398a21c

dirstate: expose a sparse matcher on dirstate (API)

The sparse extension performs a lot of monkeypatching of dirstate
to make it sparse aware. Essentially, various operations need to
take the active sparse config into account. They do this by obtaining
a matcher representing the sparse config and filtering paths through
it.

The monkeypatching is done by stuffing a reference to a repo on
dirstate and calling sparse.matcher() (which takes a repo instance)
during each function call. The reason this function takes a repo
instance is because resolving the sparse config may require resolving
file contents from filelogs, and that requires a repo. (If the
current sparse config references "profile" files, the contents of
those files from the dirstate's parent revisions is resolved.)

I seem to recall people having strong opinions that the dirstate
object not have a reference to a repo. So copying what the sparse
extension does probably won't fly in core. Plus, the dirstate
modifications shouldn't require a full repo: they only need a matcher.
So there's no good reason to stuff a reference to the repo in
dirstate.

This commit exposes a sparse matcher to dirstate via a property that
when looked up will call a function that eventually calls
sparse.matcher(). The repo instance is bound in a closure, so it
isn't exposed to dirstate.

This approach is functionally similar to what the sparse extension does
today, except it hides the repo instance from dirstate. The approach
is not optimal because we have to call a proxy function and
sparse.matcher() on every property lookup. There is room to cache
the matcher instance in dirstate. After all, the matcher only changes
if the dirstate's parents change or if the sparse config changes. It
feels like we should be able to detect both events and update the
matcher when this occurs. But for now we preserve the existing
semantics so we can move the dirstate sparseness bits into core. Once
in core, refactoring becomes a bit easier since it will be clearer how
all these components interact.

The sparse extension has been updated to use the new property.
Because all references to the repo on dirstate have been removed,
the code for setting it has been removed.

--- a/hgext/largefiles/lfutil.py	Sat Jul 08 15:42:11 2017 -0700
+++ b/hgext/largefiles/lfutil.py	Sat Jul 08 16:18:04 2017 -0700
@@ -26,6 +26,7 @@
     node,
     pycompat,
     scmutil,
+    sparse,
     util,
     vfs as vfsmod,
 )
@@ -147,7 +148,8 @@
     lfstoredir = longname
     opener = vfsmod.vfs(vfs.join(lfstoredir))
     lfdirstate = largefilesdirstate(opener, ui, repo.root,
-                                     repo.dirstate._validate)
+                                    repo.dirstate._validate,
+                                    lambda: sparse.matcher(repo))
 
     # If the largefiles dirstate does not exist, populate and create
     # it. This ensures that we create it on the first meaningful

--- a/hgext/sparse.py	Sat Jul 08 15:42:11 2017 -0700
+++ b/hgext/sparse.py	Sat Jul 08 16:18:04 2017 -0700
@@ -82,7 +82,6 @@
     error,
     extensions,
     hg,
-    localrepo,
     match as matchmod,
     registrar,
     sparse,
@@ -106,13 +105,6 @@
     _setupadd(ui)
     _setupdirstate(ui)
 
-def reposetup(ui, repo):
-    if not util.safehasattr(repo, 'dirstate'):
-        return
-
-    if 'dirstate' in repo._filecache:
-        repo.dirstate.repo = repo
-
 def replacefilecache(cls, propname, replacement):
     """Replace a filecache property with a new class. This allows changing the
     cache invalidation condition."""
@@ -200,13 +192,6 @@
     and to prevent modifications to files outside the checkout.
     """
 
-    def _dirstate(orig, repo):
-        dirstate = orig(repo)
-        dirstate.repo = repo
-        return dirstate
-    extensions.wrapfunction(
-        localrepo.localrepository.dirstate, 'func', _dirstate)
-
     # The atrocity below is needed to wrap dirstate._ignore. It is a cached
     # property, which means normal function wrapping doesn't work.
     class ignorewrapper(object):
@@ -217,10 +202,9 @@
             self.sparsematch = None
 
         def __get__(self, obj, type=None):
-            repo = obj.repo
             origignore = self.orig.__get__(obj)
 
-            sparsematch = sparse.matcher(repo)
+            sparsematch = obj._sparsematcher
             if sparsematch.always():
                 return origignore
 
@@ -241,7 +225,7 @@
 
     # dirstate.rebuild should not add non-matching files
     def _rebuild(orig, self, parent, allfiles, changedfiles=None):
-        matcher = sparse.matcher(self.repo)
+        matcher = self._sparsematcher
         if not matcher.always():
             allfiles = allfiles.matches(matcher)
             if changedfiles:
@@ -262,8 +246,7 @@
              '`hg add -s <file>` to include file directory while adding')
     for func in editfuncs:
         def _wrapper(orig, self, *args):
-            repo = self.repo
-            sparsematch = sparse.matcher(repo)
+            sparsematch = self._sparsematcher
             if not sparsematch.always():
                 for f in args:
                     if (f is not None and not sparsematch(f) and

--- a/mercurial/dirstate.py	Sat Jul 08 15:42:11 2017 -0700
+++ b/mercurial/dirstate.py	Sat Jul 08 16:18:04 2017 -0700
@@ -70,7 +70,7 @@
 
 class dirstate(object):
 
-    def __init__(self, opener, ui, root, validate):
+    def __init__(self, opener, ui, root, validate, sparsematchfn):
         '''Create a new dirstate object.
 
         opener is an open()-like callable that can be used to open the
@@ -80,6 +80,7 @@
         self._opener = opener
         self._validate = validate
         self._root = root
+        self._sparsematchfn = sparsematchfn
         # ntpath.join(root, '') of Python 2.7.9 does not add sep if root is
         # UNC path pointing to root share (issue4557)
         self._rootdir = pathutil.normasprefix(root)
@@ -197,6 +198,19 @@
             f[normcase(name)] = name
         return f
 
+    @property
+    def _sparsematcher(self):
+        """The matcher for the sparse checkout.
+
+        The working directory may not include every file from a manifest. The
+        matcher obtained by this property will match a path if it is to be
+        included in the working directory.
+        """
+        # TODO there is potential to cache this property. For now, the matcher
+        # is resolved on every access. (But the called function does use a
+        # cache to keep the lookup fast.)
+        return self._sparsematchfn()
+
     @repocache('branch')
     def _branch(self):
         try:

--- a/mercurial/localrepo.py	Sat Jul 08 15:42:11 2017 -0700
+++ b/mercurial/localrepo.py	Sat Jul 08 16:18:04 2017 -0700
@@ -53,6 +53,7 @@
     revset,
     revsetlang,
     scmutil,
+    sparse,
     store,
     subrepo,
     tags as tagsmod,
@@ -570,8 +571,10 @@
 
     @repofilecache('dirstate')
     def dirstate(self):
+        sparsematchfn = lambda: sparse.matcher(self)
+
         return dirstate.dirstate(self.vfs, self.ui, self.root,
-                                 self._dirstatevalidate)
+                                 self._dirstatevalidate, sparsematchfn)
 
     def _dirstatevalidate(self, node):
         try: