narrow: move remaining narrow-limited dirstate walks to core

narrow: move remaining narrow-limited dirstate walks to core

In most places we now filter at a higher level (the context object),
but there are few places that relied on the dirstate walk to be
filtered by the narrowspec. The important cases are those used by `hg
add` and `hg addremove`. This patch updates them to pass in a matcher
instead of relying on the dirstate to do the filtering. The dirstate
filtering is also dropped in narrowdirstate.py.

Not always filtering in the dirstate should be useful for a future `hg
status --include-outside-narrow` option.

These places now end up doing an unrestricted dirstate walk after this
patch:

 * debugfileset
 * perfwalk
 * sparse (but restricted to sparse config)
 * largefiles

I'll let anyone who cares about these cases adapt them to work with
narrow if necessary.

Differential Revision: https://phab.mercurial-scm.org/D4901

--- a/hgext/narrow/TODO.rst	Mon Oct 01 10:11:00 2018 -0700
+++ b/hgext/narrow/TODO.rst	Mon Oct 01 14:31:15 2018 -0700
@@ -14,3 +14,11 @@
 
 narrowrepo.setnarrowpats() or narrowspec.save() need to make sure
 they're holding the wlock.
+
+The follinwg places do an unrestricted dirstate walk (including files outside the
+narrowspec). Some of them should perhaps not do that.
+
+ * debugfileset
+ * perfwalk
+ * sparse (but restricted to sparse config)
+ * largefiles

--- a/hgext/narrow/narrowdirstate.py	Mon Oct 01 10:11:00 2018 -0700
+++ b/hgext/narrow/narrowdirstate.py	Mon Oct 01 14:31:15 2018 -0700
@@ -26,13 +26,6 @@
         return _wrapper
 
     class narrowdirstate(dirstate.__class__):
-        def walk(self, match, subrepos, unknown, ignored, full=True,
-                 narrowonly=True):
-            if narrowonly:
-                match = repo.narrowmatch(match, includeexact=True)
-            return super(narrowdirstate, self).walk(match, subrepos, unknown,
-                                                    ignored, full)
-
         # Prevent adding/editing/copying/deleting files that are outside the
         # sparse checkout
         @_editfunc

--- a/mercurial/cmdutil.py	Mon Oct 01 10:11:00 2018 -0700
+++ b/mercurial/cmdutil.py	Mon Oct 01 14:31:15 2018 -0700
@@ -2044,6 +2044,7 @@
     if abort or warn:
         cca = scmutil.casecollisionauditor(ui, abort, repo.dirstate)
 
+    match = repo.narrowmatch(match, includeexact=True)
     badmatch = matchmod.badmatch(match, badfn)
     dirstate = repo.dirstate
     # We don't want to just call wctx.walk here, since it would return a lot of

--- a/mercurial/scmutil.py	Mon Oct 01 10:11:00 2018 -0700
+++ b/mercurial/scmutil.py	Mon Oct 01 14:31:15 2018 -0700
@@ -1098,6 +1098,7 @@
 
     ctx = repo[None]
     dirstate = repo.dirstate
+    matcher = repo.narrowmatch(matcher, includeexact=True)
     walkresults = dirstate.walk(matcher, subrepos=sorted(ctx.substate),
                                 unknown=True, ignored=False, full=False)
     for abs, st in walkresults.iteritems():