verify: check the subrepository references in .hgsubstate

verify: check the subrepository references in .hgsubstate

While hopefully atypical, there are reasons that a subrepository revision can be
lost that aren't covered by corruption of the .hgsubstate revlog.  Such things
can happen when a subrepo is amended, stripped or simply isn't pulled from
upstream because the parent repo revision wasn't updated yet.  There's no way to
know if it is an error, but this will find potential problems sooner than when
some random revision is updated.

Until recently, convert made no attempt at rewriting the .hgsubstate file.  The
impetuous for this is to verify the conversion of some repositories, and this is
orders of magnitude faster than a bash script from 0..tip that does an
'hg update -C $rev'.  But it is equally useful to determine if everything has
been pulled down before taking a thumb drive on the go.

It feels somewhat wrong to leave this out of verifymod (mostly because the file
is already read in there, and the final summary is printed before the subrepos
are checked).  But verifymod looks very low level, so importing subrepo stuff
there seems more wrong.

--- a/mercurial/hg.py	Sun Jun 14 22:04:17 2015 -0400
+++ b/mercurial/hg.py	Tue Jun 16 16:15:15 2015 -0400
@@ -664,7 +664,28 @@
 
 def verify(repo):
     """verify the consistency of a repository"""
-    return verifymod.verify(repo)
+    ret = verifymod.verify(repo)
+
+    # Broken subrepo references in hidden csets don't seem worth worrying about,
+    # since they can't be pushed/pulled, and --hidden can be used if they are a
+    # concern.
+
+    # pathto() is needed for -R case
+    revs = repo.revs("filelog(%s)",
+                     util.pathto(repo.root, repo.getcwd(), '.hgsubstate'))
+
+    if revs:
+        repo.ui.status(_('checking subrepo links\n'))
+        for rev in revs:
+            ctx = repo[rev]
+            try:
+                for subpath in ctx.substate:
+                    ret = ctx.sub(subpath).verify() or ret
+            except Exception:
+                repo.ui.warn(_('.hgsubstate is corrupt in revision %s\n') %
+                             node.short(ctx.node()))
+
+    return ret
 
 def remoteui(src, opts):
     'build a remote ui from ui or repo and opts'

--- a/mercurial/subrepo.py	Sun Jun 14 22:04:17 2015 -0400
+++ b/mercurial/subrepo.py	Tue Jun 16 16:15:15 2015 -0400
@@ -572,6 +572,12 @@
     def shortid(self, revid):
         return revid
 
+    def verify(self):
+        '''verify the integrity of the repository.  Return 0 on success or
+        warning, 1 on any error.
+        '''
+        return 0
+
     @propertycache
     def wvfs(self):
         """return vfs to access the working directory of this subrepository
@@ -1011,6 +1017,24 @@
     def shortid(self, revid):
         return revid[:12]
 
+    def verify(self):
+        try:
+            rev = self._state[1]
+            ctx = self._repo.unfiltered()[rev]
+            if ctx.hidden():
+                # Since hidden revisions aren't pushed/pulled, it seems worth an
+                # explicit warning.
+                ui = self._repo.ui
+                ui.warn(_("subrepo '%s' is hidden in revision %s\n") %
+                        (self._relpath, node.short(self._ctx.node())))
+            return 0
+        except error.RepoLookupError:
+            # A missing subrepo revision may be a case of needing to pull it, so
+            # don't treat this as an error.
+            self._repo.ui.warn(_("subrepo '%s' not found in revision %s\n") %
+                               (self._relpath, node.short(self._ctx.node())))
+            return 0
+
     @propertycache
     def wvfs(self):
         """return own wvfs for efficiency and consitency

--- a/tests/test-static-http.t	Sun Jun 14 22:04:17 2015 -0400
+++ b/tests/test-static-http.t	Tue Jun 16 16:15:15 2015 -0400
@@ -129,6 +129,7 @@
   crosschecking files in changesets and manifests
   checking files
   3 files, 1 changesets, 3 total revisions
+  checking subrepo links
   $ cat a
   a
   $ hg paths

--- a/tests/test-subrepo-missing.t	Sun Jun 14 22:04:17 2015 -0400
+++ b/tests/test-subrepo-missing.t	Tue Jun 16 16:15:15 2015 -0400
@@ -106,4 +106,19 @@
   $ hg --hidden cat subrepo/a
   foo
 
+verify will warn if locked-in subrepo revisions are hidden or missing
+
+  $ hg ci -m "amended subrepo (again)"
+  $ hg --config extensions.strip= --hidden strip -R subrepo -qr 'tip'
+  $ hg verify
+  checking changesets
+  checking manifests
+  crosschecking files in changesets and manifests
+  checking files
+  2 files, 5 changesets, 5 total revisions
+  checking subrepo links
+  subrepo 'subrepo' is hidden in revision a66de08943b6
+  subrepo 'subrepo' is hidden in revision 674d05939c1e
+  subrepo 'subrepo' not found in revision a7d05d9055a4
+
   $ cd ..