remotefilelog: add a developer option to wait for background processes stable
authorPierre-Yves David <pierre-yves.david@octobus.net>
Mon, 09 Dec 2019 09:54:27 +0100
branchstable
changeset 43457 63bb6dc62f24
parent 43456 15a6c6783060
child 43458 88ab1295db4c
remotefilelog: add a developer option to wait for background processes In order to block the main command on the subprocess exiting, we ensure the repo's ui object will call the subprocess.wait() method to ensure the top-level hg process doesn't exit until all background processes have also done so. Currently, in the tests, most operation spawning background process as followed by commands waiting for these operations to complete. However this waiting is racy. First because it seems like we can start waiting before the background operation actually start, in which case it is prematurely detected as "done". Second, because some commands may spawn multiple background operation for the same operation (eg: rebase can apparently trigger multiple prefetch). The current approach could be updated to maybe handle the first issue, but the second one will never be properly handled. In most case, we do not care that the bg process keep running after the command end. (Since we explicitly wait for them to end before doing anything else). So we add an option to wait on the background process before exiting the command. We'll put it in use in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D7585
hgext/remotefilelog/__init__.py
hgext/remotefilelog/repack.py
hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/__init__.py	Mon Dec 09 09:53:43 2019 +0100
+++ b/hgext/remotefilelog/__init__.py	Mon Dec 09 09:54:27 2019 +0100
@@ -230,6 +230,7 @@
 configitem(b'packs', b'maxchainlen', default=1000)
 
 configitem(b'devel', b'remotefilelog.ensurestart', default=False)
+configitem(b'devel', b'remotefilelog.bg-wait', default=False)
 
 #  default TTL limit is 30 days
 _defaultlimit = 60 * 60 * 24 * 30
--- a/hgext/remotefilelog/repack.py	Mon Dec 09 09:53:43 2019 +0100
+++ b/hgext/remotefilelog/repack.py	Mon Dec 09 09:54:27 2019 +0100
@@ -48,7 +48,13 @@
         cmd.append(b'--packsonly')
     repo.ui.warn(msg)
     # We know this command will find a binary, so don't block on it starting.
-    procutil.runbgcommand(cmd, encoding.environ, ensurestart=ensurestart)
+    kwargs = {}
+    if repo.ui.configbool(b'devel', b'remotefilelog.bg-wait'):
+        kwargs['record_wait'] = repo.ui.atexit
+
+    procutil.runbgcommand(
+        cmd, encoding.environ, ensurestart=ensurestart, **kwargs
+    )
 
 
 def fullrepack(repo, options=None):
--- a/hgext/remotefilelog/shallowrepo.py	Mon Dec 09 09:53:43 2019 +0100
+++ b/hgext/remotefilelog/shallowrepo.py	Mon Dec 09 09:54:27 2019 +0100
@@ -232,8 +232,12 @@
                 cmd += [b'-r', revs]
             # We know this command will find a binary, so don't block
             # on it starting.
+            kwargs = {}
+            if repo.ui.configbool(b'devel', b'remotefilelog.bg-wait'):
+                kwargs['record_wait'] = repo.ui.atexit
+
             procutil.runbgcommand(
-                cmd, encoding.environ, ensurestart=ensurestart
+                cmd, encoding.environ, ensurestart=ensurestart, **kwargs
             )
 
         def prefetch(self, revs, base=None, pats=None, opts=None):