Thu, 24 Nov 2016 01:15:34 +0000 worker: use os._exit for posix worker in all cases
Jun Wu <quark@fb.com> [Thu, 24 Nov 2016 01:15:34 +0000] rev 30521
worker: use os._exit for posix worker in all cases Like commandserver, the worker should never run other resource cleanup logic. Previously this is not true for workers if they have exceptions other than KeyboardInterrupt. This actually caused a real-world deadlock with remotefilelog: 1. remotefilelog/fileserverclient creates a sshpeer. pipei/o/e get created. 2. worker inherits that sshpeer's pipei/o/e. 3. worker runs sshpeer.cleanup (only happens without os._exit) 4. worker closes pipeo/i, which will normally make the sshpeer read EOF from its stdin and exit. But the master process still have pipeo, so no EOF. 5. worker reads pipee (stderr of sshpeer), which never completes because the ssh process does not exit, does not close its stderr. 6. master waits for all workers, which never completes because they never complete sshpeer.cleanup. This could also be addressed by closing these fds after fork, which is not easy because Python 2.x does not have an official "afterfork" hook. Hacking os.fork is also ugly. Besides, sshpeer is probably not the only troublemarker. The patch changes _posixworker so all its code paths will use os._exit to avoid running unwanted resource clean-ups.
Thu, 24 Nov 2016 00:48:40 +0000 dispatch: move part of callcatch to scmutil
Jun Wu <quark@fb.com> [Thu, 24 Nov 2016 00:48:40 +0000] rev 30520
dispatch: move part of callcatch to scmutil Per discussion at 39149b6036e6 [1], we need "callcatch" in worker.py. Move it to scmutil.py to avoid cycles. Note that dispatch's callcatch handles some additional high-level exceptions related to config parsing, and commands. Moving them to scmutil will make scmutil depend on "commands" or require "_formatparse" and "_getsimilar" (and "difflib") to be moved as well. In the worker use-case, it is forked when config and commands are fully loaded. So it should not care about those exceptions. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-August/087116.html
(0) -30000 -10000 -3000 -1000 -300 -100 -30 -10 -2 +2 +10 +30 +100 +300 +1000 +3000 +10000 tip