view tests/test-walkrepo.py @ 30521:86cd09bc13ba

worker: use os._exit for posix worker in all cases

Like commandserver, the worker should never run other resource cleanup logic.

Previously this is not true for workers if they have exceptions other than
KeyboardInterrupt.

This actually caused a real-world deadlock with remotefilelog:

1. remotefilelog/fileserverclient creates a sshpeer. pipei/o/e get created.
2. worker inherits that sshpeer's pipei/o/e.
3. worker runs sshpeer.cleanup (only happens without os._exit)
4. worker closes pipeo/i, which will normally make the sshpeer read EOF from
   its stdin and exit. But the master process still have pipeo, so no EOF.
5. worker reads pipee (stderr of sshpeer), which never completes because
   the ssh process does not exit, does not close its stderr.
6. master waits for all workers, which never completes because they never
   complete sshpeer.cleanup.

This could also be addressed by closing these fds after fork, which is not
easy because Python 2.x does not have an official "afterfork" hook. Hacking
os.fork is also ugly. Besides, sshpeer is probably not the only troublemarker.

The patch changes _posixworker so all its code paths will use os._exit to
avoid running unwanted resource clean-ups.

from __future__ import absolute_import, print_function

import os

from mercurial import (
    hg,
    scmutil,
    ui as uimod,
    util,
)

chdir = os.chdir
mkdir = os.mkdir
pjoin = os.path.join

walkrepos = scmutil.walkrepos
checklink = util.checklink

u = uimod.ui()
sym = checklink('.')

hg.repository(u, 'top1', create=1)
mkdir('subdir')
chdir('subdir')
hg.repository(u, 'sub1', create=1)
mkdir('subsubdir')
chdir('subsubdir')
hg.repository(u, 'subsub1', create=1)
chdir(os.path.pardir)
if sym:
    os.symlink(os.path.pardir, 'circle')
    os.symlink(pjoin('subsubdir', 'subsub1'), 'subsub1')

def runtest():
    reposet = frozenset(walkrepos('.', followsym=True))
    if sym and (len(reposet) != 3):
        print("reposet = %r" % (reposet,))
        print(("Found %d repositories when I should have found 3"
               % (len(reposet),)))
    if (not sym) and (len(reposet) != 2):
        print("reposet = %r" % (reposet,))
        print(("Found %d repositories when I should have found 2"
               % (len(reposet),)))
    sub1set = frozenset((pjoin('.', 'sub1'),
                         pjoin('.', 'circle', 'subdir', 'sub1')))
    if len(sub1set & reposet) != 1:
        print("sub1set = %r" % (sub1set,))
        print("reposet = %r" % (reposet,))
        print("sub1set and reposet should have exactly one path in common.")
    sub2set = frozenset((pjoin('.', 'subsub1'),
                         pjoin('.', 'subsubdir', 'subsub1')))
    if len(sub2set & reposet) != 1:
        print("sub2set = %r" % (sub2set,))
        print("reposet = %r" % (reposet,))
        print("sub2set and reposet should have exactly one path in common.")
    sub3 = pjoin('.', 'circle', 'top1')
    if sym and sub3 not in reposet:
        print("reposet = %r" % (reposet,))
        print("Symbolic links are supported and %s is not in reposet" % (sub3,))

runtest()
if sym:
    # Simulate not having symlinks.
    del os.path.samestat
    sym = False
    runtest()