view doc/runrst @ 27895:2d6a89e79b48

scmutil: support background file closing

Closing files that have been appended to is relatively slow on
Windows/NTFS. This makes several Mercurial operations slower on
Windows.

The workaround to this issue is conceptually simple: use multiple
threads for I/O. Unfortunately, Python doesn't scale well to multiple
threads because of the GIL. And, refactoring our code to use threads
everywhere would be a huge undertaking. So, we decide to tackle this
problem by starting small: establishing a thread pool for closing
files.

This patch establishes a mechanism for closing file handles on separate
threads. The coordinator object is basically a queue of file handles to
operate on and a thread pool consuming from the queue.

When files are opened through the VFS layer, the caller can specify
that delay closing is allowed.

A proxy class for file handles has been added. We must use a proxy
because it isn't possible to modify __class__ on built-in types. This
adds some overhead. But as future patches will show, this overhead
is cancelled out by the benefit of closing file handles on background
threads.

#!/usr/bin/env python
#
# runrst - register custom roles and run correct writer
#
# Copyright 2010 Matt Mackall <[e-mail]> and others
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

"""usage: %s WRITER args...

where WRITER is the name of a Docutils writer such as 'html' or 'manpage'
"""

import sys
try:
    from docutils.parsers.rst import roles
    from docutils.core import publish_cmdline
    from docutils import nodes, utils
except ImportError:
    sys.stderr.write("abort: couldn't generate documentation: docutils "
                     "module is missing\n")
    sys.stderr.write("please install python-docutils or see "
                     "http://docutils.sourceforge.net/\n")
    sys.exit(-1)

def role_hg(name, rawtext, text, lineno, inliner,
            options={}, content=[]):
    text = "hg " + utils.unescape(text)
    linktext = nodes.literal(rawtext, text)
    parts = text.split()
    cmd, args = parts[1], parts[2:]
    if cmd == 'help' and args:
        cmd = args[0] # link to 'dates' for 'hg help dates'
    node = nodes.reference(rawtext, '', linktext,
                           refuri="hg.1.html#%s" % cmd)
    return [node], []

roles.register_local_role("hg", role_hg)

if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.stderr.write(__doc__ % sys.argv[0])
        sys.exit(1)

    writer = sys.argv[1]
    del sys.argv[1]

    publish_cmdline(writer_name=writer)