Mercurial > hg
view mercurial/scmposix.py @ 27895:2d6a89e79b48
scmutil: support background file closing
Closing files that have been appended to is relatively slow on
Windows/NTFS. This makes several Mercurial operations slower on
Windows.
The workaround to this issue is conceptually simple: use multiple
threads for I/O. Unfortunately, Python doesn't scale well to multiple
threads because of the GIL. And, refactoring our code to use threads
everywhere would be a huge undertaking. So, we decide to tackle this
problem by starting small: establishing a thread pool for closing
files.
This patch establishes a mechanism for closing file handles on separate
threads. The coordinator object is basically a queue of file handles to
operate on and a thread pool consuming from the queue.
When files are opened through the VFS layer, the caller can specify
that delay closing is allowed.
A proxy class for file handles has been added. We must use a proxy
because it isn't possible to modify __class__ on built-in types. This
adds some overhead. But as future patches will show, this overhead
is cancelled out by the benefit of closing file handles on background
threads.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 14 Jan 2016 13:34:59 -0800 |
parents | 39087ee88835 |
children | c90a05124fae |
line wrap: on
line source
from __future__ import absolute_import import os import sys from . import ( osutil, ) def _rcfiles(path): rcs = [os.path.join(path, 'hgrc')] rcdir = os.path.join(path, 'hgrc.d') try: rcs.extend([os.path.join(rcdir, f) for f, kind in osutil.listdir(rcdir) if f.endswith(".rc")]) except OSError: pass return rcs def systemrcpath(): path = [] if sys.platform == 'plan9': root = 'lib/mercurial' else: root = 'etc/mercurial' # old mod_python does not set sys.argv if len(getattr(sys, 'argv', [])) > 0: p = os.path.dirname(os.path.dirname(sys.argv[0])) if p != '/': path.extend(_rcfiles(os.path.join(p, root))) path.extend(_rcfiles('/' + root)) return path def userrcpath(): if sys.platform == 'plan9': return [os.environ['home'] + '/lib/hgrc'] else: return [os.path.expanduser('~/.hgrc')]