Mercurial > hg
diff mercurial/help/config.txt @ 27895:2d6a89e79b48
scmutil: support background file closing
Closing files that have been appended to is relatively slow on
Windows/NTFS. This makes several Mercurial operations slower on
Windows.
The workaround to this issue is conceptually simple: use multiple
threads for I/O. Unfortunately, Python doesn't scale well to multiple
threads because of the GIL. And, refactoring our code to use threads
everywhere would be a huge undertaking. So, we decide to tackle this
problem by starting small: establishing a thread pool for closing
files.
This patch establishes a mechanism for closing file handles on separate
threads. The coordinator object is basically a queue of file handles to
operate on and a thread pool consuming from the queue.
When files are opened through the VFS layer, the caller can specify
that delay closing is allowed.
A proxy class for file handles has been added. We must use a proxy
because it isn't possible to modify __class__ on built-in types. This
adds some overhead. But as future patches will show, this overhead
is cancelled out by the benefit of closing file handles on background
threads.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 14 Jan 2016 13:34:59 -0800 |
parents | 88aa4da0cba8 |
children | 7b7e16158c35 |
line wrap: on
line diff
--- a/mercurial/help/config.txt Tue Jan 12 23:56:48 2016 +0900 +++ b/mercurial/help/config.txt Thu Jan 14 13:34:59 2016 -0800 @@ -2037,3 +2037,27 @@ Number of CPUs to use for parallel operations. A zero or negative value is treated as ``use the default``. (default: 4 or the number of CPUs on the system, whichever is larger) + +``backgroundclose`` + Whether to enable closing file handles on background threads during certain + operations. Some platforms aren't very efficient at closing file + handles that have been written or appened to. By performing file closing + on background threads, file write rate can increase substantially. + (default: true on Windows, false elsewhere) + +``backgroundcloseminfilecount`` + Minimum number of files required to trigger background file closing. + Operations not writing this many files won't start background close + threads. + (default: 2048) + +``backgroundclosemaxqueue`` + The maximum number of opened file handles waiting to be closed in the + background. This option only has an effect if ``backgroundclose`` is + enabled. + (default: 384) + +``backgroundclosethreadcount`` + Number of threads to process background file closes. Only relevant if + ``backgroundclose`` is enabled. + (default: 4)