Mads Kiilerich <madski@unity3d.com> [Sun, 17 Jan 2016 19:33:02 +0100] rev 27898
graft: clarify in help that `-r` is not just optional
Positional parameters are also treated as revisions, but the order of revisions
matters and it will often be wrong if the user understands it as `-r` taking
multiple revisions as `-r REV1 REV2`.
(Alternatively, `-r` could be turned into a no-op flag as the documentation
suggests. That would however be less "semantic markup" and I agree with the
implementation in
55e7f352b1d3 but not the documentation.)
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 13:44:01 -0800] rev 27897
streamclone: use backgroundfilecloser (
issue4889)
Closing files that have been appended to is slow on Windows/NTFS.
CloseHandle() calls on this platform often take 1-10ms - and that's
on my i7-6700K Skylake processor with a modern and fast SSD. Contrast
with other I/O operations, such as writing data, which take <100us.
This means that creating/appending thousands of files can add
significant overhead. For example, cloning mozilla-central creates
~232,000 revlog files. Assuming 1ms per CloseHandle(), that yields
232s (3:52) of wall time waiting for file closes!
The impact of this overhead can be measured most directly when applying
stream clone bundles. Applying these files is effectively uncompressing
a tar archive (read: it's very fast).
Using a RAM disk (read: no I/O wait), the difference in wall time for a
`hg debugapplystreamclonebundle` for a ~1731 MB mozilla-central bundle
between Windows and Linux from the same machine is drastic:
Linux: ~12.8s (128MB/s)
Windows: ~352.0s (4.7MB/s)
Windows is ~27.5x slower. Yikes!
After this patch:
Linux: ~12.8s (128MB/s)
Windows: ~102.1s (16.1MB/s)
Windows is now ~3.4x faster. Unfortunately, it is still ~8x slower than
Linux. Profiling reveals a few hot code paths that could likely be
improved. But those are for other patches.
This patch introduces test-clone-uncompressed.t because existing tests
of `clone --uncompressed` are scattered about and adding a variation for
background thread closing to e.g. test-http.t doesn't feel correct.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Jan 2016 16:11:36 -0800] rev 27896
streamclone: indent code
This will make the subsequent patch easier to read.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 13:34:59 -0800] rev 27895
scmutil: support background file closing
Closing files that have been appended to is relatively slow on
Windows/NTFS. This makes several Mercurial operations slower on
Windows.
The workaround to this issue is conceptually simple: use multiple
threads for I/O. Unfortunately, Python doesn't scale well to multiple
threads because of the GIL. And, refactoring our code to use threads
everywhere would be a huge undertaking. So, we decide to tackle this
problem by starting small: establishing a thread pool for closing
files.
This patch establishes a mechanism for closing file handles on separate
threads. The coordinator object is basically a queue of file handles to
operate on and a thread pool consuming from the queue.
When files are opened through the VFS layer, the caller can specify
that delay closing is allowed.
A proxy class for file handles has been added. We must use a proxy
because it isn't possible to modify __class__ on built-in types. This
adds some overhead. But as future patches will show, this overhead
is cancelled out by the benefit of closing file handles on background
threads.
Yuya Nishihara <yuya@tcha.org> [Tue, 12 Jan 2016 23:56:48 +0900] rev 27894
templatekw: add {namespaces} keyword
This provides a general-purpose interface to all custom namespaces.
The {namespaces} keyword honors the definition order of namespaces as they
are kept by sortdict.
Yuya Nishihara <yuya@tcha.org> [Tue, 12 Jan 2016 23:53:56 +0900] rev 27893
templatekw: move shownames() helper to be sorted alphabetically
I'll add shownamespaces(), which is similar to this function. I want to put
them nearby.
Yuya Nishihara <yuya@tcha.org> [Sat, 16 Jan 2016 13:53:32 +0900] rev 27892
templater: make get(dict, key) return a single value
This is necessary to obtain a _hybrid object from a dict. If get() yields
a value, it would be stringified.
I see no benefit to make get() lazy, so this patch just changes "yield" to
"return".
Yuya Nishihara <yuya@tcha.org> [Sat, 16 Jan 2016 13:42:37 +0900] rev 27891
templater: make _hybrid not callable to avoid conflicting semantics
In templater, a callable symbol exists for lazy evaluation, which should have
f(**mapping) signature. On the other hand, _hybrid.__call__(), which was
introduced by
0b241d7a8c62, generates mapping for each element.
This patch renames _hybrid.__call__() to _hybrid.itermaps() so that a _hybrid
object can be a value of a mapping dict.
{namespaces % "{namespace}: {names % "{name }"}\n"}
~~~~~
a _hybrid object
Ruslan Sayfutdinov <sayfutdinov@fb.com> [Fri, 15 Jan 2016 13:46:33 -0800] rev 27890
backout: commit changeset by default (BC)
Add --no-commit flag to prevent it. This should make the hg user
experience a little better. Some discussion can be found here:
http://markmail.org/message/7jm7ro2ias6hxywy
Danek Duvall <danek.duvall@oracle.com> [Fri, 15 Jan 2016 13:01:37 -0800] rev 27889
help: move Windows 9x information to appropriate place
Simon Farnsworth <simonfar@fb.com> [Thu, 14 Jan 2016 12:52:59 -0800] rev 27888
shelve: move commitfunc closer to use site
Supporting shelving of unknown files needs this code motion.
No functional changes.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 21:21:59 -0800] rev 27887
commands: document clone bundles hooks and rollback behavior
The added content is inside a verbose container.
I figure it makes sense to explicitly document behavior, including
with the caveat it may change later. People can't say they weren't
warned!
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 22:50:55 -0800] rev 27886
clonebundles: improve BUNDLESPEC documentation
Before the existence of `hg debugbundle --spec`, the process for
defining the BUNDLESPEC value in manifests was not very clear and not
trivial to automate, especially in the case of stream clone bundles.
This patch adds documentation to note the existence of
`hg debugbundle --spec`. We drop the reference to stream clone
requirements handling because it is now redundant with
`hg debugbundle --spec`. While we are here, we further reinforce the
importance of defining BUNDLESPEC.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 22:57:55 -0800] rev 27885
commands: teach debugbundle to print bundle specification
This seems like the most logical place to put this functionality.
Test coverage over existing known bundle specs has been added.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 21:27:53 -0800] rev 27884
commands: use context manager for opened bundle file
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 14 Jan 2016 22:49:03 -0800] rev 27883
exchange: implement function for inferring bundle specification
We don't currently have a mechanism for inferring bundle spec strings
from bundle files. This patch adds one.
This will eventually be used to make the producing of clone bundles
manifests easier.