Mercurial > hg
annotate mercurial/py3kcompat.py @ 26379:39d643252b9f
revlog: use existing file handle when reading during _addrevision
_addrevision() may need to read from revlogs as part of computing
deltas. Previously, we would flush existing file handles and open
a new, short-lived file handle to perform the reading.
If we have an existing file handle, it seems logical to reuse it
for reading instead of opening a new file handle. This patch
makes that the new behavior.
After this patch, revlog files are only reopened when adding
revisions if the revlog is switched from inline to non-inline.
On Linux when unbundling a bundle of the mozilla-central repo, this
patch has the following impact on system call counts:
Call Before After Delta
write 827,639 673,390 -154,249
open 700,103 684,089 -16,014
read 74,489 74,489 0
fstat 493,924 461,896 -32,028
close 249,131 233,117 -16,014
stat 242,001 242,001 0
lstat 18,676 18,676 0
lseek 20,268 20,268 0
ioctl 14,652 13,173 -1,479
TOTAL 3,180,758 2,930,679 -250,079
It's worth noting that many of the open() calls fail due to missing
files. That's why there are many more open() calls than close().
Despite the significant system call reduction, this change does not
seem to have a significant performance impact on Linux.
On Windows 10 (not a VM, on a SSD), this patch appears to reduce
unbundle time for mozilla-central from ~960s to ~920s. This isn't
as significant as I was hoping. But a decrease it is nonetheless.
Still, Windows unbundle performance is still >2x slower than Linux.
Despite the lack of significant gains, fewer system calls is fewer
system calls. If nothing else, this will narrow the focus of potential
areas to optimize in the future.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sun, 27 Sep 2015 16:08:18 -0700 |
parents | a7a9d84f5e4a |
children | 5bfd01a3c2a9 |
rev | line source |
---|---|
11748
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
1 # py3kcompat.py - compatibility definitions for running hg in py3k |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
2 # |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2010 Renato Cunha <renatoc@gmail.com> |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
4 # |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
7 |
21292
a7a9d84f5e4a
py3kcompat: drop unused export
Matt Mackall <mpm@selenic.com>
parents:
21291
diff
changeset
|
8 import builtins |
11748
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
9 |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
10 from numbers import Number |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
11 |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
12 def bytesformatter(format, args): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
13 '''Custom implementation of a formatter for bytestrings. |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
14 |
17424
e7cfe3587ea4
fix trivial spelling errors
Mads Kiilerich <mads@kiilerich.com>
parents:
11878
diff
changeset
|
15 This function currently relies on the string formatter to do the |
11748
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
16 formatting and always returns bytes objects. |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
17 |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
18 >>> bytesformatter(20, 10) |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
19 0 |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
20 >>> bytesformatter('unicode %s, %s!', ('string', 'foo')) |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
21 b'unicode string, foo!' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
22 >>> bytesformatter(b'test %s', 'me') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
23 b'test me' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
24 >>> bytesformatter('test %s', 'me') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
25 b'test me' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
26 >>> bytesformatter(b'test %s', b'me') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
27 b'test me' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
28 >>> bytesformatter('test %s', b'me') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
29 b'test me' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
30 >>> bytesformatter('test %d: %s', (1, b'result')) |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
31 b'test 1: result' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
32 ''' |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
33 # The current implementation just converts from bytes to unicode, do |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
34 # what's needed and then convert the results back to bytes. |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
35 # Another alternative is to use the Python C API implementation. |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
36 if isinstance(format, Number): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
37 # If the fixer erroneously passes a number remainder operation to |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
38 # bytesformatter, we just return the correct operation |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
39 return format % args |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
40 if isinstance(format, bytes): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
41 format = format.decode('utf-8', 'surrogateescape') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
42 if isinstance(args, bytes): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
43 args = args.decode('utf-8', 'surrogateescape') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
44 if isinstance(args, tuple): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
45 newargs = [] |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
46 for arg in args: |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
47 if isinstance(arg, bytes): |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
48 arg = arg.decode('utf-8', 'surrogateescape') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
49 newargs.append(arg) |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
50 args = tuple(newargs) |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
51 ret = format % args |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
52 return ret.encode('utf-8', 'surrogateescape') |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
53 builtins.bytesformatter = bytesformatter |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
54 |
11878
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
55 origord = builtins.ord |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
56 def fakeord(char): |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
57 if isinstance(char, int): |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
58 return char |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
59 return origord(char) |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
60 builtins.ord = fakeord |
8bb1481cf08f
py3kcompat: added fake ord implementation for py3k
Renato Cunha <renatoc@gmail.com>
parents:
11748
diff
changeset
|
61 |
11748
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
62 if __name__ == '__main__': |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
63 import doctest |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
64 doctest.testmod() |
37a70a784397
py3kcompat: added a "compatibility layer" for py3k
Renato Cunha <renatoc@gmail.com>
parents:
diff
changeset
|
65 |