Mercurial > hg-stable
annotate mercurial/i18n.py @ 23031:3c0983cc279e
i18n: cache the result of every gettext call
In looking at profiler output for 'hg log' on mozilla-central, I
noticed we spent a _huge_ amount of time in gettext relative to what
it's doing. Caching provides a roughly 15% performance improvement
even on repositories as small as hg.
== hg repo on linux ==
Before:
% cumulative self
time seconds seconds name
5.05 0.19 0.19 i18n.py:62:gettext
4.84 0.18 0.18 revlog.py:88:decompress
2.95 0.17 0.11 changelog.py:201:node
2.32 0.09 0.09 ui.py:577:write
2.11 0.08 0.08 i18n.py:72:gettext
2.11 0.08 0.08 obsolete.py:196:_fm0readmarkers
1.89 0.07 0.07 obsolete.py:569:_load
1.68 0.63 0.06 localrepo.py:29:__get__
real 0m4.026s
user 0m3.993s
sys 0m0.034s
After:
% cumulative self
time seconds seconds name
8.05 0.26 0.26 revlog.py:88:decompress
2.68 0.22 0.09 color.py:395:write
2.20 0.07 0.07 obsolete.py:196:_fm0readmarkers
1.95 0.06 0.06 obsolete.py:174:_fm0readmarkers
1.95 0.06 0.06 ui.py:577:write
1.95 0.06 0.06 util.py:1228:datestr
1.71 0.06 0.06 utf_8.py:16:decode
1.71 0.06 0.06 revlog.py:273:__len__
real 0m3.519s
user 0m3.447s
sys 0m0.073s
== mozilla-central repo on linux ==
Before:
% cumulative self
time seconds seconds name
7.72 2.35 2.35 revlog.py:88:decompress
4.46 1.36 1.36 i18n.py:62:gettext
2.22 0.67 0.67 i18n.py:72:gettext
2.19 1.14 0.67 changelog.py:201:node
2.16 0.66 0.66 ui.py:577:write
1.96 0.60 0.60 utf_8.py:16:decode
1.93 1.97 0.59 color.py:395:write
1.85 0.81 0.56 changelog.py:136:tip
real 0m30.822s
user 0m30.660s
sys 0m0.149s
After:
% cumulative self
time seconds seconds name
9.82 2.49 2.49 revlog.py:88:decompress
2.67 1.31 0.68 localrepo.py:29:__get__
2.57 0.65 0.65 utf_8.py:16:decode
2.48 1.01 0.63 changelog.py:201:node
2.10 0.82 0.53 changelog.py:136:tip
2.01 0.51 0.51 ui.py:577:write
1.91 0.49 0.49 util.py:1232:datestr
1.85 1.65 0.47 color.py:395:write
real 0m25.619s
user 0m25.446s
sys 0m0.166s
== cpython repo on os x =
Before:
% cumulative self
time seconds seconds name
5.05 1.35 1.35 cmdutil.py:982:_show
4.59 1.22 1.22 revlog.py:274:__len__
3.98 1.06 1.06 i18n.py:62:gettext
3.91 1.04 1.04 revlog.py:1016:revision
3.68 0.98 0.98 revlog.py:337:parents
3.45 0.92 0.92 revlog.py:88:decompress
2.91 0.78 0.78 revlog.py:309:rev
2.62 0.70 0.70 revlog.py:1033:revision
real 0m30.414s
user 0m28.145s
sys 0m0.541s
After:
% cumulative self
time seconds seconds name
7.98 1.66 1.66 cmdutil.py:982:_show
6.83 1.42 1.42 changelog.py:46:decodeextra
5.18 1.08 1.08 revlog.py:274:__len__
3.94 0.82 0.82 revlog.py:1016:revision
3.41 0.71 0.71 revlog.py:309:rev
3.32 0.69 0.69 revlog.py:88:decompress
2.99 0.63 0.62 revlog.py:1033:revision
2.69 0.56 0.56 revlog.py:341:start
real 0m22.811s
user 0m21.883s
sys 0m0.397s
author | Augie Fackler <raf@durin42.com> |
---|---|
date | Fri, 17 Oct 2014 13:52:10 -0400 |
parents | 0d0350cfc7ab |
children | 2c07c6884394 |
rev | line source |
---|---|
8226
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
1 # i18n.py - internationalization support for mercurial |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
2 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
3 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com> |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
4 # |
8b2cd04a6e97
put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
10263 | 6 # GNU General Public License version 2 or any later version. |
1400
cf9a1233738a
i18n first part: make '_' available for files who need it
Benoit Boissinot <benoit.boissinot@ens-lyon.org
parents:
diff
changeset
|
7 |
8312
b87a50b7125c
separate import lines from mercurial and general python modules
Simon Heimberg <simohe@besonet.ch>
parents:
8226
diff
changeset
|
8 import encoding |
22638
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
9 import gettext as gettextmod, sys, os, locale |
7650
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
10 |
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
11 # modelled after templater.templatepath: |
14975
b64538363dbe
i18n: use getattr instead of hasattr
Augie Fackler <durin42@gmail.com>
parents:
13849
diff
changeset
|
12 if getattr(sys, 'frozen', None) is not None: |
7650
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
13 module = sys.executable |
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
14 else: |
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
15 module = __file__ |
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
16 |
85ae7aaf08e9
i18n: lookup .mo files in private locale/ directory
Martin Geisler <mg@daimi.au.dk>
parents:
3888
diff
changeset
|
17 |
21987
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
18 _languages = None |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
19 if (os.name == 'nt' |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
20 and 'LANGUAGE' not in os.environ |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
21 and 'LC_ALL' not in os.environ |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
22 and 'LC_MESSAGES' not in os.environ |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
23 and 'LANG' not in os.environ): |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
24 # Try to detect UI language by "User Interface Language Management" API |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
25 # if no locale variables are set. Note that locale.getdefaultlocale() |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
26 # uses GetLocaleInfo(), which may be different from UI language. |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
27 # (See http://msdn.microsoft.com/en-us/library/dd374098(v=VS.85).aspx ) |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
28 try: |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
29 import ctypes |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
30 langid = ctypes.windll.kernel32.GetUserDefaultUILanguage() |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
31 _languages = [locale.windows_locale[langid]] |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
32 except (ImportError, AttributeError, KeyError): |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
33 # ctypes not found or unknown langid |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
34 pass |
4953cd193e84
i18n: detect UI language without POSIX-style locale variable on Windows (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
21746
diff
changeset
|
35 |
22638
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
36 _ugettext = None |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
37 |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
38 def setdatapath(datapath): |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
39 localedir = os.path.join(datapath, 'locale') |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
40 t = gettextmod.translation('hg', localedir, _languages, fallback=True) |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
41 global _ugettext |
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
42 _ugettext = t.ugettext |
7651
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
43 |
23031
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
44 _msgcache = {} |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
45 |
7651
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
46 def gettext(message): |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
47 """Translate message. |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
48 |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
49 The message is looked up in the catalog to get a Unicode string, |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
50 which is encoded in the local encoding before being returned. |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
51 |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
52 Important: message is restricted to characters in the encoding |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
53 given by sys.getdefaultencoding() which is most likely 'ascii'. |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
54 """ |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
55 # If message is None, t.ugettext will return u'None' as the |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
56 # translation whereas our callers expect us to return None. |
22638
0d0350cfc7ab
i18n: use datapath for i18n like for templates and help
Mads Kiilerich <madski@unity3d.com>
parents:
21987
diff
changeset
|
57 if message is None or not _ugettext: |
7651
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
58 return message |
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
59 |
23031
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
60 if message not in _msgcache: |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
61 if type(message) is unicode: |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
62 # goofy unicode docstrings in test |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
63 paragraphs = message.split(u'\n\n') |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
64 else: |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
65 paragraphs = [p.decode("ascii") for p in message.split('\n\n')] |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
66 # Be careful not to translate the empty string -- it holds the |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
67 # meta data of the .po file. |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
68 u = u'\n\n'.join([p and _ugettext(p) or '' for p in paragraphs]) |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
69 try: |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
70 # encoding.tolocal cannot be used since it will first try to |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
71 # decode the Unicode string. Calling u.decode(enc) really |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
72 # means u.encode(sys.getdefaultencoding()).decode(enc). Since |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
73 # the Python encoding defaults to 'ascii', this fails if the |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
74 # translated string use non-ASCII characters. |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
75 _msgcache[message] = u.encode(encoding.encoding, "replace") |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
76 except LookupError: |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
77 # An unknown encoding results in a LookupError. |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
78 _msgcache[message] = message |
3c0983cc279e
i18n: cache the result of every gettext call
Augie Fackler <raf@durin42.com>
parents:
22638
diff
changeset
|
79 return _msgcache[message] |
7651
5b5036ef847a
i18n: encode output in user's local encoding
Martin Geisler <mg@daimi.au.dk>
parents:
7650
diff
changeset
|
80 |
13849
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
81 def _plain(): |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
82 if 'HGPLAIN' not in os.environ and 'HGPLAINEXCEPT' not in os.environ: |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
83 return False |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
84 exceptions = os.environ.get('HGPLAINEXCEPT', '').strip().split(',') |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
85 return 'i18n' not in exceptions |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
86 |
9f97de157aad
HGPLAIN: allow exceptions to plain mode, like i18n, via HGPLAINEXCEPT
Brodie Rao <brodie@bitheap.org>
parents:
11403
diff
changeset
|
87 if _plain(): |
10455
40dfd46d098f
ui: add HGPLAIN environment variable for easier scripting
Brodie Rao <me+hg@dackz.net>
parents:
10263
diff
changeset
|
88 _ = lambda message: message |
40dfd46d098f
ui: add HGPLAIN environment variable for easier scripting
Brodie Rao <me+hg@dackz.net>
parents:
10263
diff
changeset
|
89 else: |
40dfd46d098f
ui: add HGPLAIN environment variable for easier scripting
Brodie Rao <me+hg@dackz.net>
parents:
10263
diff
changeset
|
90 _ = gettext |