annotate i18n/hggettext @ 31415:5d92107dfb9b

tagmerge: use 'wvfs' instead of 'wfile' Method is about to be deprecated and the modern form is shorter.
author Pierre-Yves David <pierre-yves.david@ens-lyon.org>
date Wed, 15 Mar 2017 00:28:58 -0700
parents 041fecbb588a
children 16a175b3681e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
1 #!/usr/bin/env python
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
2 #
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
3 # hggettext - carefully extract docstrings for Mercurial
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
4 #
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
5 # Copyright 2009 Matt Mackall <mpm@selenic.com> and others
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
6 #
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9539
diff changeset
8 # GNU General Public License version 2 or any later version.
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
9
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
10 # The normalize function is taken from pygettext which is distributed
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
11 # with Python under the Python License, which is GPL compatible.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
12
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
13 """Extract docstrings from Mercurial commands.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
14
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
15 Compared to pygettext, this script knows about the cmdtable and table
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
16 dictionaries used by Mercurial, and will only extract docstrings from
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
17 functions mentioned therein.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
18
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
19 Use xgettext like normal to extract strings marked as translatable and
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
20 join the message cataloges to get the final catalog.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
21 """
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
22
29171
de28dedd1ff1 py3: make i18n/hggettext use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29170
diff changeset
23 from __future__ import absolute_import, print_function
29170
2516bba643e7 py3: make i18n/hggettext use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 12823
diff changeset
24
2516bba643e7 py3: make i18n/hggettext use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 12823
diff changeset
25 import inspect
2516bba643e7 py3: make i18n/hggettext use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 12823
diff changeset
26 import os
2516bba643e7 py3: make i18n/hggettext use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 12823
diff changeset
27 import sys
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
28
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
29
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
30 def escape(s):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
31 # The order is important, the backslash must be escaped first
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
32 # since the other replacements introduce new backslashes
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
33 # themselves.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
34 s = s.replace('\\', '\\\\')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
35 s = s.replace('\n', '\\n')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
36 s = s.replace('\r', '\\r')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
37 s = s.replace('\t', '\\t')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
38 s = s.replace('"', '\\"')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
39 return s
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
40
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
41
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
42 def normalize(s):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
43 # This converts the various Python string types into a format that
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
44 # is appropriate for .po files, namely much closer to C style.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
45 lines = s.split('\n')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
46 if len(lines) == 1:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
47 s = '"' + escape(s) + '"'
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
48 else:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
49 if not lines[-1]:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
50 del lines[-1]
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
51 lines[-1] = lines[-1] + '\n'
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
52 lines = map(escape, lines)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
53 lineterm = '\\n"\n"'
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
54 s = '""\n"' + lineterm.join(lines) + '"'
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
55 return s
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
56
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
57
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
58 def poentry(path, lineno, s):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
59 return ('#: %s:%d\n' % (path, lineno) +
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
60 'msgid %s\n' % normalize(s) +
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
61 'msgstr ""\n')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
62
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
63
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
64 def offset(src, doc, name, default):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
65 """Compute offset or issue a warning on stdout."""
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
66 # Backslashes in doc appear doubled in src.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
67 end = src.find(doc.replace('\\', '\\\\'))
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
68 if end == -1:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
69 # This can happen if the docstring contains unnecessary escape
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
70 # sequences such as \" in a triple-quoted string. The problem
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
71 # is that \" is turned into " and so doc wont appear in src.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
72 sys.stderr.write("warning: unknown offset in %s, assuming %d lines\n"
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
73 % (name, default))
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
74 return default
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
75 else:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
76 return src.count('\n', 0, end)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
77
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
78
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
79 def importpath(path):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
80 """Import a path like foo/bar/baz.py and return the baz module."""
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
81 if path.endswith('.py'):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
82 path = path[:-3]
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
83 if path.endswith('/__init__'):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
84 path = path[:-9]
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
85 path = path.replace('/', '.')
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
86 mod = __import__(path)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
87 for comp in path.split('.')[1:]:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
88 mod = getattr(mod, comp)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
89 return mod
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
90
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
91
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
92 def docstrings(path):
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
93 """Extract docstrings from path.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
94
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
95 This respects the Mercurial cmdtable/table convention and will
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
96 only extract docstrings from functions mentioned in these tables.
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
97 """
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
98 mod = importpath(path)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
99 if mod.__doc__:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
100 src = open(path).read()
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
101 lineno = 1 + offset(src, mod.__doc__, path, 7)
29171
de28dedd1ff1 py3: make i18n/hggettext use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29170
diff changeset
102 print(poentry(path, lineno, mod.__doc__))
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
103
12823
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
104 functions = list(getattr(mod, 'i18nfunctions', []))
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
105 functions = [(f, True) for f in functions]
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
106
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
107 cmdtable = getattr(mod, 'cmdtable', {})
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
108 if not cmdtable:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
109 # Maybe we are processing mercurial.commands?
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
110 cmdtable = getattr(mod, 'table', {})
12823
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
111 functions.extend((c[0], False) for c in cmdtable.itervalues())
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
112
12823
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
113 for func, rstrip in functions:
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
114 if func.__doc__:
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
115 src = inspect.getsource(func)
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
116 name = "%s.%s" % (path, func.__name__)
29720
041fecbb588a i18n: use inspect.getsourcelines() to obtain lineno from func or class
Yuya Nishihara <yuya@tcha.org>
parents: 29171
diff changeset
117 lineno = inspect.getsourcelines(func)[1]
12823
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
118 doc = func.__doc__
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
119 if rstrip:
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
120 doc = doc.rstrip()
80deae3bc5ea hggettext: handle i18nfunctions declaration for docstrings translations
Patrick Mezard <pmezard@gmail.com>
parents: 10263
diff changeset
121 lineno += offset(src, doc, name, 1)
29171
de28dedd1ff1 py3: make i18n/hggettext use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29170
diff changeset
122 print(poentry(path, lineno, doc))
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
123
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
124
9539
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
125 def rawtext(path):
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
126 src = open(path).read()
29171
de28dedd1ff1 py3: make i18n/hggettext use print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29170
diff changeset
127 print(poentry(path, 1, src))
9539
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
128
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
129
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
130 if __name__ == "__main__":
8626
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
131 # It is very important that we import the Mercurial modules from
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
132 # the source tree where hggettext is executed. Otherwise we might
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
133 # accidentally import and extract strings from a Mercurial
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
134 # installation mentioned in PYTHONPATH.
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
135 sys.path.insert(0, os.getcwd())
1fc1c77d4863 hggettext: ensure correct Mercurial is imported
Martin Geisler <mg@lazybytes.net>
parents: 8542
diff changeset
136 from mercurial import demandimport; demandimport.enable()
8542
de150a942ec8 i18n: accurately generate hg.pot
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
137 for path in sys.argv[1:]:
9539
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
138 if path.endswith('.txt'):
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
139 rawtext(path)
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
140 else:
c904e76e3834 help: move help topics from mercurial/help.py to help/*.txt
Martin Geisler <mg@lazybytes.net>
parents: 8626
diff changeset
141 docstrings(path)