Mercurial > hg
annotate hgext/win32mbcs.py @ 32572:377c74ef008d stable
win32mbcs: avoid unintentional failure at colorization
Since 176ed32dc159, pycompat.bytestr() wrapped by win32mbcs returns
unicode object, if an argument is not byte-str object. And this causes
unexpected failure at colorization.
pycompat.bytestr() is used to convert from color effect "int" value to
byte-str object in color module. Wrapped pycompat.bytestr() returns
unicode object for such "int" value, because it isn't byte-str.
If this returned unicode object is used to colorize non-ASCII byte-str
in cases below, UnicodeDecodeError is raised at an operation between
them.
- colorization uses "ansi" color mode, or
Even though this isn't default on Windows, user might use this
color mode for third party pager.
- ui.write() is buffered with labeled=True
Buffering causes "ansi" color mode internally, regardless of
actual color mode. With "win32" color mode, extra escape sequences
are omitted at writing data out.
For example, with "win32" color mode, "hg status" doesn't fail for
non-ASCII filenames, but "hg log" does for non-ASCII text, because
the latter implies buffered formatter.
There are many "color effect" value lines in color.py, and making them
byte-str objects isn't suitable for fixing on stable. In addition to
it, pycompat.bytestr will be used to get byte-str object from any
types other than int, too.
To resolve this issue, this patch does:
- replace pycompat.bytestr in checkwinfilename() with newly added
hook point util._filenamebytestr, and
- make win32mbcs reverse-wrap util._filenamebytestr
(this is a replacement of 176ed32dc159)
This patch does two things above at same time, because separately
applying the former change adds broken revision (from point of view of
win32mbcs) to stable branch.
"_" prefix is added to "filenamebytestr", because it is win32mbcs
specific hook point.
author | FUJIWARA Katsunori <foozy@lares.dti.ne.jp> |
---|---|
date | Wed, 31 May 2017 23:44:33 +0900 |
parents | 176ed32dc159 |
children | 9929af2b09b4 |
rev | line source |
---|---|
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
1 # win32mbcs.py -- MBCS filename support for Mercurial |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
2 # |
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
3 # Copyright (c) 2008 Shun-ichi Goto <shunichi.goto@gmail.com> |
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
4 # |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
5 # Version: 0.3 |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
6 # Author: Shun-ichi Goto <shunichi.goto@gmail.com> |
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
7 # |
8225
46293a0c7e9f
updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents:
8001
diff
changeset
|
8 # This software may be used and distributed according to the terms of the |
10263 | 9 # GNU General Public License version 2 or any later version. |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
10 # |
8228
eee2319c5895
add blank line after copyright notices and after header
Martin Geisler <mg@lazybytes.net>
parents:
8225
diff
changeset
|
11 |
8932
f87884329419
extensions: fix up description lines some more
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
8894
diff
changeset
|
12 '''allow the use of MBCS paths with problematic encodings |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
13 |
8001
c0e3aca616de
win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents:
7983
diff
changeset
|
14 Some MBCS encodings are not good for some path operations (i.e. |
c0e3aca616de
win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents:
7983
diff
changeset
|
15 splitting path, case conversion, etc.) with its encoded bytes. We call |
c0e3aca616de
win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents:
7983
diff
changeset
|
16 such a encoding (i.e. shift_jis and big5) as "problematic encoding". |
c0e3aca616de
win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents:
7983
diff
changeset
|
17 This extension can be used to fix the issue with those encodings by |
8665
e4ad46f9a004
win32mbcs: capitalize Unicode
Martin Geisler <mg@lazybytes.net>
parents:
8491
diff
changeset
|
18 wrapping some functions to convert to Unicode string before path |
8001
c0e3aca616de
win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents:
7983
diff
changeset
|
19 operation. |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
20 |
8668
aea3a23151bd
fixed typos found in translatable strings
Martin Geisler <mg@lazybytes.net>
parents:
8667
diff
changeset
|
21 This extension is useful for: |
9216
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
22 |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
23 - Japanese Windows users using shift_jis encoding. |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
24 - Chinese Windows users using big5 encoding. |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
25 - All users who use a repository with one of problematic encodings on |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
26 case-insensitive file system. |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
27 |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
28 This extension is not needed for: |
9216
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
29 |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
30 - Any user who use only ASCII chars in path. |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
31 - Any user who do not use any of problematic encodings. |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
32 |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
33 Note that there are some limitations on using this extension: |
9216
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
34 |
9b2649b6ce5c
win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents:
9154
diff
changeset
|
35 - You should use single encoding in one repository. |
13067
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
36 - If the repository path ends with 0x5c, .hg/hgrc cannot be read. |
13330
551856dea9a6
win32mbcs: Fix typo in documentation
Javi Merino <cibervicho@gmail.com>
parents:
13067
diff
changeset
|
37 - win32mbcs is not compatible with fixutf8 extension. |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
38 |
10067
fd6729805f44
win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents:
10050
diff
changeset
|
39 By default, win32mbcs uses encoding.encoding decided by Mercurial. |
fd6729805f44
win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents:
10050
diff
changeset
|
40 You can specify the encoding by config option:: |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
41 |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
42 [win32mbcs] |
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
43 encoding = sjis |
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
44 |
10067
fd6729805f44
win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents:
10050
diff
changeset
|
45 It is useful for the users who want to commit with UTF-8 log message. |
8894
868670dbc237
extensions: improve the consistency of synopses
Cédric Duval <cedricduval@free.fr>
parents:
8866
diff
changeset
|
46 ''' |
28417
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
47 from __future__ import absolute_import |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
48 |
28417
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
49 import os |
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
50 import sys |
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
51 |
29205
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28417
diff
changeset
|
52 from mercurial.i18n import _ |
28417
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
53 from mercurial import ( |
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
54 encoding, |
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
55 error, |
30616
6f9fcd29e290
py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30476
diff
changeset
|
56 pycompat, |
28417
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
57 ) |
588874c33b4d
win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents:
26587
diff
changeset
|
58 |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29397
diff
changeset
|
59 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
25186
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
19383
diff
changeset
|
60 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
19383
diff
changeset
|
61 # be specifying the version(s) of Mercurial they are tested with, or |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
19383
diff
changeset
|
62 # leave the attribute unspecified. |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29397
diff
changeset
|
63 testedwith = 'ships-with-hg-core' |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
64 |
13067
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
65 _encoding = None # see extsetup |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
66 |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
67 def decode(arg): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
68 if isinstance(arg, str): |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
69 uarg = arg.decode(_encoding) |
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
70 if arg == uarg.encode(_encoding): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
71 return uarg |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
72 raise UnicodeError("Not local encoding") |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
73 elif isinstance(arg, tuple): |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
74 return tuple(map(decode, arg)) |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
75 elif isinstance(arg, list): |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
76 return map(decode, arg) |
9131
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
77 elif isinstance(arg, dict): |
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
78 for k, v in arg.items(): |
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
79 arg[k] = decode(v) |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
80 return arg |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
81 |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
82 def encode(arg): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
83 if isinstance(arg, unicode): |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
84 return arg.encode(_encoding) |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
85 elif isinstance(arg, tuple): |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
86 return tuple(map(encode, arg)) |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
87 elif isinstance(arg, list): |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
88 return map(encode, arg) |
9131
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
89 elif isinstance(arg, dict): |
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
90 for k, v in arg.items(): |
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
91 arg[k] = encode(v) |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
92 return arg |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
93 |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
94 def appendsep(s): |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
95 # ensure the path ends with os.sep, appending it if necessary. |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
96 try: |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
97 us = decode(s) |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
98 except UnicodeError: |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
99 us = s |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
100 if us and us[-1] not in ':/\\': |
30616
6f9fcd29e290
py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30476
diff
changeset
|
101 s += pycompat.ossep |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
102 return s |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
103 |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
104 |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
105 def basewrapper(func, argtype, enc, dec, args, kwds): |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
106 # check check already converted, then call original |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
107 for arg in args: |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
108 if isinstance(arg, argtype): |
9131
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
109 return func(*args, **kwds) |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
110 |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
111 try: |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
112 # convert string arguments, call func, then convert back the |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
113 # return value. |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
114 return enc(func(*dec(args), **dec(kwds))) |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
115 except UnicodeError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25186
diff
changeset
|
116 raise error.Abort(_("[win32mbcs] filename conversion failed with" |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
117 " %s encoding\n") % (_encoding)) |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
118 |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
119 def wrapper(func, args, kwds): |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
120 return basewrapper(func, unicode, encode, decode, args, kwds) |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
121 |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
122 |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
123 def reversewrapper(func, args, kwds): |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
124 return basewrapper(func, str, decode, encode, args, kwds) |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
125 |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
126 def wrapperforlistdir(func, args, kwds): |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
127 # Ensure 'path' argument ends with os.sep to avoids |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
128 # misinterpreting last 0x5c of MBCS 2nd byte as path separator. |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
129 if args: |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
130 args = list(args) |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
131 args[0] = appendsep(args[0]) |
9391
2705e6816d33
use 'x in dict' instead of 'dict.has_key(x)'
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents:
9275
diff
changeset
|
132 if 'path' in kwds: |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
133 kwds['path'] = appendsep(kwds['path']) |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
134 return func(*args, **kwds) |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
135 |
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
136 def wrapname(name, wrapper): |
9098
5e4654f5522d
win32mbcs: look up modules using sys.modules (issue1729)
Brodie Rao <me+hg@dackz.net>
parents:
8932
diff
changeset
|
137 module, name = name.rsplit('.', 1) |
5e4654f5522d
win32mbcs: look up modules using sys.modules (issue1729)
Brodie Rao <me+hg@dackz.net>
parents:
8932
diff
changeset
|
138 module = sys.modules[module] |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
139 func = getattr(module, name) |
9131
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
140 def f(*args, **kwds): |
2bbb8419720d
win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9100
diff
changeset
|
141 return wrapper(func, args, kwds) |
30476
8a9681b963a3
win32mbcs: drop code that was catering to Python 2.3 and earlier
Augie Fackler <augie@google.com>
parents:
29889
diff
changeset
|
142 f.__name__ = func.__name__ |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
143 setattr(module, name, f) |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
144 |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
145 # List of functions to be wrapped. |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
146 # NOTE: os.path.dirname() and os.path.basename() are safe because |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
147 # they use result of os.path.split() |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
148 funcs = '''os.path.join os.path.split os.path.splitext |
29889
6f447b9ec263
util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
29841
diff
changeset
|
149 os.path.normpath os.makedirs mercurial.util.endswithsep |
6f447b9ec263
util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
29841
diff
changeset
|
150 mercurial.util.splitpath mercurial.util.fscasesensitive |
14841
6990340c57a8
win32mbcs: wrap two more functions to be wrapped.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
13330
diff
changeset
|
151 mercurial.util.fspath mercurial.util.pconvert mercurial.util.normpath |
19383
41c06a02814e
win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17798
diff
changeset
|
152 mercurial.util.checkwinfilename mercurial.util.checkosfilename |
41c06a02814e
win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17798
diff
changeset
|
153 mercurial.util.split''' |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
154 |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
155 # These functions are required to be called with local encoded string |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
156 # because they expects argument is local encoded string and cause |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
157 # problem with unicode string. |
32207
176ed32dc159
win32mbcs: wrap underlying pycompat.bytestr to use checkwinfilename safely
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
30642
diff
changeset
|
158 rfuncs = '''mercurial.encoding.upper mercurial.encoding.lower |
32572
377c74ef008d
win32mbcs: avoid unintentional failure at colorization
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
32207
diff
changeset
|
159 mercurial.util._filenamebytestr''' |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
160 |
15724
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
161 # List of Windows specific functions to be wrapped. |
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
162 winfuncs = '''os.path.splitunc''' |
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
163 |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
164 # codec and alias names of sjis and big5 to be faked. |
6887
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
165 problematic_encodings = '''big5 big5-tw csbig5 big5hkscs big5-hkscs |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
166 hkscs cp932 932 ms932 mskanji ms-kanji shift_jis csshiftjis shiftjis |
304484c7e0ba
Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents:
6210
diff
changeset
|
167 sjis s_jis shift_jis_2004 shiftjis2004 sjis_2004 sjis2004 |
8714
505a96cbc923
Add cp950 as problematic encoding which is used in chinese windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
8668
diff
changeset
|
168 shift_jisx0213 shiftjisx0213 sjisx0213 s_jisx0213 950 cp950 ms950 ''' |
5846
02884e56c217
New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff
changeset
|
169 |
13067
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
170 def extsetup(ui): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
171 # TODO: decide use of config section for this extension |
15724
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
172 if ((not os.path.supports_unicode_filenames) and |
30642
e995f00a9e9a
py3: replace sys.platform with pycompat.sysplatform (part 2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30640
diff
changeset
|
173 (pycompat.sysplatform != 'cygwin')): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
174 ui.warn(_("[win32mbcs] cannot activate on this platform.\n")) |
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
175 return |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
176 # determine encoding for filename |
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
177 global _encoding |
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
178 _encoding = ui.config('win32mbcs', 'encoding', encoding.encoding) |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
179 # fake is only for relevant environment. |
10050
dd37f044f1fa
win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9569
diff
changeset
|
180 if _encoding.lower() in problematic_encodings.split(): |
7877
eba7f12b0c51
cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
7598
diff
changeset
|
181 for f in funcs.split(): |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
182 wrapname(f, wrapper) |
30640
7a3e67bfa417
py3: replace os.name with pycompat.osname (part 2 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30616
diff
changeset
|
183 if pycompat.osname == 'nt': |
15724
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
184 for f in winfuncs.split(): |
9e6a13c2aeb9
win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
15672
diff
changeset
|
185 wrapname(f, wrapper) |
9132
b47d7b440c5c
win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
9131
diff
changeset
|
186 wrapname("mercurial.osutil.listdir", wrapperforlistdir) |
17798
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
187 # wrap functions to be called with local byte string arguments |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
188 for f in rfuncs.split(): |
4091b0322918
win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
17428
diff
changeset
|
189 wrapname(f, reversewrapper) |
13067
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
190 # Check sys.args manually instead of using ui.debug() because |
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
191 # command line options is not yet applied when |
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
192 # extensions.loadall() is called. |
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
193 if '--debug' in sys.argv: |
29397
844f72885fb9
check-code: detect "missing _() in ui message" more exactly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
29205
diff
changeset
|
194 ui.write(("[win32mbcs] activated with encoding: %s\n") |
13067
9696954415db
win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
10264
diff
changeset
|
195 % _encoding) |