annotate hgext/win32mbcs.py @ 48598:a6f16ec07ed7

stream-clone: add a explicit test for format change during stream clone They are different kind of requirements, the one which impact the data storage and are relevant to the files being streamed and the one which does not. For example some requirements are only relevant to the working copy, like sparse, or dirstate-v2. Since they are irrelevant to the content being streamed, they do not prevent the receiving side to use streaming clone and mercurial skip adverting them over the wire and, ideally, within the bundle. In addition, this let the client decide to use whichever format it desire for the part that does not affect the store itself. So the configuration related to these format are used as normal when doing a streaming clone. In practice, the feature was not really tested and is badly broken with bundle-2, since the requirements are not filtered out from the stream bundle. So we start with adding simple tests as a good base before the fix and adjust the feature. Differential Revision: https://phab.mercurial-scm.org/D12029
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Mon, 17 Jan 2022 18:51:47 +0100
parents 89a2afe31e82
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
1 # win32mbcs.py -- MBCS filename support for Mercurial
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
2 #
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
3 # Copyright (c) 2008 Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
4 #
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
5 # Version: 0.3
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
6 # Author: Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
7 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8001
diff changeset
8 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9569
diff changeset
9 # GNU General Public License version 2 or any later version.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
10 #
8228
eee2319c5895 add blank line after copyright notices and after header
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
11
8932
f87884329419 extensions: fix up description lines some more
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 8894
diff changeset
12 '''allow the use of MBCS paths with problematic encodings
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
13
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
14 Some MBCS encodings are not good for some path operations (i.e.
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
15 splitting path, case conversion, etc.) with its encoded bytes. We call
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
16 such a encoding (i.e. shift_jis and big5) as "problematic encoding".
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
17 This extension can be used to fix the issue with those encodings by
8665
e4ad46f9a004 win32mbcs: capitalize Unicode
Martin Geisler <mg@lazybytes.net>
parents: 8491
diff changeset
18 wrapping some functions to convert to Unicode string before path
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
19 operation.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
20
8668
aea3a23151bd fixed typos found in translatable strings
Martin Geisler <mg@lazybytes.net>
parents: 8667
diff changeset
21 This extension is useful for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
22
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
23 - Japanese Windows users using shift_jis encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
24 - Chinese Windows users using big5 encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
25 - All users who use a repository with one of problematic encodings on
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
26 case-insensitive file system.
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
27
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
28 This extension is not needed for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
29
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
30 - Any user who use only ASCII chars in path.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
31 - Any user who do not use any of problematic encodings.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
32
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
33 Note that there are some limitations on using this extension:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
34
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
35 - You should use single encoding in one repository.
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
36 - If the repository path ends with 0x5c, .hg/hgrc cannot be read.
13330
551856dea9a6 win32mbcs: Fix typo in documentation
Javi Merino <cibervicho@gmail.com>
parents: 13067
diff changeset
37 - win32mbcs is not compatible with fixutf8 extension.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
38
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
39 By default, win32mbcs uses encoding.encoding decided by Mercurial.
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
40 You can specify the encoding by config option::
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
41
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
42 [win32mbcs]
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
43 encoding = sjis
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
44
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
45 It is useful for the users who want to commit with UTF-8 log message.
8894
868670dbc237 extensions: improve the consistency of synopses
Cédric Duval <cedricduval@free.fr>
parents: 8866
diff changeset
46 '''
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
47 from __future__ import absolute_import
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
48
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
49 import os
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
50 import sys
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
51
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28417
diff changeset
52 from mercurial.i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
53 from mercurial.pycompat import getattr, setattr
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
54 from mercurial import (
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
55 encoding,
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
56 error,
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
57 pycompat,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
58 registrar,
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
59 )
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
60
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29397
diff changeset
61 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
62 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
63 # be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
64 # leave the attribute unspecified.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
65 testedwith = b'ships-with-hg-core'
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
66
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
67 configtable = {}
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
68 configitem = registrar.configitem(configtable)
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
69
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
70 # Encoding.encoding may be updated by --encoding option.
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
71 # Use a lambda do delay the resolution.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
72 configitem(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
73 b'win32mbcs',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
74 b'encoding',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
75 default=lambda: encoding.encoding,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
76 )
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
77
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
78 _encoding = None # see extsetup
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
79
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
80
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
81 def decode(arg):
43743
66210a20f727 win32mbcs: fix a `str` type conditional for py3
Matt Harbison <matt_harbison@yahoo.com>
parents: 43117
diff changeset
82 if isinstance(arg, bytes):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
83 uarg = arg.decode(_encoding)
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
84 if arg == uarg.encode(_encoding):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
85 return uarg
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
86 raise UnicodeError(b"Not local encoding")
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
87 elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
88 return tuple(map(decode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
89 elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
90 return map(decode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
91 elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
92 for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
93 arg[k] = decode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
94 return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
95
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
96
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
97 def encode(arg):
38312
79dd61a4554f py3: replace `unicode` with pycompat.unicode
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34645
diff changeset
98 if isinstance(arg, pycompat.unicode):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
99 return arg.encode(_encoding)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
100 elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
101 return tuple(map(encode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
102 elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
103 return map(encode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
104 elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
105 for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
106 arg[k] = encode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
107 return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
108
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
109
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
110 def appendsep(s):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
111 # ensure the path ends with os.sep, appending it if necessary.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
112 try:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
113 us = decode(s)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
114 except UnicodeError:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
115 us = s
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
116 if us and us[-1] not in b':/\\':
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
117 s += pycompat.ossep
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
118 return s
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
119
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
120
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
121 def basewrapper(func, argtype, enc, dec, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
122 # check check already converted, then call original
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
123 for arg in args:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
124 if isinstance(arg, argtype):
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
125 return func(*args, **kwds)
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
126
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
127 try:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
128 # convert string arguments, call func, then convert back the
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
129 # return value.
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
130 return enc(func(*dec(args), **dec(kwds)))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
131 except UnicodeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
132 raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43094
diff changeset
133 _(b"[win32mbcs] filename conversion failed with %s encoding\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
134 % _encoding
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
135 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
136
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
137
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
138 def wrapper(func, args, kwds):
38312
79dd61a4554f py3: replace `unicode` with pycompat.unicode
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34645
diff changeset
139 return basewrapper(func, pycompat.unicode, encode, decode, args, kwds)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
140
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
141
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
142 def reversewrapper(func, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
143 return basewrapper(func, str, decode, encode, args, kwds)
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
144
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
145
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
146 def wrapperforlistdir(func, args, kwds):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
147 # Ensure 'path' argument ends with os.sep to avoids
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
148 # misinterpreting last 0x5c of MBCS 2nd byte as path separator.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
149 if args:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
150 args = list(args)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
151 args[0] = appendsep(args[0])
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
152 if b'path' in kwds:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
153 kwds[b'path'] = appendsep(kwds[b'path'])
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
154 return func(*args, **kwds)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
155
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
156
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
157 def wrapname(name, wrapper):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
158 module, name = name.rsplit(b'.', 1)
9098
5e4654f5522d win32mbcs: look up modules using sys.modules (issue1729)
Brodie Rao <me+hg@dackz.net>
parents: 8932
diff changeset
159 module = sys.modules[module]
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
160 func = getattr(module, name)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
161
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
162 def f(*args, **kwds):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
163 return wrapper(func, args, kwds)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
164
30476
8a9681b963a3 win32mbcs: drop code that was catering to Python 2.3 and earlier
Augie Fackler <augie@google.com>
parents: 29889
diff changeset
165 f.__name__ = func.__name__
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
166 setattr(module, name, f)
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
167
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
168
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
169 # List of functions to be wrapped.
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
170 # NOTE: os.path.dirname() and os.path.basename() are safe because
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
171 # they use result of os.path.split()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
172 funcs = b'''os.path.join os.path.split os.path.splitext
29889
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
173 os.path.normpath os.makedirs mercurial.util.endswithsep
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
174 mercurial.util.splitpath mercurial.util.fscasesensitive
14841
6990340c57a8 win32mbcs: wrap two more functions to be wrapped.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 13330
diff changeset
175 mercurial.util.fspath mercurial.util.pconvert mercurial.util.normpath
19383
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
176 mercurial.util.checkwinfilename mercurial.util.checkosfilename
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
177 mercurial.util.split'''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
178
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
179 # These functions are required to be called with local encoded string
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
180 # because they expects argument is local encoded string and cause
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
181 # problem with unicode string.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
182 rfuncs = b'''mercurial.encoding.upper mercurial.encoding.lower
32572
377c74ef008d win32mbcs: avoid unintentional failure at colorization
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32207
diff changeset
183 mercurial.util._filenamebytestr'''
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
184
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
185 # List of Windows specific functions to be wrapped.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
186 winfuncs = b'''os.path.splitunc'''
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
187
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
188 # codec and alias names of sjis and big5 to be faked.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
189 problematic_encodings = b'''big5 big5-tw csbig5 big5hkscs big5-hkscs
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
190 hkscs cp932 932 ms932 mskanji ms-kanji shift_jis csshiftjis shiftjis
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
191 sjis s_jis shift_jis_2004 shiftjis2004 sjis_2004 sjis2004
8714
505a96cbc923 Add cp950 as problematic encoding which is used in chinese windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 8668
diff changeset
192 shift_jisx0213 shiftjisx0213 sjisx0213 s_jisx0213 950 cp950 ms950 '''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
193
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
194
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
195 def extsetup(ui):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
196 # TODO: decide use of config section for this extension
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
197 if (not os.path.supports_unicode_filenames) and (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
198 pycompat.sysplatform != b'cygwin'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
199 ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
200 ui.warn(_(b"[win32mbcs] cannot activate on this platform.\n"))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
201 return
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
202 # determine encoding for filename
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
203 global _encoding
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
204 _encoding = ui.config(b'win32mbcs', b'encoding')
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
205 # fake is only for relevant environment.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
206 if _encoding.lower() in problematic_encodings.split():
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
207 for f in funcs.split():
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
208 wrapname(f, wrapper)
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34180
diff changeset
209 if pycompat.iswindows:
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
210 for f in winfuncs.split():
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
211 wrapname(f, wrapper)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
212 wrapname(b"mercurial.util.listdir", wrapperforlistdir)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
213 wrapname(b"mercurial.windows.listdir", wrapperforlistdir)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
214 # wrap functions to be called with local byte string arguments
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
215 for f in rfuncs.split():
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
216 wrapname(f, reversewrapper)
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
217 # Check sys.args manually instead of using ui.debug() because
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
218 # command line options is not yet applied when
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
219 # extensions.loadall() is called.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
220 if b'--debug' in sys.argv:
43094
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
221 ui.writenoi18n(
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
222 b"[win32mbcs] activated with encoding: %s\n" % _encoding
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
223 )