annotate hgdemandimport/demandimportpy3.py @ 43096:813aa8cc55d4

patchbomb: use procutil.shellquote() instead of shlex to escape email address This basically backs out 2cc453284d5c, and inserts procutil.shellquote() instead. I don't care about Windows compatibility here, but shlex.quote() can't handle byte strings on Python 3.
author Yuya Nishihara <yuya@tcha.org>
date Wed, 02 Oct 2019 07:35:22 +0900
parents 2372284d9457
children 2d31ef3fb494
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
1 # demandimportpy3 - global demand-loading of modules for Mercurial
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
2 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
3 # Copyright 2017 Facebook Inc.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
4 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
7
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
8 """Lazy loading for Python 3.6 and above.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
9
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
10 This uses the new importlib finder/loader functionality available in Python 3.5
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
11 and up. The code reuses most of the mechanics implemented inside importlib.util,
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
12 but with a few additions:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
13
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
14 * Allow excluding certain modules from lazy imports.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
15 * Expose an interface that's substantially the same as demandimport for
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
16 Python 2.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
17
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
18 This also has some limitations compared to the Python 2 implementation:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
19
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
20 * Much of the logic is per-package, not per-module, so any packages loaded
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
21 before demandimport is enabled will not be lazily imported in the future. In
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
22 practice, we only expect builtins to be loaded before demandimport is
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
23 enabled.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
24 """
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
25
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
26 # This line is unnecessary, but it satisfies test-check-py3-compat.t.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
27 from __future__ import absolute_import
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
28
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
29 import contextlib
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
30 import importlib.abc
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
31 import importlib.machinery
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
32 import importlib.util
33898
3595e4e0ae57 demandimportpy3: update to pass import checker
Augie Fackler <raf@durin42.com>
parents: 33859
diff changeset
33 import sys
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
34
42474
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
35 from . import tracing
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
36
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
37 _deactivated = False
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
38
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
39
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
40 class _lazyloaderex(importlib.util.LazyLoader):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
41 """This is a LazyLoader except it also follows the _deactivated global and
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
42 the ignore list.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
43 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
44
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
45 def exec_module(self, module):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
46 """Make the module load lazily."""
42474
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
47 with tracing.log('demandimport %s', module):
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
48 if _deactivated or module.__name__ in ignores:
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
49 self.loader.exec_module(module)
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
50 else:
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
51 super().exec_module(module)
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
52
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
53
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
54 # This is 3.6+ because with Python 3.5 it isn't possible to lazily load
35524
fcb1ecf2bef7 hgdemandimport: use correct hyperlink to python-bug in comments (issue5765)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33898
diff changeset
55 # extensions. See the discussion in https://bugs.python.org/issue26186 for more.
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
56 _extensions_loader = _lazyloaderex.factory(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
57 importlib.machinery.ExtensionFileLoader
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
58 )
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
59 _bytecode_loader = _lazyloaderex.factory(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
60 importlib.machinery.SourcelessFileLoader
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
61 )
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
62 _source_loader = _lazyloaderex.factory(importlib.machinery.SourceFileLoader)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
63
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
64
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
65 def _makefinder(path):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
66 return importlib.machinery.FileFinder(
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
67 path,
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
68 # This is the order in which loaders are passed in in core Python.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
69 (_extensions_loader, importlib.machinery.EXTENSION_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
70 (_source_loader, importlib.machinery.SOURCE_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
71 (_bytecode_loader, importlib.machinery.BYTECODE_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
72 )
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
73
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
74
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
75 ignores = set()
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
76
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
77
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
78 def init(ignoreset):
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
79 global ignores
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
80 ignores = ignoreset
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
81
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
82
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
83 def isenabled():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
84 return _makefinder in sys.path_hooks and not _deactivated
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
85
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
86
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
87 def disable():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
88 try:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
89 while True:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
90 sys.path_hooks.remove(_makefinder)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
91 except ValueError:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
92 pass
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
93
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
94
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
95 def enable():
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
96 sys.path_hooks.insert(0, _makefinder)
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
97
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
98
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
99 @contextlib.contextmanager
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
100 def deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
101 # This implementation is a bit different from Python 2's. Python 3
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
102 # maintains a per-package finder cache in sys.path_importer_cache (see
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
103 # PEP 302). This means that we can't just call disable + enable.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
104 # If we do that, in situations like:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
105 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
106 # demandimport.enable()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
107 # ...
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
108 # from foo.bar import mod1
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
109 # with demandimport.deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
110 # from foo.bar import mod2
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
111 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
112 # mod2 will be imported lazily. (The converse also holds -- whatever finder
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
113 # first gets cached will be used.)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
114 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
115 # Instead, have a global flag the LazyLoader can use.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
116 global _deactivated
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
117 demandenabled = isenabled()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
118 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
119 _deactivated = True
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
120 try:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
121 yield
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
122 finally:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
123 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
124 _deactivated = False