annotate hgdemandimport/demandimportpy3.py @ 43644:e01e0641f18a

cmdutil: use field names instead of field numbers on scmutil.status As part of my pytype adventures I want to make scmutil.status no longer a subclass of tuple. This is part of that process. Differential Revision: https://phab.mercurial-scm.org/D7396
author Augie Fackler <augie@google.com>
date Thu, 14 Nov 2019 15:26:17 -0500
parents 2d31ef3fb494
children c5e0a9b97b8a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
1 # demandimportpy3 - global demand-loading of modules for Mercurial
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
2 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
3 # Copyright 2017 Facebook Inc.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
4 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
7
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
8 """Lazy loading for Python 3.6 and above.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
9
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
10 This uses the new importlib finder/loader functionality available in Python 3.5
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
11 and up. The code reuses most of the mechanics implemented inside importlib.util,
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
12 but with a few additions:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
13
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
14 * Allow excluding certain modules from lazy imports.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
15 * Expose an interface that's substantially the same as demandimport for
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
16 Python 2.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
17
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
18 This also has some limitations compared to the Python 2 implementation:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
19
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
20 * Much of the logic is per-package, not per-module, so any packages loaded
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
21 before demandimport is enabled will not be lazily imported in the future. In
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
22 practice, we only expect builtins to be loaded before demandimport is
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
23 enabled.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
24 """
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
25
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
26 # This line is unnecessary, but it satisfies test-check-py3-compat.t.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
27 from __future__ import absolute_import
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
28
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
29 import contextlib
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
30 import importlib.abc
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
31 import importlib.machinery
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
32 import importlib.util
33898
3595e4e0ae57 demandimportpy3: update to pass import checker
Augie Fackler <raf@durin42.com>
parents: 33859
diff changeset
33 import sys
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
34
42474
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
35 from . import tracing
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
36
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
37 _deactivated = False
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
38
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
39
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
40 class _lazyloaderex(importlib.util.LazyLoader):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
41 """This is a LazyLoader except it also follows the _deactivated global and
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
42 the ignore list.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
43 """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
44
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
45 def exec_module(self, module):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
46 """Make the module load lazily."""
42474
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
47 with tracing.log('demandimport %s', module):
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
48 if _deactivated or module.__name__ in ignores:
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
49 self.loader.exec_module(module)
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
50 else:
adb636392b3f demandimport: add tracing coverage for Python 3
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
51 super().exec_module(module)
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
52
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
53
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
54 # This is 3.6+ because with Python 3.5 it isn't possible to lazily load
35524
fcb1ecf2bef7 hgdemandimport: use correct hyperlink to python-bug in comments (issue5765)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33898
diff changeset
55 # extensions. See the discussion in https://bugs.python.org/issue26186 for more.
43378
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
56 if sys.version_info[0:2] >= (3, 6):
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
57 _extensions_loader = _lazyloaderex.factory(
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
58 importlib.machinery.ExtensionFileLoader
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
59 )
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
60 else:
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
61 _extensions_loader = importlib.machinery.ExtensionFileLoader
2d31ef3fb494 demandimportpy3: only use lazy extension loader on Python 3.6+
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
62
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
63 _bytecode_loader = _lazyloaderex.factory(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
64 importlib.machinery.SourcelessFileLoader
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
65 )
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
66 _source_loader = _lazyloaderex.factory(importlib.machinery.SourceFileLoader)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
67
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
68
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
69 def _makefinder(path):
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
70 return importlib.machinery.FileFinder(
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
71 path,
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
72 # This is the order in which loaders are passed in in core Python.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
73 (_extensions_loader, importlib.machinery.EXTENSION_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
74 (_source_loader, importlib.machinery.SOURCE_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
75 (_bytecode_loader, importlib.machinery.BYTECODE_SUFFIXES),
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
76 )
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
77
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
78
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
79 ignores = set()
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
80
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
81
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
82 def init(ignoreset):
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
83 global ignores
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35524
diff changeset
84 ignores = ignoreset
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
85
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
86
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
87 def isenabled():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
88 return _makefinder in sys.path_hooks and not _deactivated
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
89
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
90
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
91 def disable():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
92 try:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
93 while True:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
94 sys.path_hooks.remove(_makefinder)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
95 except ValueError:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
96 pass
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
97
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
98
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
99 def enable():
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
100 sys.path_hooks.insert(0, _makefinder)
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
101
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42474
diff changeset
102
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
103 @contextlib.contextmanager
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
104 def deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
105 # This implementation is a bit different from Python 2's. Python 3
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
106 # maintains a per-package finder cache in sys.path_importer_cache (see
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
107 # PEP 302). This means that we can't just call disable + enable.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
108 # If we do that, in situations like:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
109 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
110 # demandimport.enable()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
111 # ...
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
112 # from foo.bar import mod1
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
113 # with demandimport.deactivated():
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
114 # from foo.bar import mod2
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
115 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
116 # mod2 will be imported lazily. (The converse also holds -- whatever finder
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
117 # first gets cached will be used.)
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
118 #
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
119 # Instead, have a global flag the LazyLoader can use.
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
120 global _deactivated
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
121 demandenabled = isenabled()
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
122 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
123 _deactivated = True
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
124 try:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
125 yield
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
126 finally:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
127 if demandenabled:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
128 _deactivated = False