annotate hgdemandimport/__init__.py @ 45464:4a0ccbecbaa6

repo: avoid copying/updating a dict on every `repo.__getitem__` This has some mild performance benefits. I'm looking into a pathological case where one of our `hg log` invocations takes several seconds, and according to hyperfine this reduces the wall time of the entire operation (running in chg) from: ``` Time (mean ± σ): 7.390 s ± 0.106 s [User: 7.058 s, System: 0.271 s] Range (min … max): 7.300 s … 7.625 s ``` to: ``` Time (mean ± σ): 7.046 s ± 0.091 s [User: 6.714 s, System: 0.279 s] Range (min … max): 6.916 s … 7.169 s ``` Note: the log command is slow due to an issue in our custom stuff executing `repo[<arg>]` 298,800 times. This performance improvement is likely not noticeable during normal operation, but I don't feel like it's making the code more difficult to understand, and every small bit helps. Differential Revision: https://phab.mercurial-scm.org/D9022
author Kyle Lippincott <spectral@google.com>
date Fri, 11 Sep 2020 15:52:06 -0700
parents 81b4e7c866ec
children 07731064ac41
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
32420
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
1 # hgdemandimport - global demand-loading of modules for Mercurial
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
2 #
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
3 # Copyright 2017 Facebook Inc.
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
4 #
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
7
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
8 '''demandimport - automatic demand-loading of modules'''
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
9
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
10 # This is in a separate package from mercurial because in Python 3,
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
11 # demand loading is per-package. Keeping demandimport in the mercurial package
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
12 # would disable demand loading for any modules in mercurial.
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
13
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
14 from __future__ import absolute_import
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
15
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
16 import os
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
17 import sys
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
18
32423
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents: 32422
diff changeset
19 if sys.version_info[0] >= 3:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents: 32422
diff changeset
20 from . import demandimportpy3 as demandimport
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents: 32422
diff changeset
21 else:
859496bb6db3 demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents: 32422
diff changeset
22 from . import demandimportpy2 as demandimport
32420
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
23
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
24 # Full module names which can't be lazy imported.
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
25 # Extensions can add to this set.
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
26 IGNORES = {
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
27 '__future__',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
28 '_hashlib',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
29 # ImportError during pkg_resources/__init__.py:fixup_namespace_package
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
30 '_imp',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
31 '_xmlplus',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
32 'fcntl',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
33 'nt', # pathlib2 tests the existence of built-in 'nt' module
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
34 'win32com.gen_py',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
35 'win32com.shell', # 'appdirs' tries to import win32com.shell
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
36 '_winreg', # 2.7 mimetypes needs immediate ImportError
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
37 'pythoncom',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
38 # imported by tarfile, not available under Windows
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
39 'pwd',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
40 'grp',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
41 # imported by profile, itself imported by hotshot.stats,
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
42 # not available under Windows
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
43 'resource',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
44 # this trips up many extension authors
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
45 'gtk',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
46 # setuptools' pkg_resources.py expects "from __main__ import x" to
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
47 # raise ImportError if x not defined
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
48 '__main__',
45415
81b4e7c866ec hgdemandimport: bypass demandimport for _ast module (issue6407)
Augie Fackler <augie@google.com>
parents: 45047
diff changeset
49 '_ast', # https://bugs.python.org/issue41631
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
50 '_ssl', # conditional imports in the stdlib, issue1964
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
51 '_sre', # issue4920
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
52 'rfc822',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
53 'mimetools',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
54 'sqlalchemy.events', # has import-time side effects (issue5085)
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
55 # setuptools 8 expects this module to explode early when not on windows
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
56 'distutils.msvc9compiler',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
57 '__builtin__',
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
58 'builtins',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
59 'urwid.command_map', # for pudb
45047
40120de810ba demandimport: ignore `lzma` module for demandimport
Manuel Jacob <me@manueljacob.de>
parents: 43076
diff changeset
60 'lzma',
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
61 }
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
62
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
63 _pypy = '__pypy__' in sys.builtin_module_names
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
64
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
65 if _pypy:
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
66 # _ctypes.pointer is shadowed by "from ... import pointer" (PyPy 5)
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
67 IGNORES.add('_ctypes.pointer')
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
68
37843
670eb4fa1b86 demandimport: make module ignores a set (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 33860
diff changeset
69 demandimport.init(IGNORES)
32422
f37f9499fea8 demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents: 32420
diff changeset
70
32420
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
71 # Re-export.
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
72 isenabled = demandimport.isenabled
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
73 disable = demandimport.disable
0906b85bf222 demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff changeset
74 deactivated = demandimport.deactivated
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
75
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
76
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
77 def enable():
33860
3cfc9070245f demandimport: disable if chg is being used
Jun Wu <quark@fb.com>
parents: 33859
diff changeset
78 # chg pre-imports modules so do not enable demandimport for it
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
79 if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
80 'CHGINTERNALMARK' not in os.environ
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
81 and os.environ.get('HGDEMANDIMPORT') != 'disable'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 37843
diff changeset
82 ):
33859
8fb5212652ec demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents: 32423
diff changeset
83 demandimport.enable()