Mercurial > hg
annotate hgdemandimport/__init__.py @ 36875:16499427f6de
hgweb: refactor fake file object proxy for archiving
Python's zip file writer operates on a file object. When doing work,
it periodically calls write(), flush(), and tell() on that object.
In WSGI contexts, the start_response function returns a write()
function. That's a function to write data, not a full file object.
So, when the archival code was first introduced in 2b03c6733efa in
2006, someone invented a proxy "tellable" type that wrapped a file
object like object and kept track of write count so it could
implement tell() and satisfy zipfile's needs.
When our archival code runs, it attempts to tell() the destination
and if that fails, converts it to a "tellable" instance. Our WSGI
application passes the "wsgirequest" instance to the archival
function. It fails the tell() test and is converted to a "tellable."
It's worth noting that "wsgirequest" implements flush(), so
"tellable" doesn't.
This hackery all seems very specific to the WSGI code. So this commit
moves the "tellable" type and the conversion of the destination file
object into the WSGI code. There's a chance some other caller may be
passing a file object like object that doesn't implement tell(). But
I doubt it.
As part of the refactor, our new type implements flush() and doesn't
implement __getattr__. Given the intended limited use of this type,
I want things to fail fast if there is an attempt to access attributes
because I think it is important to document which attributes are being
used for what purposes.
Differential Revision: https://phab.mercurial-scm.org/D2791
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sat, 10 Mar 2018 16:17:51 -0800 |
parents | 3cfc9070245f |
children | 670eb4fa1b86 |
rev | line source |
---|---|
32420
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
1 # hgdemandimport - global demand-loading of modules for Mercurial |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
2 # |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
3 # Copyright 2017 Facebook Inc. |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
4 # |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
7 |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
8 '''demandimport - automatic demand-loading of modules''' |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
9 |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
10 # This is in a separate package from mercurial because in Python 3, |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
11 # demand loading is per-package. Keeping demandimport in the mercurial package |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
12 # would disable demand loading for any modules in mercurial. |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
13 |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
14 from __future__ import absolute_import |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
15 |
33859
8fb5212652ec
demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents:
32423
diff
changeset
|
16 import os |
32422
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
17 import sys |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
18 |
32423
859496bb6db3
demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
32422
diff
changeset
|
19 if sys.version_info[0] >= 3: |
859496bb6db3
demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
32422
diff
changeset
|
20 from . import demandimportpy3 as demandimport |
859496bb6db3
demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
32422
diff
changeset
|
21 else: |
859496bb6db3
demandimport: add python 3 implementation
Siddharth Agarwal <sid0@fb.com>
parents:
32422
diff
changeset
|
22 from . import demandimportpy2 as demandimport |
32420
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
23 |
32422
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
24 # Extensions can add to this list if necessary. |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
25 ignore = [ |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
26 '__future__', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
27 '_hashlib', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
28 # ImportError during pkg_resources/__init__.py:fixup_namespace_package |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
29 '_imp', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
30 '_xmlplus', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
31 'fcntl', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
32 'nt', # pathlib2 tests the existence of built-in 'nt' module |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
33 'win32com.gen_py', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
34 'win32com.shell', # 'appdirs' tries to import win32com.shell |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
35 '_winreg', # 2.7 mimetypes needs immediate ImportError |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
36 'pythoncom', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
37 # imported by tarfile, not available under Windows |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
38 'pwd', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
39 'grp', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
40 # imported by profile, itself imported by hotshot.stats, |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
41 # not available under Windows |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
42 'resource', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
43 # this trips up many extension authors |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
44 'gtk', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
45 # setuptools' pkg_resources.py expects "from __main__ import x" to |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
46 # raise ImportError if x not defined |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
47 '__main__', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
48 '_ssl', # conditional imports in the stdlib, issue1964 |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
49 '_sre', # issue4920 |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
50 'rfc822', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
51 'mimetools', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
52 'sqlalchemy.events', # has import-time side effects (issue5085) |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
53 # setuptools 8 expects this module to explode early when not on windows |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
54 'distutils.msvc9compiler', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
55 '__builtin__', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
56 'builtins', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
57 'urwid.command_map', # for pudb |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
58 ] |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
59 |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
60 _pypy = '__pypy__' in sys.builtin_module_names |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
61 |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
62 if _pypy: |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
63 ignore.extend([ |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
64 # _ctypes.pointer is shadowed by "from ... import pointer" (PyPy 5) |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
65 '_ctypes.pointer', |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
66 ]) |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
67 |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
68 demandimport.init(ignore) |
f37f9499fea8
demandimport: move ignore list to __init__.py
Siddharth Agarwal <sid0@fb.com>
parents:
32420
diff
changeset
|
69 |
32420
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
70 # Re-export. |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
71 isenabled = demandimport.isenabled |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
72 disable = demandimport.disable |
0906b85bf222
demandimport: move to separate package
Siddharth Agarwal <sid0@fb.com>
parents:
diff
changeset
|
73 deactivated = demandimport.deactivated |
33859
8fb5212652ec
demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents:
32423
diff
changeset
|
74 |
8fb5212652ec
demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents:
32423
diff
changeset
|
75 def enable(): |
33860
3cfc9070245f
demandimport: disable if chg is being used
Jun Wu <quark@fb.com>
parents:
33859
diff
changeset
|
76 # chg pre-imports modules so do not enable demandimport for it |
3cfc9070245f
demandimport: disable if chg is being used
Jun Wu <quark@fb.com>
parents:
33859
diff
changeset
|
77 if ('CHGINTERNALMARK' not in os.environ |
3cfc9070245f
demandimport: disable if chg is being used
Jun Wu <quark@fb.com>
parents:
33859
diff
changeset
|
78 and os.environ.get('HGDEMANDIMPORT') != 'disable'): |
33859
8fb5212652ec
demandimport: move HGDEMANDIMPORT test to __init__.py
Jun Wu <quark@fb.com>
parents:
32423
diff
changeset
|
79 demandimport.enable() |