Mercurial > hg
view contrib/check-py3-compat.py @ 29642:8960fcb76ca4 stable
demandimport: avoid infinite recursion at actual module importing (issue5304)
Before this patch, importing C module on Windows environment causes
infinite recursion call, if py2exe is used with -b2 option.
At importing C module "a.b", extra hooking by zipextimporter of py2exe
causes:
0. assumption before accessing "b" of "a":
- built-in module object is created for "a",
(= "a" is actually imported)
- _demandmod is created for "a.b" as a proxy object, and
(= "a.b" is not yet imported)
- an attribute "b" of "a" is initialized by the latter
1. invocation of __import__ via _hgextimport() in _demandmod._load()
for "a.b" implies _demandimport() for "a.b"
This is unintentional, because _demandmod might be returned by
_hgextimport() instead of built-in module object.
2. _demandimport() at (1) is invoked with not context of "a", but
context of zipextimporter
Just after invocation of _hgextimport() in _demandimport(), an
attribute "b" of the built-in module object for "a" is still
bound to the proxy object for "a.b", because context of "a" isn't
updated by actual importing "a.b". even though the built-in
module object for "a.b" already appears in sys.modules.
Therefore, chainmodules() returns _demandmod for "a.b", which is
gotten from the attribute "b" of "a".
3. processfromitem() on "a.b" causes _demandmod._load() for "a.b"
again
_demandimport() takes context of "a" in this case.
Therefore, attributes below are bound to built-in module object
for "a.b", as expected:
- "b" of built-in module object for "a"
- _module of _demandmod for "a.b"
4. but _demandimport() invoked at (1) returns _demandmod object
because _demandimport() just returns the object returned by
chainmodules() at (3) above.
5. then, _demandmod._load() causes infinite recursion call
_demandimport() returns _demandmod for "a.b", and it is "self" at
_demandmod._load().
To avoid infinite recursion at actual module importing, this patch
uses self._module, if _hgextimport() returns _demandmod itself. If
_demandmod._module isn't yet bound at this point, execution should be
aborted, because actual importing failed.
In this patch, _demandmod._module is examined not on _demandimport()
side, but on _demandmod._load() side, because:
- the former has some exit points
- only the latter uses _hgextimport(), except for _demandimport()
BTW, this issue occurs only in the code path for non .py/.pyc files in
zipextimporter (strictly speaking, in _memimporter) of py2exe.
Even if zipextimporter is enabled, .py/.pyc files are handled by
zipimporter, and it doesn't imply unintentional _demandimport() at
invocation of __import__ via _hgextimport().
author | FUJIWARA Katsunori <foozy@lares.dti.ne.jp> |
---|---|
date | Sun, 31 Jul 2016 05:39:59 +0900 |
parents | 1c22400db72d |
children | 8faac092bb0c |
line wrap: on
line source
#!/usr/bin/env python # # check-py3-compat - check Python 3 compatibility of Mercurial files # # Copyright 2015 Gregory Szorc <gregory.szorc@gmail.com> # # This software may be used and distributed according to the terms of the # GNU General Public License version 2 or any later version. from __future__ import absolute_import, print_function import ast import imp import os import sys import traceback def check_compat_py2(f): """Check Python 3 compatibility for a file with Python 2""" with open(f, 'rb') as fh: content = fh.read() root = ast.parse(content) # Ignore empty files. if not root.body: return futures = set() haveprint = False for node in ast.walk(root): if isinstance(node, ast.ImportFrom): if node.module == '__future__': futures |= set(n.name for n in node.names) elif isinstance(node, ast.Print): haveprint = True if 'absolute_import' not in futures: print('%s not using absolute_import' % f) if haveprint and 'print_function' not in futures: print('%s requires print_function' % f) def check_compat_py3(f): """Check Python 3 compatibility of a file with Python 3.""" with open(f, 'rb') as fh: content = fh.read() try: ast.parse(content) except SyntaxError as e: print('%s: invalid syntax: %s' % (f, e)) return # Try to import the module. # For now we only support mercurial.* and hgext.* modules because figuring # out module paths for things not in a package can be confusing. if f.startswith(('hgext/', 'mercurial/')) and not f.endswith('__init__.py'): assert f.endswith('.py') name = f.replace('/', '.')[:-3] with open(f, 'r') as fh: try: imp.load_module(name, fh, '', ('py', 'r', imp.PY_SOURCE)) except Exception as e: exc_type, exc_value, tb = sys.exc_info() # We walk the stack and ignore frames from our custom importer, # import mechanisms, and stdlib modules. This kinda/sorta # emulates CPython behavior in import.c while also attempting # to pin blame on a Mercurial file. for frame in reversed(traceback.extract_tb(tb)): if frame.name == '_call_with_frames_removed': continue if 'importlib' in frame.filename: continue if 'mercurial/__init__.py' in frame.filename: continue if frame.filename.startswith(sys.prefix): continue break if frame.filename: filename = os.path.basename(frame.filename) print('%s: error importing: <%s> %s (error at %s:%d)' % ( f, type(e).__name__, e, filename, frame.lineno)) else: print('%s: error importing module: <%s> %s (line %d)' % ( f, type(e).__name__, e, frame.lineno)) if __name__ == '__main__': if sys.version_info[0] == 2: fn = check_compat_py2 else: fn = check_compat_py3 for f in sys.argv[1:]: fn(f) sys.exit(0)