contrib/import-checker.py
author Martin von Zweigbergk <martinvonz@google.com>
Thu, 07 Jan 2021 12:26:32 -0800
changeset 46272 a68d3386138c
parent 45830 c102b704edb5
child 47377 26127236b229
permissions -rwxr-xr-x
shelve: raise more specific errors Differential Revision: https://phab.mercurial-scm.org/D9699
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
45830
c102b704edb5 global: use python3 in shebangs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44413
diff changeset
     1
#!/usr/bin/env python3
26954
f804bf27439b import-checker: make it executable for convenience
Yuya Nishihara <yuya@tcha.org>
parents: 26781
diff changeset
     2
28703
a274c4f9087a py3: use print_function in import-checker
timeless <timeless@mozdev.org>
parents: 28702
diff changeset
     3
from __future__ import absolute_import, print_function
28702
e44f671018e3 py3: use absolute_import in import-checker
timeless <timeless@mozdev.org>
parents: 28700
diff changeset
     4
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
     5
import ast
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
     6
import collections
43415
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
     7
import io
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
     8
import os
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
     9
import sys
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    10
20198
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
    11
# Import a minimal set of stdlib modules needed for list_stdlib_modules()
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
    12
# to work when run from a virtualenv.  The modules were chosen empirically
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
    13
# so that the return value matches the return value without virtualenv.
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    14
if True:  # disable lexical sorting checks
33876
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
    15
    try:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
    16
        import BaseHTTPServer as basehttpserver
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
    17
    except ImportError:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
    18
        basehttpserver = None
29211
b42c2a66a698 py3: make contrib/import-checker.py get along with itself
Yuya Nishihara <yuya@tcha.org>
parents: 29208
diff changeset
    19
    import zlib
20198
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
    20
40095
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
    21
import testparseutil
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
    22
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    23
# Whitelist of modules that symbols can be directly imported from.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    24
allowsymbolimports = (
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    25
    '__future__',
33919
5ed0be4d9df9 contrib: add bzrlib to list of packages from which we import symbols
Augie Fackler <raf@durin42.com>
parents: 33915
diff changeset
    26
    'bzrlib',
33915
2d64b2f1787b contrib: allow symbol imports from hgclient for tests
Augie Fackler <raf@durin42.com>
parents: 33897
diff changeset
    27
    'hgclient',
33893
c9cf69d0c3b9 contrib: allow importing "symbols" from mercurial
Augie Fackler <raf@durin42.com>
parents: 33890
diff changeset
    28
    'mercurial',
27018
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request
Yuya Nishihara <yuya@tcha.org>
parents: 26965
diff changeset
    29
    'mercurial.hgweb.common',
e5be48dd8215 import-checker: allow symbol imports from hgweb.common and .request
Yuya Nishihara <yuya@tcha.org>
parents: 26965
diff changeset
    30
    'mercurial.hgweb.request',
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    31
    'mercurial.i18n',
42813
268662aac075 interfaces: create a new folder for interfaces and move repository.py in it
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 42728
diff changeset
    32
    'mercurial.interfaces',
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    33
    'mercurial.node',
43084
c2e284cee333 import-checker: allow symbol imports from mercurial.pycompat
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43075
diff changeset
    34
    'mercurial.pycompat',
39329
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents: 38797
diff changeset
    35
    # for revlog to re-export constant to extensions
729082bb9938 revlog: split constants into a new `revlogutils.constants` module
Boris Feld <boris.feld@octobus.net>
parents: 38797
diff changeset
    36
    'mercurial.revlogutils.constants',
42728
ca5ca3badd3c flagutil: create a `mercurial.revlogutils.flagutil` module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42388
diff changeset
    37
    'mercurial.revlogutils.flagutil',
32507
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    38
    # for cffi modules to re-export pure functions
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    39
    'mercurial.pure.base85',
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    40
    'mercurial.pure.bdiff',
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    41
    'mercurial.pure.mpatch',
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    42
    'mercurial.pure.osutil',
95085d747db8 import-checker: allow importing symbols from pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32420
diff changeset
    43
    'mercurial.pure.parsers',
34395
41401f502c83 tests: disable lints on mercurial/thirdparty
Siddharth Agarwal <sid0@fb.com>
parents: 34038
diff changeset
    44
    # third-party imports should be directly imported
41401f502c83 tests: disable lints on mercurial/thirdparty
Siddharth Agarwal <sid0@fb.com>
parents: 34038
diff changeset
    45
    'mercurial.thirdparty',
38797
8751d1e2a7ff util: create a context manager to handle timing
Martijn Pieters <mj@zopatista.com>
parents: 37711
diff changeset
    46
    'mercurial.thirdparty.attr',
37180
922b3fae9c7d setup: register zope.interface packages and compile C extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34395
diff changeset
    47
    'mercurial.thirdparty.zope',
922b3fae9c7d setup: register zope.interface packages and compile C extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34395
diff changeset
    48
    'mercurial.thirdparty.zope.interface',
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    49
)
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    50
32419
d02888308235 import-checker: add a way to directly import certain symbols
Siddharth Agarwal <sid0@fb.com>
parents: 32374
diff changeset
    51
# Whitelist of symbols that can be directly imported.
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    52
directsymbols = ('demandimport',)
32419
d02888308235 import-checker: add a way to directly import certain symbols
Siddharth Agarwal <sid0@fb.com>
parents: 32374
diff changeset
    53
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    54
# Modules that must be aliased because they are commonly confused with
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    55
# common variables and can create aliasing and readability issues.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    56
requirealias = {
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    57
    'ui': 'uimod',
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    58
}
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    59
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    60
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    61
def usingabsolute(root):
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    62
    """Whether absolute imports are being used."""
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    63
    if sys.version_info[0] >= 3:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    64
        return True
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    65
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    66
    for node in ast.walk(root):
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    67
        if isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    68
            if node.module == '__future__':
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    69
                for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    70
                    if n.name == 'absolute_import':
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    71
                        return True
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    72
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    73
    return False
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
    74
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    75
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    76
def walklocal(root):
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    77
    """Recursively yield all descendant nodes but not in a different scope"""
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    78
    todo = collections.deque(ast.iter_child_nodes(root))
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    79
    yield root, False
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    80
    while todo:
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    81
        node = todo.popleft()
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    82
        newscope = isinstance(node, ast.FunctionDef)
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    83
        if not newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    84
            todo.extend(ast.iter_child_nodes(node))
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    85
        yield node, newscope
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
    86
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    87
32374
194b0f781132 import-checker: drop workaround for pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32372
diff changeset
    88
def dotted_name_of_path(path):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    89
    """Given a relative path to a source file, return its dotted module name.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    90
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    91
    >>> dotted_name_of_path('mercurial/error.py')
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    92
    'mercurial.error'
20383
4990abb4729d import-checker: fix names of dynamically loaded modules
Mads Kiilerich <madski@unity3d.com>
parents: 20238
diff changeset
    93
    >>> dotted_name_of_path('zlibmodule.so')
4990abb4729d import-checker: fix names of dynamically loaded modules
Mads Kiilerich <madski@unity3d.com>
parents: 20238
diff changeset
    94
    'zlib'
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
    95
    """
27620
0c60843b55b5 import-checker: normalize directory separator to get module name on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 27520
diff changeset
    96
    parts = path.replace(os.sep, '/').split('/')
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
    97
    parts[-1] = parts[-1].split('.', 1)[0]  # remove .py and .so and .ARCH.so
20383
4990abb4729d import-checker: fix names of dynamically loaded modules
Mads Kiilerich <madski@unity3d.com>
parents: 20238
diff changeset
    98
    if parts[-1].endswith('module'):
4990abb4729d import-checker: fix names of dynamically loaded modules
Mads Kiilerich <madski@unity3d.com>
parents: 20238
diff changeset
    99
        parts[-1] = parts[-1][:-6]
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   100
    return '.'.join(parts)
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   101
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   102
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   103
def fromlocalfunc(modulename, localmods):
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   104
    """Get a function to examine which locally defined module the
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   105
    target source imports via a specified name.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   106
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   107
    `modulename` is an `dotted_name_of_path()`-ed source file path,
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   108
    which may have `.__init__` at the end of it, of the target source.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   109
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   110
    `localmods` is a set of absolute `dotted_name_of_path()`-ed source file
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   111
    paths of locally defined (= Mercurial specific) modules.
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   112
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   113
    This function assumes that module names not existing in
26781
1aee2ab0f902 spelling: trivial spell checking
Mads Kiilerich <madski@unity3d.com>
parents: 26221
diff changeset
   114
    `localmods` are from the Python standard library.
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   115
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   116
    This function returns the function, which takes `name` argument,
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   117
    and returns `(absname, dottedpath, hassubmod)` tuple if `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   118
    matches against locally defined module. Otherwise, it returns
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   119
    False.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   120
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   121
    It is assumed that `name` doesn't have `.__init__`.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   122
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   123
    `absname` is an absolute module name of specified `name`
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   124
    (e.g. "hgext.convert"). This can be used to compose prefix for sub
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   125
    modules or so.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   126
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   127
    `dottedpath` is a `dotted_name_of_path()`-ed source file path
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   128
    (e.g. "hgext.convert.__init__") of `name`. This is used to look
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   129
    module up in `localmods` again.
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   130
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   131
    `hassubmod` is whether it may have sub modules under it (for
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   132
    convenient, even though this is also equivalent to "absname !=
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   133
    dottednpath")
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   134
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   135
    >>> localmods = {'foo.__init__', 'foo.foo1',
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   136
    ...              'foo.bar.__init__', 'foo.bar.bar1',
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   137
    ...              'baz.__init__', 'baz.baz1'}
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   138
    >>> fromlocal = fromlocalfunc('foo.xxx', localmods)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   139
    >>> # relative
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   140
    >>> fromlocal('foo1')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   141
    ('foo.foo1', 'foo.foo1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   142
    >>> fromlocal('bar')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   143
    ('foo.bar', 'foo.bar.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   144
    >>> fromlocal('bar.bar1')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   145
    ('foo.bar.bar1', 'foo.bar.bar1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   146
    >>> # absolute
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   147
    >>> fromlocal('baz')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   148
    ('baz', 'baz.__init__', True)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   149
    >>> fromlocal('baz.baz1')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   150
    ('baz.baz1', 'baz.baz1', False)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   151
    >>> # unknown = maybe standard library
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   152
    >>> fromlocal('os')
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   153
    False
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   154
    >>> fromlocal(None, 1)
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   155
    ('foo', 'foo.__init__', True)
29122
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   156
    >>> fromlocal('foo1', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   157
    ('foo.foo1', 'foo.foo1', False)
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   158
    >>> fromlocal2 = fromlocalfunc('foo.xxx.yyy', localmods)
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   159
    >>> fromlocal2(None, 2)
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   160
    ('foo', 'foo.__init__', True)
29122
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   161
    >>> fromlocal2('bar2', 1)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   162
    False
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   163
    >>> fromlocal2('bar', 2)
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   164
    ('foo.bar', 'foo.bar.__init__', True)
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   165
    """
33890
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   166
    if not isinstance(modulename, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   167
        modulename = modulename.decode('ascii')
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   168
    prefix = '.'.join(modulename.split('.')[:-1])
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   169
    if prefix:
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   170
        prefix += '.'
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   171
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   172
    def fromlocal(name, level=0):
29374
7712fcde2d56 import-checker: increase portability for python 2.6.x
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29234
diff changeset
   173
        # name is false value when relative imports are used.
7712fcde2d56 import-checker: increase portability for python 2.6.x
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 29234
diff changeset
   174
        if not name:
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   175
            # If relative imports are used, level must not be absolute.
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   176
            assert level > 0
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   177
            candidates = ['.'.join(modulename.split('.')[:-level])]
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   178
        else:
29122
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   179
            if not level:
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   180
                # Check relative name first.
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   181
                candidates = [prefix + name, name]
660d8d4ec7aa import-checker: recognize relative imports from parents of current package
liscju <piotr.listkiewicz@gmail.com>
parents: 28922
diff changeset
   182
            else:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   183
                candidates = [
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   184
                    '.'.join(modulename.split('.')[:-level]) + '.' + name
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   185
                ]
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   186
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   187
        for n in candidates:
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   188
            if n in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   189
                return (n, n, False)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   190
            dottedpath = n + '.__init__'
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   191
            if dottedpath in localmods:
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   192
                return (n, dottedpath, True)
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   193
        return False
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   194
25173
7358b5d9991e import-checker: add utility to examine what module is imported easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25064
diff changeset
   195
    return fromlocal
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   196
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   197
32509
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   198
def populateextmods(localmods):
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   199
    """Populate C extension modules based on pure modules"""
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   200
    newlocalmods = set(localmods)
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   201
    for n in localmods:
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   202
        if n.startswith('mercurial.pure.'):
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   203
            m = n[len('mercurial.pure.') :]
32509
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   204
            newlocalmods.add('mercurial.cext.' + m)
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   205
            newlocalmods.add('mercurial.cffi._' + m)
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   206
    return newlocalmods
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   207
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   208
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   209
def list_stdlib_modules():
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   210
    """List the modules present in the stdlib.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   211
33876
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   212
    >>> py3 = sys.version_info[0] >= 3
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   213
    >>> mods = set(list_stdlib_modules())
33876
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   214
    >>> 'BaseHTTPServer' in mods or py3
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   215
    True
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   216
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   217
    os.path isn't really a module, so it's missing:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   218
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   219
    >>> 'os.path' in mods
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   220
    False
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   221
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   222
    sys requires special treatment, because it's baked into the
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   223
    interpreter, but it should still appear:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   224
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   225
    >>> 'sys' in mods
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   226
    True
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   227
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   228
    >>> 'collections' in mods
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   229
    True
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   230
33876
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   231
    >>> 'cStringIO' in mods or py3
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   232
    True
29395
4c8026babe8c import-checker: ensure cffi is always a system module
Augie Fackler <raf@durin42.com>
parents: 29374
diff changeset
   233
4c8026babe8c import-checker: ensure cffi is always a system module
Augie Fackler <raf@durin42.com>
parents: 29374
diff changeset
   234
    >>> 'cffi' in mods
4c8026babe8c import-checker: ensure cffi is always a system module
Augie Fackler <raf@durin42.com>
parents: 29374
diff changeset
   235
    True
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   236
    """
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   237
    for m in sys.builtin_module_names:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   238
        yield m
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   239
    # These modules only exist on windows, but we should always
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   240
    # consider them stdlib.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   241
    for m in ['msvcrt', '_winreg']:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   242
        yield m
33894
c856cb1c29be contrib: inform import checker that __builtin__ is a thing
Augie Fackler <raf@durin42.com>
parents: 33893
diff changeset
   243
    yield '__builtin__'
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   244
    yield 'builtins'  # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   245
    yield 'importlib.abc'  # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   246
    yield 'importlib.machinery'  # python3 only
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   247
    yield 'importlib.util'  # python3 only
24669
fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules
Matt Harbison <matt_harbison@yahoo.com>
parents: 24668
diff changeset
   248
    for m in 'fcntl', 'grp', 'pwd', 'termios':  # Unix only
fbdbff1b486a import-checker: force 'fcntl', 'grp', 'pwd', and 'termios' to stdlib modules
Matt Harbison <matt_harbison@yahoo.com>
parents: 24668
diff changeset
   249
        yield m
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   250
    for m in 'cPickle', 'datetime':  # in Python (not C) on PyPy
28713
806d260c6f3b tests: fix builtin module test on pypy
Maciej Fijalkowski <fijall@gmail.com>
parents: 28704
diff changeset
   251
        yield m
29395
4c8026babe8c import-checker: ensure cffi is always a system module
Augie Fackler <raf@durin42.com>
parents: 29374
diff changeset
   252
    for m in ['cffi']:
4c8026babe8c import-checker: ensure cffi is always a system module
Augie Fackler <raf@durin42.com>
parents: 29374
diff changeset
   253
        yield m
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 32212
diff changeset
   254
    stdlib_prefixes = {sys.prefix, sys.exec_prefix}
20198
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   255
    # We need to supplement the list of prefixes for the search to work
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   256
    # when run from within a virtualenv.
33876
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   257
    for mod in (basehttpserver, zlib):
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   258
        if mod is None:
998fad4b3072 contrib: work around some modules not existing on Py3 in import checker
Augie Fackler <raf@durin42.com>
parents: 32595
diff changeset
   259
            continue
20198
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   260
        try:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   261
            # Not all module objects have a __file__ attribute.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   262
            filename = mod.__file__
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   263
        except AttributeError:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   264
            continue
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   265
        dirname = os.path.dirname(filename)
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   266
        for prefix in stdlib_prefixes:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   267
            if dirname.startswith(prefix):
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   268
                # Then this directory is redundant.
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   269
                break
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   270
        else:
f5393a9dc4e5 import-checker: make test-module-imports.t work using virtualenv (issue4129)
Chris Jerdonek <chris.jerdonek@gmail.com>
parents: 20197
diff changeset
   271
            stdlib_prefixes.add(dirname)
40687
dd028bca9221 tests: make test-check-module-imports more robust
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 40481
diff changeset
   272
    sourceroot = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   273
    for libpath in sys.path:
40687
dd028bca9221 tests: make test-check-module-imports more robust
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 40481
diff changeset
   274
        # We want to walk everything in sys.path that starts with something in
dd028bca9221 tests: make test-check-module-imports more robust
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 40481
diff changeset
   275
        # stdlib_prefixes, but not directories from the hg sources.
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   276
        if os.path.abspath(libpath).startswith(sourceroot) or not any(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   277
            libpath.startswith(p) for p in stdlib_prefixes
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   278
        ):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   279
            continue
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   280
        for top, dirs, files in os.walk(libpath):
25733
f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists
Yuya Nishihara <yuya@tcha.org>
parents: 25731
diff changeset
   281
            for i, d in reversed(list(enumerate(dirs))):
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   282
                if (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   283
                    not os.path.exists(os.path.join(top, d, '__init__.py'))
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   284
                    or top == libpath
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   285
                    and d in ('hgdemandimport', 'hgext', 'mercurial')
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   286
                ):
25733
f99c066f5f9a import-checker: recurse into subtree of sys.path only if __init__.py exists
Yuya Nishihara <yuya@tcha.org>
parents: 25731
diff changeset
   287
                    del dirs[i]
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   288
            for name in files:
26221
ae65b1b4cb46 import-checker: use modern .endswith for multiple suffixes
Augie Fackler <augie@google.com>
parents: 26166
diff changeset
   289
                if not name.endswith(('.py', '.so', '.pyc', '.pyo', '.pyd')):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   290
                    continue
27621
39845b064041 import-checker: list package directory as stdlib module
Yuya Nishihara <yuya@tcha.org>
parents: 27620
diff changeset
   291
                if name.startswith('__init__.py'):
39845b064041 import-checker: list package directory as stdlib module
Yuya Nishihara <yuya@tcha.org>
parents: 27620
diff changeset
   292
                    full_path = top
39845b064041 import-checker: list package directory as stdlib module
Yuya Nishihara <yuya@tcha.org>
parents: 27620
diff changeset
   293
                else:
39845b064041 import-checker: list package directory as stdlib module
Yuya Nishihara <yuya@tcha.org>
parents: 27620
diff changeset
   294
                    full_path = os.path.join(top, name)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   295
                rel_path = full_path[len(libpath) + 1 :]
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   296
                mod = dotted_name_of_path(rel_path)
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   297
                yield mod
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   298
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   299
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   300
stdlib_modules = set(list_stdlib_modules())
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   301
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   302
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   303
def imported_modules(source, modulename, f, localmods, ignore_nested=False):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   304
    """Given the source of a file as a string, yield the names
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   305
    imported by that file.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   306
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   307
    Args:
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   308
      source: The python source to examine as a string.
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   309
      modulename: of specified python source (may have `__init__`)
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   310
      localmods: set of locally defined module names (may have `__init__`)
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   311
      ignore_nested: If true, import statements that do not start in
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   312
                     column zero will be ignored.
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   313
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   314
    Returns:
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   315
      A list of absolute module names imported by the given source.
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   316
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   317
    >>> f = 'foo/xxx.py'
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   318
    >>> modulename = 'foo.xxx'
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   319
    >>> localmods = {'foo.__init__': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   320
    ...              'foo.foo1': True, 'foo.foo2': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   321
    ...              'foo.bar.__init__': True, 'foo.bar.bar1': True,
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   322
    ...              'baz.__init__': True, 'baz.baz1': True }
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   323
    >>> # standard library (= not locally defined ones)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   324
    >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   325
    ...        'from stdlib1 import foo, bar; import stdlib2',
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   326
    ...        modulename, f, localmods))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   327
    []
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   328
    >>> # relative importing
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   329
    >>> sorted(imported_modules(
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   330
    ...        'import foo1; from bar import bar1',
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   331
    ...        modulename, f, localmods))
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   332
    ['foo.bar.bar1', 'foo.foo1']
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   333
    >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   334
    ...        'from bar.bar1 import name1, name2, name3',
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   335
    ...        modulename, f, localmods))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   336
    ['foo.bar.bar1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   337
    >>> # absolute importing
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   338
    >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   339
    ...        'from baz import baz1, name1',
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   340
    ...        modulename, f, localmods))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   341
    ['baz.__init__', 'baz.baz1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   342
    >>> # mixed importing, even though it shouldn't be recommended
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   343
    >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   344
    ...        'import stdlib, foo1, baz',
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   345
    ...        modulename, f, localmods))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   346
    ['baz.__init__', 'foo.foo1']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   347
    >>> # ignore_nested
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   348
    >>> sorted(imported_modules(
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   349
    ... '''import foo
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   350
    ... def wat():
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   351
    ...     import bar
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   352
    ... ''', modulename, f, localmods))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   353
    ['foo.__init__', 'foo.bar.__init__']
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   354
    >>> sorted(imported_modules(
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   355
    ... '''import foo
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   356
    ... def wat():
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   357
    ...     import bar
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   358
    ... ''', modulename, f, localmods, ignore_nested=True))
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   359
    ['foo.__init__']
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   360
    """
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   361
    fromlocal = fromlocalfunc(modulename, localmods)
28921
02ee31a50002 import-checker: track filenames for SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28920
diff changeset
   362
    for node in ast.walk(ast.parse(source, f)):
20037
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   363
        if ignore_nested and getattr(node, 'col_offset', 0) > 0:
957b43371928 import-checker: ignore nested imports
Augie Fackler <raf@durin42.com>
parents: 20036
diff changeset
   364
            continue
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   365
        if isinstance(node, ast.Import):
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   366
            for n in node.names:
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   367
                found = fromlocal(n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   368
                if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   369
                    # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   370
                    continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   371
                yield found[1]
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   372
        elif isinstance(node, ast.ImportFrom):
25701
1f88c0f6ff5a import-checker: resolve relative imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25660
diff changeset
   373
            found = fromlocal(node.module, node.level)
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   374
            if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   375
                # this should import standard library
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   376
                continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   377
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   378
            absname, dottedpath, hassubmod = found
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   379
            if not hassubmod:
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   380
                # "dottedpath" is not a package; must be imported
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   381
                yield dottedpath
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   382
                # examination of "node.names" should be redundant
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   383
                # e.g.: from mercurial.node import nullid, nullrev
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   384
                continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   385
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   386
            modnotfound = False
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   387
            prefix = absname + '.'
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   388
            for n in node.names:
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   389
                found = fromlocal(prefix + n.name)
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   390
                if not found:
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   391
                    # this should be a function or a property of "node.module"
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   392
                    modnotfound = True
25174
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   393
                    continue
86298718b01c import-checker: make imported_modules yield absolute dotted_name_of_path
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25173
diff changeset
   394
                yield found[1]
44413
4cabeea6d214 hgext: start building a library for simple hooks
Joerg Sonnenberger <joerg@bec.de>
parents: 43954
diff changeset
   395
            if modnotfound and dottedpath != modulename:
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   396
                # "dottedpath" is a package, but imported because of non-module
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   397
                # lookup
44413
4cabeea6d214 hgext: start building a library for simple hooks
Joerg Sonnenberger <joerg@bec.de>
parents: 43954
diff changeset
   398
                # specifically allow "from . import foo" from __init__.py
26964
5abba2c92da3 import-checker: allow import of child modules from package root
Yuya Nishihara <yuya@tcha.org>
parents: 26956
diff changeset
   399
                yield dottedpath
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   400
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   401
27272
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages
Yuya Nishihara <yuya@tcha.org>
parents: 27018
diff changeset
   402
def verify_import_convention(module, source, localmods):
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   403
    """Verify imports match our established coding convention.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   404
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   405
    We have 2 conventions: legacy and modern. The modern convention is in
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   406
    effect when using absolute imports.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   407
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   408
    The legacy convention only looks for mixed imports. The modern convention
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   409
    is much more thorough.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   410
    """
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   411
    root = ast.parse(source)
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   412
    absolute = usingabsolute(root)
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   413
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   414
    if absolute:
27272
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages
Yuya Nishihara <yuya@tcha.org>
parents: 27018
diff changeset
   415
        return verify_modern_convention(module, root, localmods)
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   416
    else:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   417
        return verify_stdlib_on_own_line(root)
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   418
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   419
27272
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages
Yuya Nishihara <yuya@tcha.org>
parents: 27018
diff changeset
   420
def verify_modern_convention(module, root, localmods, root_col_offset=0):
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   421
    """Verify a file conforms to the modern import convention rules.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   422
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   423
    The rules of the modern convention are:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   424
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   425
    * Ordering is stdlib followed by local imports. Each group is lexically
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   426
      sorted.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   427
    * Importing multiple modules via "import X, Y" is not allowed: use
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   428
      separate import statements.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   429
    * Importing multiple modules via "from X import ..." is allowed if using
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   430
      parenthesis and one entry per line.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   431
    * Only 1 relative import statement per import level ("from .", "from ..")
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   432
      is allowed.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   433
    * Relative imports from higher levels must occur before lower levels. e.g.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   434
      "from .." must be before "from .".
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   435
    * Imports from peer packages should use relative import (e.g. do not
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   436
      "import mercurial.foo" from a "mercurial.*" module).
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   437
    * Symbols can only be imported from specific modules (see
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   438
      `allowsymbolimports`). For other modules, first import the module then
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   439
      assign the symbol to a module-level variable. In addition, these imports
29208
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   440
      must be performed before other local imports. This rule only
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   441
      applies to import statements outside of any blocks.
34038
bc2535238de2 import-checker: allow relative import a module being checked
Jun Wu <quark@fb.com>
parents: 33920
diff changeset
   442
    * Relative imports from the standard library are not allowed, unless that
bc2535238de2 import-checker: allow relative import a module being checked
Jun Wu <quark@fb.com>
parents: 33920
diff changeset
   443
      library is also a local module.
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   444
    * Certain modules must be aliased to alternate names to avoid aliasing
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   445
      and readability problems. See `requirealias`.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   446
    """
33890
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   447
    if not isinstance(module, str):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   448
        module = module.decode('ascii')
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   449
    topmodule = module.split('.')[0]
27272
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages
Yuya Nishihara <yuya@tcha.org>
parents: 27018
diff changeset
   450
    fromlocal = fromlocalfunc(module, localmods)
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   451
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   452
    # Whether a local/non-stdlib import has been performed.
28330
f3fb24e36d61 import-checker: report local with stdlib late warning
timeless <timeless@mozdev.org>
parents: 27621
diff changeset
   453
    seenlocal = None
29208
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   454
    # Whether a local/non-stdlib, non-symbol import has been seen.
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   455
    seennonsymbollocal = False
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   456
    # The last name to be imported (for sorting).
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   457
    lastname = None
30590
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   458
    laststdlib = None
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   459
    # Relative import levels encountered so far.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   460
    seenlevels = set()
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   461
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   462
    for node, newscope in walklocal(root):
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   463
26955
c4114e335b49 import-checker: extract function to generate a formatted warning
Yuya Nishihara <yuya@tcha.org>
parents: 26954
diff changeset
   464
        def msg(fmt, *args):
26956
4b56214ebb7a import-checker: include lineno in warning message
Yuya Nishihara <yuya@tcha.org>
parents: 26955
diff changeset
   465
            return (fmt % args, node.lineno)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   466
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   467
        if newscope:
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   468
            # Check for local imports in function
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   469
            for r in verify_modern_convention(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   470
                module, node, localmods, node.col_offset + 4
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   471
            ):
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   472
                yield r
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   473
        elif isinstance(node, ast.Import):
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   474
            # Disallow "import foo, bar" and require separate imports
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   475
            # for each module.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   476
            if len(node.names) > 1:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   477
                yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   478
                    'multiple imported names: %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   479
                    ', '.join(n.name for n in node.names),
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   480
                )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   481
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   482
            name = node.names[0].name
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   483
            asname = node.names[0].asname
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   484
30590
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   485
            stdlib = name in stdlib_modules
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   486
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   487
            # Ignore sorting rules on imports inside blocks.
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   488
            if node.col_offset == root_col_offset:
30590
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   489
                if lastname and name < lastname and laststdlib == stdlib:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   490
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   491
                        'imports not lexically sorted: %s < %s', name, lastname
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   492
                    )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   493
30590
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   494
            lastname = name
74eecb93c617 import-checker: do not enforce lexical sort accross stdlib/local boundary
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 29395
diff changeset
   495
            laststdlib = stdlib
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   496
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   497
            # stdlib imports should be before local imports.
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   498
            if stdlib and seenlocal and node.col_offset == root_col_offset:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   499
                yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   500
                    'stdlib import "%s" follows local import: %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   501
                    name,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   502
                    seenlocal,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   503
                )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   504
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   505
            if not stdlib:
28330
f3fb24e36d61 import-checker: report local with stdlib late warning
timeless <timeless@mozdev.org>
parents: 27621
diff changeset
   506
                seenlocal = name
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   507
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   508
            # Import of sibling modules should use relative imports.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   509
            topname = name.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   510
            if topname == topmodule:
26955
c4114e335b49 import-checker: extract function to generate a formatted warning
Yuya Nishihara <yuya@tcha.org>
parents: 26954
diff changeset
   511
                yield msg('import should be relative: %s', name)
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   512
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   513
            if name in requirealias and asname != requirealias[name]:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   514
                yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   515
                    '%s module must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   516
                    name,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   517
                    requirealias[name],
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   518
                )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   519
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   520
        elif isinstance(node, ast.ImportFrom):
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   521
            # Resolve the full imported module name.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   522
            if node.level > 0:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   523
                fullname = '.'.join(module.split('.')[: -node.level])
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   524
                if node.module:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   525
                    fullname += '.%s' % node.module
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   526
            else:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   527
                assert node.module
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   528
                fullname = node.module
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   529
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   530
                topname = fullname.split('.')[0]
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   531
                if topname == topmodule:
26955
c4114e335b49 import-checker: extract function to generate a formatted warning
Yuya Nishihara <yuya@tcha.org>
parents: 26954
diff changeset
   532
                    yield msg('import should be relative: %s', fullname)
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   533
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   534
            # __future__ is special since it needs to come first and use
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   535
            # symbol import.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   536
            if fullname != '__future__':
34038
bc2535238de2 import-checker: allow relative import a module being checked
Jun Wu <quark@fb.com>
parents: 33920
diff changeset
   537
                if not fullname or (
bc2535238de2 import-checker: allow relative import a module being checked
Jun Wu <quark@fb.com>
parents: 33920
diff changeset
   538
                    fullname in stdlib_modules
43596
0ad5d6c4bfad import-checker: allow 'from typing import ...'
Yuya Nishihara <yuya@tcha.org>
parents: 43415
diff changeset
   539
                    # allow standard 'from typing import ...' style
43954
303576116ac1 import-checker: allow all absolute imports of stdlib modules
Martin von Zweigbergk <martinvonz@google.com>
parents: 43596
diff changeset
   540
                    and fullname.startswith('.')
34038
bc2535238de2 import-checker: allow relative import a module being checked
Jun Wu <quark@fb.com>
parents: 33920
diff changeset
   541
                    and fullname not in localmods
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   542
                    and fullname + '.__init__' not in localmods
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   543
                ):
26955
c4114e335b49 import-checker: extract function to generate a formatted warning
Yuya Nishihara <yuya@tcha.org>
parents: 26954
diff changeset
   544
                    yield msg('relative import of stdlib module')
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   545
                else:
28330
f3fb24e36d61 import-checker: report local with stdlib late warning
timeless <timeless@mozdev.org>
parents: 27621
diff changeset
   546
                    seenlocal = fullname
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   547
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   548
            # Direct symbol import is only allowed from certain modules and
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   549
            # must occur before non-symbol imports.
29207
a09098c61fea import-checker: always build a list of imported symbols
Yuya Nishihara <yuya@tcha.org>
parents: 29122
diff changeset
   550
            found = fromlocal(node.module, node.level)
a09098c61fea import-checker: always build a list of imported symbols
Yuya Nishihara <yuya@tcha.org>
parents: 29122
diff changeset
   551
            if found and found[2]:  # node.module is a package
a09098c61fea import-checker: always build a list of imported symbols
Yuya Nishihara <yuya@tcha.org>
parents: 29122
diff changeset
   552
                prefix = found[0] + '.'
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   553
                symbols = (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   554
                    n.name for n in node.names if not fromlocal(prefix + n.name)
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   555
                )
29207
a09098c61fea import-checker: always build a list of imported symbols
Yuya Nishihara <yuya@tcha.org>
parents: 29122
diff changeset
   556
            else:
32419
d02888308235 import-checker: add a way to directly import certain symbols
Siddharth Agarwal <sid0@fb.com>
parents: 32374
diff changeset
   557
                symbols = (n.name for n in node.names)
d02888308235 import-checker: add a way to directly import certain symbols
Siddharth Agarwal <sid0@fb.com>
parents: 32374
diff changeset
   558
            symbols = [sym for sym in symbols if sym not in directsymbols]
26965
1fa66d3ad28d import-checker: reset context to verify convention in function scope
Yuya Nishihara <yuya@tcha.org>
parents: 26964
diff changeset
   559
            if node.module and node.col_offset == root_col_offset:
27272
69308357ecd1 import-checker: allow absolute imports of sub modules from local packages
Yuya Nishihara <yuya@tcha.org>
parents: 27018
diff changeset
   560
                if symbols and fullname not in allowsymbolimports:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   561
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   562
                        'direct symbol import %s from %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   563
                        ', '.join(symbols),
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   564
                        fullname,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   565
                    )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   566
29208
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   567
                if symbols and seennonsymbollocal:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   568
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   569
                        'symbol import follows non-symbol import: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   570
                    )
29208
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   571
            if not symbols and fullname not in stdlib_modules:
cba8bc11ed10 import-checker: extend check of symbol-import order to all local modules
Yuya Nishihara <yuya@tcha.org>
parents: 29207
diff changeset
   572
                seennonsymbollocal = True
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   573
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   574
            if not node.module:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   575
                assert node.level
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   576
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   577
                # Only allow 1 group per level.
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   578
                if (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   579
                    node.level in seenlevels
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   580
                    and node.col_offset == root_col_offset
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   581
                ):
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   582
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   583
                        'multiple "from %s import" statements', '.' * node.level
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   584
                    )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   585
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   586
                # Higher-level groups come before lower-level groups.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   587
                if any(node.level > l for l in seenlevels):
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   588
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   589
                        'higher-level import should come first: %s', fullname
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   590
                    )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   591
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   592
                seenlevels.add(node.level)
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   593
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   594
            # Entries in "from .X import ( ... )" lists must be lexically
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   595
            # sorted.
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   596
            lastentryname = None
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   597
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   598
            for n in node.names:
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   599
                if lastentryname and n.name < lastentryname:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   600
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   601
                        'imports from %s not lexically sorted: %s < %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   602
                        fullname,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   603
                        n.name,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   604
                        lastentryname,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   605
                    )
25703
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   606
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   607
                lastentryname = n.name
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   608
1a6a117d0b95 import-checker: establish modern import convention
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25702
diff changeset
   609
                if n.name in requirealias and n.asname != requirealias[n.name]:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   610
                    yield msg(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   611
                        '%s from %s must be "as" aliased to %s',
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   612
                        n.name,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   613
                        fullname,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   614
                        requirealias[n.name],
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   615
                    )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   616
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   617
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   618
def verify_stdlib_on_own_line(root):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   619
    """Given some python source, verify that stdlib imports are done
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   620
    in separate statements from relative local module imports.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   621
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   622
    >>> list(verify_stdlib_on_own_line(ast.parse('import sys, foo')))
26956
4b56214ebb7a import-checker: include lineno in warning message
Yuya Nishihara <yuya@tcha.org>
parents: 26955
diff changeset
   623
    [('mixed imports\\n   stdlib:    sys\\n   relative:  foo', 1)]
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   624
    >>> list(verify_stdlib_on_own_line(ast.parse('import sys, os')))
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   625
    []
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   626
    >>> list(verify_stdlib_on_own_line(ast.parse('import foo, bar')))
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   627
    []
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   628
    """
25702
ab2c5163900e import-checker: establish new function for verifying import conventions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25701
diff changeset
   629
    for node in ast.walk(root):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   630
        if isinstance(node, ast.Import):
20386
a05d31b040d7 import-checker: show stdlib and relative imports separately
Mads Kiilerich <madski@unity3d.com>
parents: 20383
diff changeset
   631
            from_stdlib = {False: [], True: []}
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   632
            for n in node.names:
20386
a05d31b040d7 import-checker: show stdlib and relative imports separately
Mads Kiilerich <madski@unity3d.com>
parents: 20383
diff changeset
   633
                from_stdlib[n.name in stdlib_modules].append(n.name)
a05d31b040d7 import-checker: show stdlib and relative imports separately
Mads Kiilerich <madski@unity3d.com>
parents: 20383
diff changeset
   634
            if from_stdlib[True] and from_stdlib[False]:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   635
                yield (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   636
                    'mixed imports\n   stdlib:    %s\n   relative:  %s'
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   637
                    % (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   638
                        ', '.join(sorted(from_stdlib[True])),
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   639
                        ', '.join(sorted(from_stdlib[False])),
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   640
                    ),
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   641
                    node.lineno,
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   642
                )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   643
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   644
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   645
class CircularImport(Exception):
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   646
    pass
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   647
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   648
24490
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   649
def checkmod(mod, imports):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   650
    shortest = {}
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   651
    visit = [[mod]]
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   652
    while visit:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   653
        path = visit.pop(0)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   654
        for i in sorted(imports.get(path[-1], [])):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   655
            if len(path) < shortest.get(i, 1000):
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   656
                shortest[i] = len(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   657
                if i in path:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   658
                    if i == path[0]:
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   659
                        raise CircularImport(path)
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   660
                    continue
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   661
                visit.append(path + [i])
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   662
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   663
20038
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   664
def rotatecycle(cycle):
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   665
    """arrange a cycle so that the lexicographically first module listed first
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   666
24488
4b3fc46097f7 import-checker: drop duplicate element from cycle
Matt Mackall <mpm@selenic.com>
parents: 24487
diff changeset
   667
    >>> rotatecycle(['foo', 'bar'])
20038
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   668
    ['bar', 'foo', 'bar']
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   669
    """
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   670
    lowest = min(cycle)
c65a6937b828 import-checker: try a little harder to show fewer cycles
Augie Fackler <raf@durin42.com>
parents: 20037
diff changeset
   671
    idx = cycle.index(lowest)
24488
4b3fc46097f7 import-checker: drop duplicate element from cycle
Matt Mackall <mpm@selenic.com>
parents: 24487
diff changeset
   672
    return cycle[idx:] + cycle[:idx] + [lowest]
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   673
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   674
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   675
def find_cycles(imports):
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   676
    """Find cycles in an already-loaded import graph.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   677
25175
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   678
    All module names recorded in `imports` should be absolute one.
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   679
28703
a274c4f9087a py3: use print_function in import-checker
timeless <timeless@mozdev.org>
parents: 28702
diff changeset
   680
    >>> from __future__ import print_function
25175
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   681
    >>> imports = {'top.foo': ['top.bar', 'os.path', 'top.qux'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   682
    ...            'top.bar': ['top.baz', 'sys'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   683
    ...            'top.baz': ['top.foo'],
10e6c4b7121b import-checker: don't treat modules as relative one if not found
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25174
diff changeset
   684
    ...            'top.qux': ['top.foo']}
28703
a274c4f9087a py3: use print_function in import-checker
timeless <timeless@mozdev.org>
parents: 28702
diff changeset
   685
    >>> print('\\n'.join(sorted(find_cycles(imports))))
24487
642d245ff537 import-checker: fix rotatecycle
Matt Mackall <mpm@selenic.com>
parents: 22975
diff changeset
   686
    top.bar -> top.baz -> top.foo -> top.bar
642d245ff537 import-checker: fix rotatecycle
Matt Mackall <mpm@selenic.com>
parents: 22975
diff changeset
   687
    top.foo -> top.qux -> top.foo
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   688
    """
24491
784b278b349c import-checker: rotatecycle is actually the canonical cycle key
Matt Mackall <mpm@selenic.com>
parents: 24490
diff changeset
   689
    cycles = set()
28704
1fa6fdb72275 py3: handle iter/iterkeys+iteritems python3 divergence in import-checker
timeless <timeless@mozdev.org>
parents: 28703
diff changeset
   690
    for mod in sorted(imports.keys()):
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   691
        try:
24490
fb4639d5268e import-checker: make search algorithm non-recursive breadth-first
Matt Mackall <mpm@selenic.com>
parents: 24489
diff changeset
   692
            checkmod(mod, imports)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25175
diff changeset
   693
        except CircularImport as e:
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   694
            cycle = e.args[0]
24491
784b278b349c import-checker: rotatecycle is actually the canonical cycle key
Matt Mackall <mpm@selenic.com>
parents: 24490
diff changeset
   695
            cycles.add(" -> ".join(rotatecycle(cycle)))
784b278b349c import-checker: rotatecycle is actually the canonical cycle key
Matt Mackall <mpm@selenic.com>
parents: 24490
diff changeset
   696
    return cycles
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   697
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   698
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   699
def _cycle_sortkey(c):
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   700
    return len(c), c
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   701
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   702
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   703
def embedded(f, modname, src):
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   704
    """Extract embedded python code
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   705
33877
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   706
    >>> def _forcestr(thing):
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   707
    ...     if not isinstance(thing, str):
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   708
    ...         return thing.decode('ascii')
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   709
    ...     return thing
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   710
    >>> def test(fn, lines):
33877
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   711
    ...     for s, m, f, l in embedded(fn, b"example", lines):
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   712
    ...         print("%s %s %d" % (_forcestr(m), _forcestr(f), l))
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   713
    ...         print(repr(_forcestr(s)))
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   714
    >>> lines = [
42388
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   715
    ...   'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   716
    ...   '  >>> from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   717
    ...   "  >>> ' multiline",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   718
    ...   "  ... string'",
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   719
    ...   '  ',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   720
    ...   'comment',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   721
    ...   '  $ cat > foo.py <<EOF',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   722
    ...   '  > from __future__ import print_function',
04eb3c5607af contrib: fix import-checker to operate on str instead of bytes
Augie Fackler <augie@google.com>
parents: 41161
diff changeset
   723
    ...   '  > EOF',
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   724
    ... ]
33877
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   725
    >>> test(b"example.t", lines)
40095
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   726
    example[2] doctest.py 1
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   727
    "from __future__ import print_function\\n' multiline\\nstring'\\n\\n"
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   728
    example[8] foo.py 7
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   729
    'from __future__ import print_function\\n'
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   730
    """
40095
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   731
    errors = []
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   732
    for name, starts, ends, code in testparseutil.pyembedded(f, src, errors):
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   733
        if not name:
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   734
            # use 'doctest.py', in order to make already existing
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   735
            # doctest above pass instantly
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   736
            name = 'doctest.py'
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   737
        # "starts" is "line number" (1-origin), but embedded() is
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   738
        # expected to return "line offset" (0-origin). Therefore, this
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   739
        # yields "starts - 1".
40480
99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3
Augie Fackler <augie@google.com>
parents: 40095
diff changeset
   740
        if not isinstance(modname, str):
99d5424eedc8 contrib: fix import-checker to not b'' module names on Python 3
Augie Fackler <augie@google.com>
parents: 40095
diff changeset
   741
            modname = modname.decode('utf8')
40095
7288838bec1f import-checker: use testparseutil.embedded() to centralize detection logic
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 39329
diff changeset
   742
        yield code, "%s[%d]" % (modname, starts), name, starts - 1
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   743
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   744
28919
a94f34306bb9 import-checker: refactor source reading
timeless <timeless@mozdev.org>
parents: 28713
diff changeset
   745
def sources(f, modname):
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   746
    """Yields possibly multiple sources from a filepath
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   747
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   748
    input: filepath, modulename
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   749
    yields:  script(string), modulename, filepath, linenumber
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   750
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   751
    For embedded scripts, the modulename and filepath will be different
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   752
    from the function arguments. linenumber is an offset relative to
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   753
    the input file.
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   754
    """
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   755
    py = False
29234
393aef802535 tests: enable import checker for all python files (including no .py files)
Yuya Nishihara <yuya@tcha.org>
parents: 29211
diff changeset
   756
    if not f.endswith('.t'):
33877
bcf53149ebce contrib: make import checker always think in terms of bytes
Augie Fackler <raf@durin42.com>
parents: 33876
diff changeset
   757
        with open(f, 'rb') as src:
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   758
            yield src.read(), modname, f, 0
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   759
            py = True
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   760
    if py or f.endswith('.t'):
43415
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
   761
        # Strictly speaking we should sniff for the magic header that denotes
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
   762
        # Python source file encoding. But in reality we don't use anything
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
   763
        # other than ASCII (mainly) and UTF-8 (in a few exceptions), so
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
   764
        # simplicity is fine.
a8454e846736 import-checker: open all source files as utf-8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43084
diff changeset
   765
        with io.open(f, 'r', encoding='utf-8') as src:
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   766
            for script, modname, t, line in embedded(f, modname, src):
40481
90517fad4293 contrib: tweak import-checker to always use bytes for module names
Augie Fackler <augie@google.com>
parents: 40480
diff changeset
   767
                yield script, modname.encode('utf8'), t, line
28919
a94f34306bb9 import-checker: refactor source reading
timeless <timeless@mozdev.org>
parents: 28713
diff changeset
   768
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   769
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   770
def main(argv):
25063
723e364488f4 import-checker: add xargs like mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24669
diff changeset
   771
    if len(argv) < 2 or (argv[1] == '-' and len(argv) > 2):
28703
a274c4f9087a py3: use print_function in import-checker
timeless <timeless@mozdev.org>
parents: 28702
diff changeset
   772
        print('Usage: %s {-|file [file] [file] ...}')
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   773
        return 1
25063
723e364488f4 import-checker: add xargs like mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24669
diff changeset
   774
    if argv[1] == '-':
723e364488f4 import-checker: add xargs like mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24669
diff changeset
   775
        argv = argv[:1]
723e364488f4 import-checker: add xargs like mode
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 24669
diff changeset
   776
        argv.extend(l.rstrip() for l in sys.stdin.readlines())
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   777
    localmodpaths = {}
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   778
    used_imports = {}
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   779
    any_errors = False
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   780
    for source_path in argv[1:]:
32374
194b0f781132 import-checker: drop workaround for pure modules
Yuya Nishihara <yuya@tcha.org>
parents: 32372
diff changeset
   781
        modname = dotted_name_of_path(source_path)
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   782
        localmodpaths[modname] = source_path
32509
a025ec43856c import-checker: guess names of C extension modules
Yuya Nishihara <yuya@tcha.org>
parents: 32508
diff changeset
   783
    localmods = populateextmods(localmodpaths)
32508
4c712b90c60a import-checker: convert localmods to a set of module names
Yuya Nishihara <yuya@tcha.org>
parents: 32507
diff changeset
   784
    for localmodname, source_path in sorted(localmodpaths.items()):
33890
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   785
        if not isinstance(localmodname, bytes):
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   786
            # This is only safe because all hg's files are ascii
3de9a2df6411 contrib: have import-checker work mostly with native strings for mod names
Augie Fackler <raf@durin42.com>
parents: 33877
diff changeset
   787
            localmodname = localmodname.encode('ascii')
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   788
        for src, modname, name, line in sources(source_path, localmodname):
28920
cdf331b54eb8 import-checker: track SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28919
diff changeset
   789
            try:
cdf331b54eb8 import-checker: track SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28919
diff changeset
   790
                used_imports[modname] = sorted(
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   791
                    imported_modules(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   792
                        src, modname, name, localmods, ignore_nested=True
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   793
                    )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   794
                )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   795
                for error, lineno in verify_import_convention(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   796
                    modname, src, localmods
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   797
                ):
28920
cdf331b54eb8 import-checker: track SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28919
diff changeset
   798
                    any_errors = True
28922
4ec62a084e5c import-checker: parse python code from .t files
timeless <timeless@mozdev.org>
parents: 28921
diff changeset
   799
                    print('%s:%d: %s' % (source_path, lineno + line, error))
28920
cdf331b54eb8 import-checker: track SyntaxErrors
timeless <timeless@mozdev.org>
parents: 28919
diff changeset
   800
            except SyntaxError as e:
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   801
                print(
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   802
                    '%s:%d: SyntaxError: %s' % (source_path, e.lineno + line, e)
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   803
                )
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   804
    cycles = find_cycles(used_imports)
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   805
    if cycles:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   806
        firstmods = set()
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   807
        for c in sorted(cycles, key=_cycle_sortkey):
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   808
            first = c.split()[0]
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   809
            # As a rough cut, ignore any cycle that starts with the
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   810
            # same module as some other cycle. Otherwise we see lots
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   811
            # of cycles that are effectively duplicates.
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   812
            if first in firstmods:
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   813
                continue
28703
a274c4f9087a py3: use print_function in import-checker
timeless <timeless@mozdev.org>
parents: 28702
diff changeset
   814
            print('Import cycle:', c)
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   815
            firstmods.add(first)
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   816
        any_errors = True
25731
cd1daab5d036 import-checker.py: exit with code 0 if no error is detected
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25703
diff changeset
   817
    return any_errors != 0
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   818
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 42813
diff changeset
   819
20036
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   820
if __name__ == '__main__':
e5d51fa51aba contrib: add an import checker
Augie Fackler <raf@durin42.com>
parents:
diff changeset
   821
    sys.exit(int(main(sys.argv)))