mercurial/packagescan.py
author Brendan Cully <brendan@kublai.com>
Tue, 24 Oct 2006 13:17:41 -0700
changeset 3498 776c317566b6
parent 2996 799811087044
child 3673 eb0b4a2d70a9
permissions -rw-r--r--
zsh: cache hg help output
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
     1
# packagescan.py - Helper module for identifing used modules.
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     2
# Used for the py2exe distutil.
2323
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
     3
# This module must be the first mercurial module imported in setup.py
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     4
#
2859
345bac2bc4ec update copyrights.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2600
diff changeset
     5
# Copyright 2005, 2006 Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     6
#
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     7
# This software may be used and distributed according to the terms
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     8
# of the GNU General Public License, incorporated herein by reference.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
     9
import glob
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    10
import os
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    11
import sys
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    12
import ihooks
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    13
import types
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    14
import string
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    15
2323
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    16
# Install this module as fake demandload module
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    17
sys.modules['mercurial.demandload'] = sys.modules[__name__]
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    18
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    19
# Requiredmodules contains the modules imported by demandload.
2600
c4325f0a9b91 clean up trailing white space.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2497
diff changeset
    20
# Please note that demandload can be invoked before the
2323
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    21
# mercurial.packagescan.scan method is invoked in case a mercurial
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    22
# module is imported.
2600
c4325f0a9b91 clean up trailing white space.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2497
diff changeset
    23
requiredmodules = {}
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    24
def demandload(scope, modules):
2600
c4325f0a9b91 clean up trailing white space.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2497
diff changeset
    25
    """ fake demandload function that collects the required modules
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    26
        foo            import foo
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    27
        foo bar        import foo, bar
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    28
        foo.bar        import foo.bar
2993
985594e891b8 Support foo@bar notation as demandload module spec.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 2859
diff changeset
    29
        foo@bar        import foo as bar
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    30
        foo:bar        from foo import bar
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    31
        foo:bar,quux   from foo import bar, quux
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    32
        foo.bar:quux   from foo.bar import quux"""
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    33
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    34
    for m in modules.split():
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    35
        mod = None
1841
7f12a63568ae Fix py2exe packagescan problem with new demandload
Eung-ju Park <eungju@gmail.com>
parents: 1308
diff changeset
    36
        try:
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    37
            module, fromlist = m.split(':')
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    38
            fromlist = fromlist.split(',')
1841
7f12a63568ae Fix py2exe packagescan problem with new demandload
Eung-ju Park <eungju@gmail.com>
parents: 1308
diff changeset
    39
        except:
7f12a63568ae Fix py2exe packagescan problem with new demandload
Eung-ju Park <eungju@gmail.com>
parents: 1308
diff changeset
    40
            module = m
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    41
            fromlist = []
2996
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2995 2993
diff changeset
    42
        as_ = None
2993
985594e891b8 Support foo@bar notation as demandload module spec.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 2859
diff changeset
    43
        if '@' in module:
985594e891b8 Support foo@bar notation as demandload module spec.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 2859
diff changeset
    44
            module, as_ = module.split('@')
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    45
        mod = __import__(module, scope, scope, fromlist)
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    46
        if fromlist == []:
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    47
            # mod is only the top package, but we need all packages
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    48
            comp = module.split('.')
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    49
            i = 1
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    50
            mn = comp[0]
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    51
            while True:
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    52
                # mn and mod.__name__ might not be the same
2996
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2995 2993
diff changeset
    53
                if not as_:
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2995 2993
diff changeset
    54
                    as_ = mn
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2995 2993
diff changeset
    55
                scope[as_] = mod
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    56
                requiredmodules[mod.__name__] = 1
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    57
                if len(comp) == i: break
2600
c4325f0a9b91 clean up trailing white space.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2497
diff changeset
    58
                mod = getattr(mod,comp[i])
2322
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    59
                mn = string.join(comp[:i+1],'.')
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    60
                i += 1
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    61
        else:
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    62
            # mod is the last package in the component list
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    63
            requiredmodules[mod.__name__] = 1
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    64
            for f in fromlist:
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    65
                scope[f] = getattr(mod,f)
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    66
                if type(scope[f]) == types.ModuleType:
685597676a13 packagescan: handle demandload module naming changes.
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    67
                    requiredmodules[scope[f].__name__] = 1
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    68
2497
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    69
class SkipPackage(Exception):
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    70
    def __init__(self, reason):
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    71
        self.reason = reason
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    72
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    73
scan_in_progress = False
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    74
2323
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
    75
def scan(libpath,packagename):
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    76
    """ helper for finding all required modules of package <packagename> """
2497
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    77
    global scan_in_progress
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
    78
    scan_in_progress = True
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    79
    # Use the package in the build directory
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    80
    libpath = os.path.abspath(libpath)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    81
    sys.path.insert(0,libpath)
2402
82cef38fea56 packagescan can't scan nested packages
Eung-Ju PARK <eungju@gmail.com>
parents: 2325
diff changeset
    82
    packdir = os.path.join(libpath,packagename.replace('.', '/'))
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
    83
    # A normal import would not find the package in
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    84
    # the build directory. ihook is used to force the import.
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
    85
    # After the package is imported the import scope for
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    86
    # the following imports is settled.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    87
    p = importfrom(packdir)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    88
    globals()[packagename] = p
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    89
    sys.modules[packagename] = p
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    90
    # Fetch the python modules in the package
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    91
    cwd = os.getcwd()
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    92
    os.chdir(packdir)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    93
    pymodulefiles = glob.glob('*.py')
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    94
    extmodulefiles = glob.glob('*.pyd')
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    95
    os.chdir(cwd)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    96
    # Import all python modules and by that run the fake demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    97
    for m in pymodulefiles:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    98
        if m == '__init__.py': continue
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
    99
        tmp = {}
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   100
        mname,ext = os.path.splitext(m)
1308
2073e5a71008 Cleanup of tabs and trailing spaces.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1283
diff changeset
   101
        fullname = packagename+'.'+mname
2497
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
   102
        try:
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
   103
            __import__(fullname,tmp,tmp)
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
   104
        except SkipPackage, inst:
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
   105
            print >> sys.stderr, 'skipping %s: %s' % (fullname, inst.reason)
976b6b2a1613 do not try to package lsprof if not available.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2402
diff changeset
   106
            continue
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   107
        requiredmodules[fullname] = 1
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   108
    # Import all extension modules and by that run the fake demandload
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   109
    for m in extmodulefiles:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   110
        tmp = {}
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   111
        mname,ext = os.path.splitext(m)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   112
        fullname = packagename+'.'+mname
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   113
        __import__(fullname,tmp,tmp)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   114
        requiredmodules[fullname] = 1
2323
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
   115
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
   116
def getmodules():
c58a403aa830 setup.py: install packagescan before any mercurial modules is imported
Volker Kleinfeld <Volker.Kleinfeld@gmx.de>
parents: 1841
diff changeset
   117
    return requiredmodules.keys()
1283
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   118
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   119
def importfrom(filename):
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   120
    """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   121
    import module/package from a named file and returns the module.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   122
    It does not check on sys.modules or includes the module in the scope.
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   123
    """
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   124
    loader = ihooks.BasicModuleLoader()
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   125
    path, file = os.path.split(filename)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   126
    name, ext  = os.path.splitext(file)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   127
    m = loader.find_module_in_dir(name, path)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   128
    if not m:
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   129
        raise ImportError, name
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   130
    m = loader.load_module(name, m)
f5faab34f32e Support for the distutils extention 'py2exe' added.
Volker.Kleinfeld@gmx.de
parents:
diff changeset
   131
    return m