mercurial/demandimport.py
author Greg Ward <greg-hg@gerg.ca>
Sat, 28 Mar 2009 12:24:53 -0400
changeset 8081 6c3b8132078e
parent 7861 2bc14da14992
child 8225 46293a0c7e9f
permissions -rw-r--r--
issue1577: fix broken test by assuming less about CVS output. Specifically, output of "cvs ci" varies unpredictably across CVS versions, so any test that includes the output of "cvs ci" is doomed to fail some of the time. This fixes that by discarding the output of "cvs ci".
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     1
# demandimport.py - global demand-loading of modules for Mercurial
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     2
#
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4631
diff changeset
     3
# Copyright 2006, 2007 Matt Mackall <mpm@selenic.com>
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     4
#
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     6
# of the GNU General Public License, incorporated herein by reference.
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     7
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     8
'''
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     9
demandimport - automatic demandloading of modules
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    10
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    11
To enable this module, do:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    12
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    13
  import demandimport; demandimport.enable()
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    14
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    15
Imports of the following forms will be demand-loaded:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    16
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    17
  import a, b.c
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    18
  import a.b as c
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    19
  from a import b,c # a will be loaded immediately
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    20
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    21
These imports will not be delayed:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    22
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    23
  from a import *
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    24
  b = __import__(a)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    25
'''
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    26
7727
25fc4c620e54 demandimport: patch __builtin__ instead of __builtins__
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5929
diff changeset
    27
import __builtin__
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    28
_origimport = __import__
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    29
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    30
class _demandmod(object):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    31
    """module demand-loader and proxy"""
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    32
    def __init__(self, name, globals, locals):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    33
        if '.' in name:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    34
            head, rest = name.split('.', 1)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    35
            after = [rest]
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    36
        else:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    37
            head = name
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    38
            after = []
3896
3b628b5da9e9 use parent.__setattr__ instead of __dict__
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3877
diff changeset
    39
        object.__setattr__(self, "_data", (head, globals, locals, after))
3b628b5da9e9 use parent.__setattr__ instead of __dict__
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3877
diff changeset
    40
        object.__setattr__(self, "_module", None)
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    41
    def _extend(self, name):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    42
        """add to the list of submodules to load"""
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    43
        self._data[3].append(name)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    44
    def _load(self):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    45
        if not self._module:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    46
            head, globals, locals, after = self._data
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    47
            mod = _origimport(head, globals, locals)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    48
            # load submodules
3921
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    49
            def subload(mod, p):
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    50
                h, t = p, None
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    51
                if '.' in p:
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    52
                    h, t = p.split('.', 1)
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    53
                if not hasattr(mod, h):
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    54
                    setattr(mod, h, _demandmod(p, mod.__dict__, mod.__dict__))
3926
de6ae8f016af demandimport: handle already-loaded nested modules in subload
Brendan Cully <brendan@kublai.com>
parents: 3921
diff changeset
    55
                elif t:
3921
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    56
                    subload(getattr(mod, h), t)
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    57
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    58
            for x in after:
3921
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    59
                subload(mod, x)
6d0d025e125a demandimport: fix import x.y.z as a when x.y is already imported.
Matt Mackall <mpm@selenic.com>
parents: 3903
diff changeset
    60
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    61
            # are we in the locals dictionary still?
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    62
            if locals and locals.get(head) == self:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    63
                locals[head] = mod
3896
3b628b5da9e9 use parent.__setattr__ instead of __dict__
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 3877
diff changeset
    64
            object.__setattr__(self, "_module", mod)
4631
e3afa670e484 demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents: 4626
diff changeset
    65
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    66
    def __repr__(self):
4631
e3afa670e484 demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents: 4626
diff changeset
    67
        if self._module:
e3afa670e484 demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents: 4626
diff changeset
    68
            return "<proxied module '%s'>" % self._data[0]
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    69
        return "<unloaded module '%s'>" % self._data[0]
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    70
    def __call__(self, *args, **kwargs):
5639
7dd5cf9d1e09 demandload: give better diagnostic for call of an unloaded module
Matt Mackall <mpm@selenic.com>
parents: 5098
diff changeset
    71
        raise TypeError("%s object is not callable" % repr(self))
3903
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    72
    def __getattribute__(self, attr):
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    73
        if attr in ('_data', '_extend', '_load', '_module'):
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    74
            return object.__getattribute__(self, attr)
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    75
        self._load()
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    76
        return getattr(self._module, attr)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    77
    def __setattr__(self, attr, val):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    78
        self._load()
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    79
        setattr(self._module, attr, val)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    80
5929
e160f2312815 demandimport: handling new relative imports
Ali Gholami Rudi <aligrudi@users.sourceforge.net>
parents: 5639
diff changeset
    81
def _demandimport(name, globals=None, locals=None, fromlist=None, level=None):
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    82
    if not locals or name in ignore or fromlist == ('*',):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    83
        # these cases we can't really delay
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    84
        return _origimport(name, globals, locals, fromlist)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    85
    elif not fromlist:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    86
        # import a [as b]
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    87
        if '.' in name: # a.b
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    88
            base, rest = name.split('.', 1)
3903
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    89
            # email.__init__ loading email.mime
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    90
            if globals and globals.get('__name__', None) == base:
f9136599700f Make demandimport pass all tests on python2.5.
Brendan Cully <brendan@kublai.com>
parents: 3898
diff changeset
    91
                return _origimport(name, globals, locals, fromlist)
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    92
            # if a is already demand-loaded, add b to its submodule list
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    93
            if base in locals:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    94
                if isinstance(locals[base], _demandmod):
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    95
                    locals[base]._extend(rest)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    96
                return locals[base]
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    97
        return _demandmod(name, globals, locals)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    98
    else:
5929
e160f2312815 demandimport: handling new relative imports
Ali Gholami Rudi <aligrudi@users.sourceforge.net>
parents: 5639
diff changeset
    99
        if level is not None:
e160f2312815 demandimport: handling new relative imports
Ali Gholami Rudi <aligrudi@users.sourceforge.net>
parents: 5639
diff changeset
   100
            # from . import b,c,d or from .a import b,c,d
e160f2312815 demandimport: handling new relative imports
Ali Gholami Rudi <aligrudi@users.sourceforge.net>
parents: 5639
diff changeset
   101
            return _origimport(name, globals, locals, fromlist, level)
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   102
        # from a import b,c,d
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   103
        mod = _origimport(name, globals, locals)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   104
        # recurse down the module chain
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   105
        for comp in name.split('.')[1:]:
4626
eb99af2d845e Fix for demandimport.py and Windows compiled version.
Lee Cantey <lcantey@gmail.com>
parents: 4126
diff changeset
   106
            if not hasattr(mod, comp):
eb99af2d845e Fix for demandimport.py and Windows compiled version.
Lee Cantey <lcantey@gmail.com>
parents: 4126
diff changeset
   107
                setattr(mod, comp, _demandmod(comp, mod.__dict__, mod.__dict__))
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   108
            mod = getattr(mod, comp)
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   109
        for x in fromlist:
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   110
            # set requested submodules for demand load
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   111
            if not(hasattr(mod, x)):
4631
e3afa670e484 demandimport: fix issue579 and add a test
Matt Mackall <mpm@selenic.com>
parents: 4626
diff changeset
   112
                setattr(mod, x, _demandmod(x, mod.__dict__, locals))
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   113
        return mod
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   114
5098
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   115
ignore = [
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   116
    '_hashlib',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   117
    '_xmlplus',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   118
    'fcntl',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   119
    'win32com.gen_py',
7861
2bc14da14992 demandimport: blacklist pythoncom
Steve Borho <steve@borho.org>
parents: 7727
diff changeset
   120
    'pythoncom',
5098
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   121
    # imported by tarfile, not available under Windows
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   122
    'pwd',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   123
    'grp',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   124
    # imported by profile, itself imported by hotshot.stats,
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   125
    # not available under Windows
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   126
    'resource',
0bbd86b847dd demandimport: ignore resource module, not available under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5097
diff changeset
   127
    ]
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   128
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   129
def enable():
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   130
    "enable global demand-loading of modules"
7727
25fc4c620e54 demandimport: patch __builtin__ instead of __builtins__
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5929
diff changeset
   131
    __builtin__.__import__ = _demandimport
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   132
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   133
def disable():
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   134
    "disable global demand-loading of modules"
7727
25fc4c620e54 demandimport: patch __builtin__ instead of __builtins__
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5929
diff changeset
   135
    __builtin__.__import__ = _origimport
3877
abaee83ce0a6 Replace demandload with new demandimport
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   136