demandimport: do not raise ImportError for unknown item in fromlist stable

demandimport: do not raise ImportError for unknown item in fromlist

This is the behavior of the default __import__() function, which doesn't
validate the existence of the fromlist items. Later on, the missing attribute
is detected while processing the import statement.

https://hg.python.org/cpython/file/v2.7.13/Python/import.c#l2575

The comtypes library relies on this (maybe) undocumented behavior, and we
got a bug report to TortoiseHg, sigh.

https://bitbucket.org/tortoisehg/thg/issues/4647/

The test added at 26a4e46af2bc verifies the behavior of the import statement,
so this patch only adds the test of __import__() function and works around
CPython/PyPy difference.

--- a/mercurial/demandimport.py	Thu Dec 08 23:59:36 2016 +0800
+++ b/mercurial/demandimport.py	Mon Dec 19 22:46:00 2016 +0900
@@ -199,8 +199,11 @@
             nonpkg = getattr(mod, '__path__', nothing) is nothing
             if symbol is nothing:
                 if nonpkg:
-                    # do not try relative import, which would raise ValueError
-                    raise ImportError('cannot import name %s' % attr)
+                    # do not try relative import, which would raise ValueError,
+                    # and leave unknown attribute as the default __import__()
+                    # would do. the missing attribute will be detected later
+                    # while processing the import statement.
+                    return
                 mn = '%s.%s' % (mod.__name__, attr)
                 if mn in ignore:
                     importfunc = _origimport

--- a/tests/test-demandimport.py	Thu Dec 08 23:59:36 2016 +0800
+++ b/tests/test-demandimport.py	Mon Dec 19 22:46:00 2016 +0900
@@ -70,7 +70,16 @@
     print('no demandmod should be created for attribute of non-package '
           'module:\ncontextlib.unknownattr =', f(unknownattr))
 except ImportError as inst:
-    print('contextlib.unknownattr = ImportError: %s' % inst)
+    print('contextlib.unknownattr = ImportError: %s'
+          % rsub(r"'", '', str(inst)))
+
+# Unlike the import statement, __import__() function should not raise
+# ImportError even if fromlist has an unknown item
+# (see Python/import.c:import_module_level() and ensure_fromlist())
+contextlibimp = __import__('contextlib', globals(), locals(), ['unknownattr'])
+print("__import__('contextlib', ..., ['unknownattr']) =", f(contextlibimp))
+print("hasattr(contextlibimp, 'unknownattr') =",
+      util.safehasattr(contextlibimp, 'unknownattr'))
 
 demandimport.disable()
 os.environ['HGDEMANDIMPORT'] = 'disable'

--- a/tests/test-demandimport.py.out	Thu Dec 08 23:59:36 2016 +0800
+++ b/tests/test-demandimport.py.out	Mon Dec 19 22:46:00 2016 +0900
@@ -18,4 +18,6 @@
 re = <proxied module 'sys'>
 contextlib = <unloaded module 'contextlib'>
 contextlib.unknownattr = ImportError: cannot import name unknownattr
+__import__('contextlib', ..., ['unknownattr']) = <module 'contextlib' from '?'>
+hasattr(contextlibimp, 'unknownattr') = False
 node = <module 'mercurial.node' from '?'>