view tests/test-extension.t @ 40326:fed697fa1734

sqlitestore: file storage backend using SQLite

This commit provides an extension which uses SQLite to store file
data (as opposed to revlogs).

As the inline documentation describes, there are still several
aspects to the extension that are incomplete. But it's a start.
The extension does support basic clone, checkout, and commit
workflows, which makes it suitable for simple use cases.

One notable missing feature is support for "bundlerepos." This is
probably responsible for the most test failures when the extension
is activated as part of the test suite.

All revision data is stored in SQLite. Data is stored as zstd
compressed chunks (default if zstd is available), zlib compressed
chunks (default if zstd is not available), or raw chunks (if
configured or if a compressed delta is not smaller than the raw
delta). This makes things very similar to revlogs.

Unlike revlogs, the extension doesn't yet enforce a limit on delta
chain length. This is an obvious limitation and should be addressed.
This is somewhat mitigated by the use of zstd, which is much faster
than zlib to decompress.

There is a dedicated table for storing deltas. Deltas are stored
by the SHA-1 hash of their uncompressed content. The "fileindex" table
has columns that reference the delta for each revision and the base
delta that delta should be applied against. A recursive SQL query
is used to resolve the delta chain along with the delta data.

By storing deltas by hash, we are able to de-duplicate delta storage!
With revlogs, the same deltas in different revlogs would result in
duplicate storage of that delta. In this scheme, inserting the
duplicate delta is a no-op and delta chains simply reference the
existing delta.

When initially implementing this extension, I did not have
content-indexed deltas and deltas could be duplicated across files
(just like revlogs). When I implemented content-indexed deltas, the
size of the SQLite database for a full clone of mozilla-unified
dropped:

before: 2,554,261,504 bytes
after:  2,488,754,176 bytes

Surprisingly, this is still larger than the bytes size of revlog
files:

revlog files: 2,104,861,230 bytes
du -b:        2,254,381,614

I would have expected storage to be smaller since we're not limiting
delta chain length and since we're using zstd instead of zlib. I
suspect the SQLite indexes and per-column overhead account for the
bulk of the differences. (Keep in mind that revlog uses a 64-byte
packed struct for revision index data and deltas are stored without
padding. Aside from the 12 unused bytes in the 32 byte node field,
revlogs are pretty efficient.) Another source of overhead is file
name storage. With revlogs, file names are stored in the filesystem.
But with SQLite, we need to store file names in the database. This is
roughly equivalent to the size of the fncache file, which for the
mozilla-unified repository is ~34MB.

Since the SQLite database isn't append-only and since delta chains
can reference any delta, this opens some interesting possibilities.
For example, we could store deltas in reverse, such that fulltexts
are stored for newer revisions and deltas are applied to reconstruct
older revisions. This is likely a more optimal storage strategy for
version control, as new data tends to be more frequently accessed
than old data. We would obviously need wire protocol support for
transferring revision data from newest to oldest. And we would
probably need some kind of mechanism for "re-encoding" stores. But
it should be doable.

This extension is very much experimental quality. There are a handful
of features that don't work. It probably isn't suitable for day-to-day
use. But it could be used in limited cases (e.g. read-only checkouts
like in CI). And it is also a good proving ground for alternate
storage backends. As we continue to define interfaces for all things
storage, it will be useful to have a viable alternate storage backend
to see how things shake out in practice.

test-storage.py passes on Python 2 and introduces no new test failures on
Python 3. Having the storage-level unit tests has proved to be insanely
useful when developing this extension. Those tests caught numerous bugs
during development and I'm convinced this style of testing is the way
forward for ensuring alternate storage backends work as intended. Of
course, test coverage isn't close to what it needs to be. But it is
a start. And what coverage we have gives me confidence that basic store
functionality is implemented properly.

Differential Revision: https://phab.mercurial-scm.org/D4928

Test basic extension support
  $ cat > unflush.py <<EOF
  > import sys
  > from mercurial import pycompat
  > if pycompat.ispy3:
  >     # no changes required
  >     sys.exit(0)
  > with open(sys.argv[1], 'rb') as f:
  >     data = f.read()
  > with open(sys.argv[1], 'wb') as f:
  >     f.write(data.replace(b', flush=True', b''))
  > EOF

  $ cat > foobar.py <<EOF
  > import os
  > from mercurial import commands, registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > configtable = {}
  > configitem = registrar.configitem(configtable)
  > configitem(b'tests', b'foo', default=b"Foo")
  > def uisetup(ui):
  >     ui.debug(b"uisetup called [debug]\\n")
  >     ui.write(b"uisetup called\\n")
  >     ui.status(b"uisetup called [status]\\n")
  >     ui.flush()
  > def reposetup(ui, repo):
  >     ui.write(b"reposetup called for %s\\n" % os.path.basename(repo.root))
  >     ui.write(b"ui %s= repo.ui\\n" % (ui == repo.ui and b"=" or b"!"))
  >     ui.flush()
  > @command(b'foo', [], b'hg foo')
  > def foo(ui, *args, **kwargs):
  >     foo = ui.config(b'tests', b'foo')
  >     ui.write(foo)
  >     ui.write(b"\\n")
  > @command(b'bar', [], b'hg bar', norepo=True)
  > def bar(ui, *args, **kwargs):
  >     ui.write(b"Bar\\n")
  > EOF
  $ abspath=`pwd`/foobar.py

  $ mkdir barfoo
  $ cp foobar.py barfoo/__init__.py
  $ barfoopath=`pwd`/barfoo

  $ hg init a
  $ cd a
  $ echo foo > file
  $ hg add file
  $ hg commit -m 'add file'

  $ echo '[extensions]' >> $HGRCPATH
  $ echo "foobar = $abspath" >> $HGRCPATH
  $ hg foo
  uisetup called
  uisetup called [status]
  reposetup called for a
  ui == repo.ui
  reposetup called for a (chg !)
  ui == repo.ui (chg !)
  Foo
  $ hg foo --quiet
  uisetup called (no-chg !)
  reposetup called for a (chg !)
  ui == repo.ui
  Foo
  $ hg foo --debug
  uisetup called [debug] (no-chg !)
  uisetup called (no-chg !)
  uisetup called [status] (no-chg !)
  reposetup called for a (chg !)
  ui == repo.ui
  Foo

  $ cd ..
  $ hg clone a b
  uisetup called (no-chg !)
  uisetup called [status] (no-chg !)
  reposetup called for a
  ui == repo.ui
  reposetup called for b
  ui == repo.ui
  updating to branch default
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ hg bar
  uisetup called (no-chg !)
  uisetup called [status] (no-chg !)
  Bar
  $ echo 'foobar = !' >> $HGRCPATH

module/__init__.py-style

  $ echo "barfoo = $barfoopath" >> $HGRCPATH
  $ cd a
  $ hg foo
  uisetup called
  uisetup called [status]
  reposetup called for a
  ui == repo.ui
  reposetup called for a (chg !)
  ui == repo.ui (chg !)
  Foo
  $ echo 'barfoo = !' >> $HGRCPATH

Check that extensions are loaded in phases:

  $ cat > foo.py <<EOF
  > from __future__ import print_function
  > import os
  > name = os.path.basename(__file__).rsplit('.', 1)[0]
  > print("1) %s imported" % name, flush=True)
  > def uisetup(ui):
  >     print("2) %s uisetup" % name, flush=True)
  > def extsetup():
  >     print("3) %s extsetup" % name, flush=True)
  > def reposetup(ui, repo):
  >    print("4) %s reposetup" % name, flush=True)
  > 
  > bytesname = name.encode('utf-8')
  > # custom predicate to check registration of functions at loading
  > from mercurial import (
  >     registrar,
  >     smartset,
  > )
  > revsetpredicate = registrar.revsetpredicate()
  > @revsetpredicate(bytesname, safe=True) # safe=True for query via hgweb
  > def custompredicate(repo, subset, x):
  >     return smartset.baseset([r for r in subset if r in {0}])
  > EOF
  $ $PYTHON $TESTTMP/unflush.py foo.py

  $ cp foo.py bar.py
  $ echo 'foo = foo.py' >> $HGRCPATH
  $ echo 'bar = bar.py' >> $HGRCPATH

Check normal command's load order of extensions and registration of functions

  $ hg log -r "foo() and bar()" -q
  1) foo imported
  1) bar imported
  2) foo uisetup
  2) bar uisetup
  3) foo extsetup
  3) bar extsetup
  4) foo reposetup
  4) bar reposetup
  0:c24b9ac61126

Check hgweb's load order of extensions and registration of functions

  $ cat > hgweb.cgi <<EOF
  > #!$PYTHON
  > from mercurial import demandimport; demandimport.enable()
  > from mercurial.hgweb import hgweb
  > from mercurial.hgweb import wsgicgi
  > application = hgweb(b'.', b'test repo')
  > wsgicgi.launch(application)
  > EOF
  $ . "$TESTDIR/cgienv"

  $ PATH_INFO='/' SCRIPT_NAME='' "$PYTHON" hgweb.cgi \
  >    | grep '^[0-9]) ' # ignores HTML output
  1) foo imported
  1) bar imported
  2) foo uisetup
  2) bar uisetup
  3) foo extsetup
  3) bar extsetup
  4) foo reposetup
  4) bar reposetup

(check that revset predicate foo() and bar() are available)

#if msys
  $ PATH_INFO='//shortlog'
#else
  $ PATH_INFO='/shortlog'
#endif
  $ export PATH_INFO
  $ SCRIPT_NAME='' QUERY_STRING='rev=foo() and bar()' "$PYTHON" hgweb.cgi \
  >     | grep '<a href="/rev/[0-9a-z]*">'
     <a href="/rev/c24b9ac61126">add file</a>

  $ echo 'foo = !' >> $HGRCPATH
  $ echo 'bar = !' >> $HGRCPATH

Check "from __future__ import absolute_import" support for external libraries

(import-checker.py reports issues for some of heredoc python code
fragments below, because import-checker.py does not know test specific
package hierarchy. NO_CHECK_* should be used as a limit mark of
heredoc, in order to make import-checker.py ignore them. For
simplicity, all python code fragments below are generated with such
limit mark, regardless of importing module or not.)

#if windows
  $ PATHSEP=";"
#else
  $ PATHSEP=":"
#endif
  $ export PATHSEP

  $ mkdir $TESTTMP/libroot
  $ echo "s = 'libroot/ambig.py'" > $TESTTMP/libroot/ambig.py
  $ mkdir $TESTTMP/libroot/mod
  $ touch $TESTTMP/libroot/mod/__init__.py
  $ echo "s = 'libroot/mod/ambig.py'" > $TESTTMP/libroot/mod/ambig.py

  $ cat > $TESTTMP/libroot/mod/ambigabs.py <<NO_CHECK_EOF
  > from __future__ import absolute_import, print_function
  > import ambig # should load "libroot/ambig.py"
  > s = ambig.s
  > NO_CHECK_EOF
  $ cat > loadabs.py <<NO_CHECK_EOF
  > import mod.ambigabs as ambigabs
  > def extsetup():
  >     print('ambigabs.s=%s' % ambigabs.s, flush=True)
  > NO_CHECK_EOF
  $ $PYTHON $TESTTMP/unflush.py loadabs.py
  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}/libroot; hg --config extensions.loadabs=loadabs.py root)
  ambigabs.s=libroot/ambig.py
  $TESTTMP/a

#if no-py3
  $ cat > $TESTTMP/libroot/mod/ambigrel.py <<NO_CHECK_EOF
  > from __future__ import print_function
  > import ambig # should load "libroot/mod/ambig.py"
  > s = ambig.s
  > NO_CHECK_EOF
  $ cat > loadrel.py <<NO_CHECK_EOF
  > import mod.ambigrel as ambigrel
  > def extsetup():
  >     print('ambigrel.s=%s' % ambigrel.s, flush=True)
  > NO_CHECK_EOF
  $ $PYTHON $TESTTMP/unflush.py loadrel.py
  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}/libroot; hg --config extensions.loadrel=loadrel.py root)
  ambigrel.s=libroot/mod/ambig.py
  $TESTTMP/a
#endif

Check absolute/relative import of extension specific modules

  $ mkdir $TESTTMP/extroot
  $ cat > $TESTTMP/extroot/bar.py <<NO_CHECK_EOF
  > s = b'this is extroot.bar'
  > NO_CHECK_EOF
  $ mkdir $TESTTMP/extroot/sub1
  $ cat > $TESTTMP/extroot/sub1/__init__.py <<NO_CHECK_EOF
  > s = b'this is extroot.sub1.__init__'
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extroot/sub1/baz.py <<NO_CHECK_EOF
  > s = b'this is extroot.sub1.baz'
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extroot/__init__.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > s = b'this is extroot.__init__'
  > from . import foo
  > def extsetup(ui):
  >     ui.write(b'(extroot) ', foo.func(), b'\n')
  >     ui.flush()
  > NO_CHECK_EOF

  $ cat > $TESTTMP/extroot/foo.py <<NO_CHECK_EOF
  > # test absolute import
  > buf = []
  > def func():
  >     # "not locals" case
  >     import extroot.bar
  >     buf.append(b'import extroot.bar in func(): %s' % extroot.bar.s)
  >     return b'\n(extroot) '.join(buf)
  > # b"fromlist == ('*',)" case
  > from extroot.bar import *
  > buf.append(b'from extroot.bar import *: %s' % s)
  > # "not fromlist" and "if '.' in name" case
  > import extroot.sub1.baz
  > buf.append(b'import extroot.sub1.baz: %s' % extroot.sub1.baz.s)
  > # "not fromlist" and NOT "if '.' in name" case
  > import extroot
  > buf.append(b'import extroot: %s' % extroot.s)
  > # NOT "not fromlist" and NOT "level != -1" case
  > from extroot.bar import s
  > buf.append(b'from extroot.bar import s: %s' % s)
  > NO_CHECK_EOF
  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config extensions.extroot=$TESTTMP/extroot root)
  (extroot) from extroot.bar import *: this is extroot.bar
  (extroot) import extroot.sub1.baz: this is extroot.sub1.baz
  (extroot) import extroot: this is extroot.__init__
  (extroot) from extroot.bar import s: this is extroot.bar
  (extroot) import extroot.bar in func(): this is extroot.bar
  $TESTTMP/a

#if no-py3
  $ rm "$TESTTMP"/extroot/foo.*
  $ rm -Rf "$TESTTMP/extroot/__pycache__"
  $ cat > $TESTTMP/extroot/foo.py <<NO_CHECK_EOF
  > # test relative import
  > buf = []
  > def func():
  >     # "not locals" case
  >     import bar
  >     buf.append('import bar in func(): %s' % bar.s)
  >     return '\n(extroot) '.join(buf)
  > # "fromlist == ('*',)" case
  > from bar import *
  > buf.append('from bar import *: %s' % s)
  > # "not fromlist" and "if '.' in name" case
  > import sub1.baz
  > buf.append('import sub1.baz: %s' % sub1.baz.s)
  > # "not fromlist" and NOT "if '.' in name" case
  > import sub1
  > buf.append('import sub1: %s' % sub1.s)
  > # NOT "not fromlist" and NOT "level != -1" case
  > from bar import s
  > buf.append('from bar import s: %s' % s)
  > NO_CHECK_EOF
  $ hg --config extensions.extroot=$TESTTMP/extroot root
  (extroot) from bar import *: this is extroot.bar
  (extroot) import sub1.baz: this is extroot.sub1.baz
  (extroot) import sub1: this is extroot.sub1.__init__
  (extroot) from bar import s: this is extroot.bar
  (extroot) import bar in func(): this is extroot.bar
  $TESTTMP/a
#endif

#if demandimport

Examine whether module loading is delayed until actual referring, even
though module is imported with "absolute_import" feature.

Files below in each packages are used for described purpose:

- "called": examine whether "from MODULE import ATTR" works correctly
- "unused": examine whether loading is delayed correctly
- "used":   examine whether "from PACKAGE import MODULE" works correctly

Package hierarchy is needed to examine whether demand importing works
as expected for "from SUB.PACK.AGE import MODULE".

Setup "external library" to be imported with "absolute_import"
feature.

  $ mkdir -p $TESTTMP/extlibroot/lsub1/lsub2
  $ touch $TESTTMP/extlibroot/__init__.py
  $ touch $TESTTMP/extlibroot/lsub1/__init__.py
  $ touch $TESTTMP/extlibroot/lsub1/lsub2/__init__.py

  $ cat > $TESTTMP/extlibroot/lsub1/lsub2/called.py <<NO_CHECK_EOF
  > def func():
  >     return b"this is extlibroot.lsub1.lsub2.called.func()"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/lsub1/lsub2/unused.py <<NO_CHECK_EOF
  > raise Exception("extlibroot.lsub1.lsub2.unused is loaded unintentionally")
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/lsub1/lsub2/used.py <<NO_CHECK_EOF
  > detail = b"this is extlibroot.lsub1.lsub2.used"
  > NO_CHECK_EOF

Setup sub-package of "external library", which causes instantiation of
demandmod in "recurse down the module chain" code path. Relative
importing with "absolute_import" feature isn't tested, because "level
>=1 " doesn't cause instantiation of demandmod.

  $ mkdir -p $TESTTMP/extlibroot/recursedown/abs
  $ cat > $TESTTMP/extlibroot/recursedown/abs/used.py <<NO_CHECK_EOF
  > detail = b"this is extlibroot.recursedown.abs.used"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/recursedown/abs/__init__.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from extlibroot.recursedown.abs.used import detail
  > NO_CHECK_EOF

  $ mkdir -p $TESTTMP/extlibroot/recursedown/legacy
  $ cat > $TESTTMP/extlibroot/recursedown/legacy/used.py <<NO_CHECK_EOF
  > detail = b"this is extlibroot.recursedown.legacy.used"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/recursedown/legacy/__init__.py <<NO_CHECK_EOF
  > # legacy style (level == -1) import
  > from extlibroot.recursedown.legacy.used import detail
  > NO_CHECK_EOF

  $ cat > $TESTTMP/extlibroot/recursedown/__init__.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from extlibroot.recursedown.abs import detail as absdetail
  > from .legacy import detail as legacydetail
  > NO_CHECK_EOF

Setup package that re-exports an attribute of its submodule as the same
name. This leaves 'shadowing.used' pointing to 'used.detail', but still
the submodule 'used' should be somehow accessible. (issue5617)

  $ mkdir -p $TESTTMP/extlibroot/shadowing
  $ cat > $TESTTMP/extlibroot/shadowing/used.py <<NO_CHECK_EOF
  > detail = b"this is extlibroot.shadowing.used"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/shadowing/proxied.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from extlibroot.shadowing.used import detail
  > NO_CHECK_EOF
  $ cat > $TESTTMP/extlibroot/shadowing/__init__.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from .used import detail as used
  > NO_CHECK_EOF

Setup extension local modules to be imported with "absolute_import"
feature.

  $ mkdir -p $TESTTMP/absextroot/xsub1/xsub2
  $ touch $TESTTMP/absextroot/xsub1/__init__.py
  $ touch $TESTTMP/absextroot/xsub1/xsub2/__init__.py

  $ cat > $TESTTMP/absextroot/xsub1/xsub2/called.py <<NO_CHECK_EOF
  > def func():
  >     return b"this is absextroot.xsub1.xsub2.called.func()"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/absextroot/xsub1/xsub2/unused.py <<NO_CHECK_EOF
  > raise Exception("absextroot.xsub1.xsub2.unused is loaded unintentionally")
  > NO_CHECK_EOF
  $ cat > $TESTTMP/absextroot/xsub1/xsub2/used.py <<NO_CHECK_EOF
  > detail = b"this is absextroot.xsub1.xsub2.used"
  > NO_CHECK_EOF

Setup extension local modules to examine whether demand importing
works as expected in "level > 1" case.

  $ cat > $TESTTMP/absextroot/relimportee.py <<NO_CHECK_EOF
  > detail = b"this is absextroot.relimportee"
  > NO_CHECK_EOF
  $ cat > $TESTTMP/absextroot/xsub1/xsub2/relimporter.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from mercurial import pycompat
  > from ... import relimportee
  > detail = b"this relimporter imports %r" % (
  >     pycompat.bytestr(relimportee.detail))
  > NO_CHECK_EOF

Setup modules, which actually import extension local modules at
runtime.

  $ cat > $TESTTMP/absextroot/absolute.py << NO_CHECK_EOF
  > from __future__ import absolute_import
  > 
  > # import extension local modules absolutely (level = 0)
  > from absextroot.xsub1.xsub2 import used, unused
  > from absextroot.xsub1.xsub2.called import func
  > 
  > def getresult():
  >     result = []
  >     result.append(used.detail)
  >     result.append(func())
  >     return result
  > NO_CHECK_EOF

  $ cat > $TESTTMP/absextroot/relative.py << NO_CHECK_EOF
  > from __future__ import absolute_import
  > 
  > # import extension local modules relatively (level == 1)
  > from .xsub1.xsub2 import used, unused
  > from .xsub1.xsub2.called import func
  > 
  > # import a module, which implies "importing with level > 1"
  > from .xsub1.xsub2 import relimporter
  > 
  > def getresult():
  >     result = []
  >     result.append(used.detail)
  >     result.append(func())
  >     result.append(relimporter.detail)
  >     return result
  > NO_CHECK_EOF

Setup main procedure of extension.

  $ cat > $TESTTMP/absextroot/__init__.py <<NO_CHECK_EOF
  > from __future__ import absolute_import
  > from mercurial import registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > 
  > # "absolute" and "relative" shouldn't be imported before actual
  > # command execution, because (1) they import same modules, and (2)
  > # preceding import (= instantiate "demandmod" object instead of
  > # real "module" object) might hide problem of succeeding import.
  > 
  > @command(b'showabsolute', [], norepo=True)
  > def showabsolute(ui, *args, **opts):
  >     from absextroot import absolute
  >     ui.write(b'ABS: %s\n' % b'\nABS: '.join(absolute.getresult()))
  > 
  > @command(b'showrelative', [], norepo=True)
  > def showrelative(ui, *args, **opts):
  >     from . import relative
  >     ui.write(b'REL: %s\n' % b'\nREL: '.join(relative.getresult()))
  > 
  > # import modules from external library
  > from extlibroot.lsub1.lsub2 import used as lused, unused as lunused
  > from extlibroot.lsub1.lsub2.called import func as lfunc
  > from extlibroot.recursedown import absdetail, legacydetail
  > from extlibroot.shadowing import proxied
  > 
  > def uisetup(ui):
  >     result = []
  >     result.append(lused.detail)
  >     result.append(lfunc())
  >     result.append(absdetail)
  >     result.append(legacydetail)
  >     result.append(proxied.detail)
  >     ui.write(b'LIB: %s\n' % b'\nLIB: '.join(result))
  > NO_CHECK_EOF

Examine module importing.

  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config extensions.absextroot=$TESTTMP/absextroot showabsolute)
  LIB: this is extlibroot.lsub1.lsub2.used
  LIB: this is extlibroot.lsub1.lsub2.called.func()
  LIB: this is extlibroot.recursedown.abs.used
  LIB: this is extlibroot.recursedown.legacy.used
  LIB: this is extlibroot.shadowing.used
  ABS: this is absextroot.xsub1.xsub2.used
  ABS: this is absextroot.xsub1.xsub2.called.func()

  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config extensions.absextroot=$TESTTMP/absextroot showrelative)
  LIB: this is extlibroot.lsub1.lsub2.used
  LIB: this is extlibroot.lsub1.lsub2.called.func()
  LIB: this is extlibroot.recursedown.abs.used
  LIB: this is extlibroot.recursedown.legacy.used
  LIB: this is extlibroot.shadowing.used
  REL: this is absextroot.xsub1.xsub2.used
  REL: this is absextroot.xsub1.xsub2.called.func()
  REL: this relimporter imports 'this is absextroot.relimportee'

Examine whether sub-module is imported relatively as expected.

See also issue5208 for detail about example case on Python 3.x.

  $ f -q $TESTTMP/extlibroot/lsub1/lsub2/notexist.py
  $TESTTMP/extlibroot/lsub1/lsub2/notexist.py: file not found

  $ cat > $TESTTMP/notexist.py <<NO_CHECK_EOF
  > text = 'notexist.py at root is loaded unintentionally\n'
  > NO_CHECK_EOF

  $ cat > $TESTTMP/checkrelativity.py <<NO_CHECK_EOF
  > from mercurial import registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > 
  > # demand import avoids failure of importing notexist here
  > import extlibroot.lsub1.lsub2.notexist
  > 
  > @command(b'checkrelativity', [], norepo=True)
  > def checkrelativity(ui, *args, **opts):
  >     try:
  >         ui.write(extlibroot.lsub1.lsub2.notexist.text)
  >         return 1 # unintentional success
  >     except ImportError:
  >         pass # intentional failure
  > NO_CHECK_EOF

  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config extensions.checkrelativity=$TESTTMP/checkrelativity.py checkrelativity)

#endif

(Here, module importing tests are finished. Therefore, use other than
NO_CHECK_* limit mark for heredoc python files, in order to apply
import-checker.py or so on their contents)

Make sure a broken uisetup doesn't globally break hg:
  $ cat > $TESTTMP/baduisetup.py <<EOF
  > def uisetup(ui):
  >     1/0
  > EOF

Even though the extension fails during uisetup, hg is still basically usable:
  $ hg --config extensions.baduisetup=$TESTTMP/baduisetup.py version
  Traceback (most recent call last):
    File "*/mercurial/extensions.py", line *, in _runuisetup (glob)
      uisetup(ui)
    File "$TESTTMP/baduisetup.py", line 2, in uisetup
      1/0
  ZeroDivisionError: * by zero (glob)
  *** failed to set up extension baduisetup: * by zero (glob)
  Mercurial Distributed SCM (version *) (glob)
  (see https://mercurial-scm.org for more information)
  
  Copyright (C) 2005-* Matt Mackall and others (glob)
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

  $ cd ..

hide outer repo
  $ hg init

  $ cat > empty.py <<EOF
  > '''empty cmdtable
  > '''
  > cmdtable = {}
  > EOF
  $ emptypath=`pwd`/empty.py
  $ echo "empty = $emptypath" >> $HGRCPATH
  $ hg help empty
  empty extension - empty cmdtable
  
  no commands defined


  $ echo 'empty = !' >> $HGRCPATH

  $ cat > debugextension.py <<EOF
  > '''only debugcommands
  > '''
  > from mercurial import registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > @command(b'debugfoobar', [], b'hg debugfoobar')
  > def debugfoobar(ui, repo, *args, **opts):
  >     "yet another debug command"
  >     pass
  > @command(b'foo', [], b'hg foo')
  > def foo(ui, repo, *args, **opts):
  >     """yet another foo command
  >     This command has been DEPRECATED since forever.
  >     """
  >     pass
  > EOF
  $ debugpath=`pwd`/debugextension.py
  $ echo "debugextension = $debugpath" >> $HGRCPATH

  $ hg help debugextension
  hg debugextensions
  
  show information about active extensions
  
  options:
  
   -T --template TEMPLATE display with template
  
  (some details hidden, use --verbose to show complete help)


  $ hg --verbose help debugextension
  hg debugextensions
  
  show information about active extensions
  
  options:
  
   -T --template TEMPLATE display with template
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)






  $ hg --debug help debugextension
  hg debugextensions
  
  show information about active extensions
  
  options:
  
   -T --template TEMPLATE display with template
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)





  $ echo 'debugextension = !' >> $HGRCPATH

Asking for help about a deprecated extension should do something useful:

  $ hg help glog
  'glog' is provided by the following extension:
  
      graphlog      command to view revision graphs from a shell (DEPRECATED)
  
  (use 'hg help extensions' for information on enabling extensions)

Extension module help vs command help:

  $ echo 'extdiff =' >> $HGRCPATH
  $ hg help extdiff
  hg extdiff [OPT]... [FILE]...
  
  use external program to diff repository (or selected files)
  
      Show differences between revisions for the specified files, using an
      external program. The default program used is diff, with default options
      "-Npru".
  
      To select a different program, use the -p/--program option. The program
      will be passed the names of two directories to compare. To pass additional
      options to the program, use -o/--option. These will be passed before the
      names of the directories to compare.
  
      When two revision arguments are given, then changes are shown between
      those revisions. If only one revision is specified then that revision is
      compared to the working directory, and, when no revisions are specified,
      the working directory files are compared to its parent.
  
  (use 'hg help -e extdiff' to show help for the extdiff extension)
  
  options ([+] can be repeated):
  
   -p --program CMD         comparison program to run
   -o --option OPT [+]      pass option to comparison program
   -r --rev REV [+]         revision
   -c --change REV          change made by revision
      --patch               compare patches for two revisions
   -I --include PATTERN [+] include names matching the given patterns
   -X --exclude PATTERN [+] exclude names matching the given patterns
   -S --subrepos            recurse into subrepositories
  
  (some details hidden, use --verbose to show complete help)










  $ hg help --extension extdiff
  extdiff extension - command to allow external programs to compare revisions
  
  The extdiff Mercurial extension allows you to use external programs to compare
  revisions, or revision with working directory. The external diff programs are
  called with a configurable set of options and two non-option arguments: paths
  to directories containing snapshots of files to compare.
  
  If there is more than one file being compared and the "child" revision is the
  working directory, any modifications made in the external diff program will be
  copied back to the working directory from the temporary directory.
  
  The extdiff extension also allows you to configure new diff commands, so you
  do not need to type 'hg extdiff -p kdiff3' always.
  
    [extdiff]
    # add new command that runs GNU diff(1) in 'context diff' mode
    cdiff = gdiff -Nprc5
    ## or the old way:
    #cmd.cdiff = gdiff
    #opts.cdiff = -Nprc5
  
    # add new command called meld, runs meld (no need to name twice).  If
    # the meld executable is not available, the meld tool in [merge-tools]
    # will be used, if available
    meld =
  
    # add new command called vimdiff, runs gvimdiff with DirDiff plugin
    # (see http://www.vim.org/scripts/script.php?script_id=102) Non
    # English user, be sure to put "let g:DirDiffDynamicDiffText = 1" in
    # your .vimrc
    vimdiff = gvim -f "+next" \
              "+execute 'DirDiff' fnameescape(argv(0)) fnameescape(argv(1))"
  
  Tool arguments can include variables that are expanded at runtime:
  
    $parent1, $plabel1 - filename, descriptive label of first parent
    $child,   $clabel  - filename, descriptive label of child revision
    $parent2, $plabel2 - filename, descriptive label of second parent
    $root              - repository root
    $parent is an alias for $parent1.
  
  The extdiff extension will look in your [diff-tools] and [merge-tools]
  sections for diff tool arguments, when none are specified in [extdiff].
  
    [extdiff]
    kdiff3 =
  
    [diff-tools]
    kdiff3.diffargs=--L1 '$plabel1' --L2 '$clabel' $parent $child
  
  You can use -I/-X and list of file or directory names like normal 'hg diff'
  command. The extdiff extension makes snapshots of only needed files, so
  running the external diff program will actually be pretty fast (at least
  faster than having to compare the entire tree).
  
  list of commands:
  
   extdiff       use external program to diff repository (or selected files)
  
  (use 'hg help -v -e extdiff' to show built-in aliases and global options)
















  $ echo 'extdiff = !' >> $HGRCPATH

Test help topic with same name as extension

  $ cat > multirevs.py <<EOF
  > from mercurial import commands, registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > """multirevs extension
  > Big multi-line module docstring."""
  > @command(b'multirevs', [], b'ARG', norepo=True)
  > def multirevs(ui, repo, arg, *args, **opts):
  >     """multirevs command"""
  >     pass
  > EOF
  $ echo "multirevs = multirevs.py" >> $HGRCPATH

  $ hg help multirevs | tail
        used):
  
          hg update :@
  
      - Show diff between tags 1.3 and 1.5 (this works because the first and the
        last revisions of the revset are used):
  
          hg diff -r 1.3::1.5
  
  use 'hg help -c multirevs' to see help for the multirevs command






  $ hg help -c multirevs
  hg multirevs ARG
  
  multirevs command
  
  (some details hidden, use --verbose to show complete help)



  $ hg multirevs
  hg multirevs: invalid arguments
  hg multirevs ARG
  
  multirevs command
  
  (use 'hg multirevs -h' to show more help)
  [255]



  $ echo "multirevs = !" >> $HGRCPATH

Issue811: Problem loading extensions twice (by site and by user)

  $ cat <<EOF >> $HGRCPATH
  > mq =
  > strip =
  > hgext.mq =
  > hgext/mq =
  > EOF

Show extensions:
(note that mq force load strip, also checking it's not loaded twice)

#if no-extraextensions
  $ hg debugextensions
  mq
  strip
#endif

For extensions, which name matches one of its commands, help
message should ask '-v -e' to get list of built-in aliases
along with extension help itself

  $ mkdir $TESTTMP/d
  $ cat > $TESTTMP/d/dodo.py <<EOF
  > """
  > This is an awesome 'dodo' extension. It does nothing and
  > writes 'Foo foo'
  > """
  > from mercurial import commands, registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > @command(b'dodo', [], b'hg dodo')
  > def dodo(ui, *args, **kwargs):
  >     """Does nothing"""
  >     ui.write(b"I do nothing. Yay\\n")
  > @command(b'foofoo', [], b'hg foofoo')
  > def foofoo(ui, *args, **kwargs):
  >     """Writes 'Foo foo'"""
  >     ui.write(b"Foo foo\\n")
  > EOF
  $ dodopath=$TESTTMP/d/dodo.py

  $ echo "dodo = $dodopath" >> $HGRCPATH

Make sure that user is asked to enter '-v -e' to get list of built-in aliases
  $ hg help -e dodo
  dodo extension -
  
  This is an awesome 'dodo' extension. It does nothing and writes 'Foo foo'
  
  list of commands:
  
   dodo          Does nothing
   foofoo        Writes 'Foo foo'
  
  (use 'hg help -v -e dodo' to show built-in aliases and global options)

Make sure that '-v -e' prints list of built-in aliases along with
extension help itself
  $ hg help -v -e dodo
  dodo extension -
  
  This is an awesome 'dodo' extension. It does nothing and writes 'Foo foo'
  
  list of commands:
  
   dodo          Does nothing
   foofoo        Writes 'Foo foo'
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)

Make sure that single '-v' option shows help and built-ins only for 'dodo' command
  $ hg help -v dodo
  hg dodo
  
  Does nothing
  
  (use 'hg help -e dodo' to show help for the dodo extension)
  
  options:
  
    --mq operate on patch repository
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)

In case when extension name doesn't match any of its commands,
help message should ask for '-v' to get list of built-in aliases
along with extension help
  $ cat > $TESTTMP/d/dudu.py <<EOF
  > """
  > This is an awesome 'dudu' extension. It does something and
  > also writes 'Beep beep'
  > """
  > from mercurial import commands, registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > @command(b'something', [], b'hg something')
  > def something(ui, *args, **kwargs):
  >     """Does something"""
  >     ui.write(b"I do something. Yaaay\\n")
  > @command(b'beep', [], b'hg beep')
  > def beep(ui, *args, **kwargs):
  >     """Writes 'Beep beep'"""
  >     ui.write(b"Beep beep\\n")
  > EOF
  $ dudupath=$TESTTMP/d/dudu.py

  $ echo "dudu = $dudupath" >> $HGRCPATH

  $ hg help -e dudu
  dudu extension -
  
  This is an awesome 'dudu' extension. It does something and also writes 'Beep
  beep'
  
  list of commands:
  
   beep          Writes 'Beep beep'
   something     Does something
  
  (use 'hg help -v dudu' to show built-in aliases and global options)

In case when extension name doesn't match any of its commands,
help options '-v' and '-v -e' should be equivalent
  $ hg help -v dudu
  dudu extension -
  
  This is an awesome 'dudu' extension. It does something and also writes 'Beep
  beep'
  
  list of commands:
  
   beep          Writes 'Beep beep'
   something     Does something
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)

  $ hg help -v -e dudu
  dudu extension -
  
  This is an awesome 'dudu' extension. It does something and also writes 'Beep
  beep'
  
  list of commands:
  
   beep          Writes 'Beep beep'
   something     Does something
  
  global options ([+] can be repeated):
  
   -R --repository REPO   repository root directory or name of overlay bundle
                          file
      --cwd DIR           change working directory
   -y --noninteractive    do not prompt, automatically pick the first choice for
                          all prompts
   -q --quiet             suppress output
   -v --verbose           enable additional output
      --color TYPE        when to colorize (boolean, always, auto, never, or
                          debug)
      --config CONFIG [+] set/override config option (use 'section.name=value')
      --debug             enable debugging output
      --debugger          start debugger
      --encoding ENCODE   set the charset encoding (default: ascii)
      --encodingmode MODE set the charset encoding mode (default: strict)
      --traceback         always print a traceback on exception
      --time              time how long the command takes
      --profile           print command execution profile
      --version           output version information and exit
   -h --help              display help and exit
      --hidden            consider hidden changesets
      --pager TYPE        when to paginate (boolean, always, auto, or never)
                          (default: auto)

Disabled extension commands:

  $ ORGHGRCPATH=$HGRCPATH
  $ HGRCPATH=
  $ export HGRCPATH
  $ hg help email
  'email' is provided by the following extension:
  
      patchbomb     command to send changesets as (a series of) patch emails
  
  (use 'hg help extensions' for information on enabling extensions)


  $ hg qdel
  hg: unknown command 'qdel'
  'qdelete' is provided by the following extension:
  
      mq            manage a stack of patches
  
  (use 'hg help extensions' for information on enabling extensions)
  [255]


  $ hg churn
  hg: unknown command 'churn'
  'churn' is provided by the following extension:
  
      churn         command to display statistics about repository history
  
  (use 'hg help extensions' for information on enabling extensions)
  [255]



Disabled extensions:

  $ hg help churn
  churn extension - command to display statistics about repository history
  
  (use 'hg help extensions' for information on enabling extensions)

  $ hg help patchbomb
  patchbomb extension - command to send changesets as (a series of) patch emails
  
  The series is started off with a "[PATCH 0 of N]" introduction, which
  describes the series as a whole.
  
  Each patch email has a Subject line of "[PATCH M of N] ...", using the first
  line of the changeset description as the subject text. The message contains
  two or three body parts:
  
  - The changeset description.
  - [Optional] The result of running diffstat on the patch.
  - The patch itself, as generated by 'hg export'.
  
  Each message refers to the first in the series using the In-Reply-To and
  References headers, so they will show up as a sequence in threaded mail and
  news readers, and in mail archives.
  
  To configure other defaults, add a section like this to your configuration
  file:
  
    [email]
    from = My Name <my@email>
    to = recipient1, recipient2, ...
    cc = cc1, cc2, ...
    bcc = bcc1, bcc2, ...
    reply-to = address1, address2, ...
  
  Use "[patchbomb]" as configuration section name if you need to override global
  "[email]" address settings.
  
  Then you can use the 'hg email' command to mail a series of changesets as a
  patchbomb.
  
  You can also either configure the method option in the email section to be a
  sendmail compatible mailer or fill out the [smtp] section so that the
  patchbomb extension can automatically send patchbombs directly from the
  commandline. See the [email] and [smtp] sections in hgrc(5) for details.
  
  By default, 'hg email' will prompt for a "To" or "CC" header if you do not
  supply one via configuration or the command line.  You can override this to
  never prompt by configuring an empty value:
  
    [email]
    cc =
  
  You can control the default inclusion of an introduction message with the
  "patchbomb.intro" configuration option. The configuration is always
  overwritten by command line flags like --intro and --desc:
  
    [patchbomb]
    intro=auto   # include introduction message if more than 1 patch (default)
    intro=never  # never include an introduction message
    intro=always # always include an introduction message
  
  You can specify a template for flags to be added in subject prefixes. Flags
  specified by --flag option are exported as "{flags}" keyword:
  
    [patchbomb]
    flagtemplate = "{separate(' ',
                              ifeq(branch, 'default', '', branch|upper),
                              flags)}"
  
  You can set patchbomb to always ask for confirmation by setting
  "patchbomb.confirm" to true.
  
  (use 'hg help extensions' for information on enabling extensions)


Broken disabled extension and command:

  $ mkdir hgext
  $ echo > hgext/__init__.py
  $ cat > hgext/broken.py <<NO_CHECK_EOF
  > "broken extension'
  > NO_CHECK_EOF
  $ cat > path.py <<EOF
  > import os, sys
  > sys.path.insert(0, os.environ['HGEXTPATH'])
  > EOF
  $ HGEXTPATH=`pwd`
  $ export HGEXTPATH

  $ hg --config extensions.path=./path.py help broken
  broken extension - (no help text available)
  
  (use 'hg help extensions' for information on enabling extensions)


  $ cat > hgext/forest.py <<EOF
  > cmdtable = None
  > @command()
  > def f():
  >     pass
  > @command(123)
  > def g():
  >     pass
  > EOF
  $ hg --config extensions.path=./path.py help foo
  abort: no such help topic: foo
  (try 'hg help --keyword foo')
  [255]

  $ cat > throw.py <<EOF
  > from mercurial import commands, registrar, util
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > class Bogon(Exception): pass
  > @command(b'throw', [], b'hg throw', norepo=True)
  > def throw(ui, **opts):
  >     """throws an exception"""
  >     raise Bogon()
  > EOF

No declared supported version, extension complains:
  $ hg --config extensions.throw=throw.py throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension throw
  ** which supports versions unknown of Mercurial.
  ** Please disable throw and try your action again.
  ** If that fixes the bug please report it to the extension author.
  ** Python * (glob)
  ** Mercurial Distributed SCM * (glob)
  ** Extensions loaded: throw

empty declaration of supported version, extension complains:
  $ echo "testedwith = ''" >> throw.py
  $ hg --config extensions.throw=throw.py throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension throw
  ** which supports versions unknown of Mercurial.
  ** Please disable throw and try your action again.
  ** If that fixes the bug please report it to the extension author.
  ** Python * (glob)
  ** Mercurial Distributed SCM (*) (glob)
  ** Extensions loaded: throw

If the extension specifies a buglink, show that:
  $ echo 'buglink = "http://example.com/bts"' >> throw.py
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension throw
  ** which supports versions unknown of Mercurial.
  ** Please disable throw and try your action again.
  ** If that fixes the bug please report it to http://example.com/bts
  ** Python * (glob)
  ** Mercurial Distributed SCM (*) (glob)
  ** Extensions loaded: throw

If the extensions declare outdated versions, accuse the older extension first:
  $ echo "from mercurial import util" >> older.py
  $ echo "util.version = lambda:b'2.2'" >> older.py
  $ echo "testedwith = b'1.9.3'" >> older.py
  $ echo "testedwith = b'2.1.1'" >> throw.py
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py --config extensions.older=older.py \
  >   throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension older
  ** which supports versions 1.9 of Mercurial.
  ** Please disable older and try your action again.
  ** If that fixes the bug please report it to the extension author.
  ** Python * (glob)
  ** Mercurial Distributed SCM (version 2.2)
  ** Extensions loaded: throw, older

One extension only tested with older, one only with newer versions:
  $ echo "util.version = lambda:b'2.1'" >> older.py
  $ rm -f older.pyc older.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py --config extensions.older=older.py \
  >   throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension older
  ** which supports versions 1.9 of Mercurial.
  ** Please disable older and try your action again.
  ** If that fixes the bug please report it to the extension author.
  ** Python * (glob)
  ** Mercurial Distributed SCM (version 2.1)
  ** Extensions loaded: throw, older

Older extension is tested with current version, the other only with newer:
  $ echo "util.version = lambda:b'1.9.3'" >> older.py
  $ rm -f older.pyc older.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py --config extensions.older=older.py \
  >   throw 2>&1 | egrep '^\*\*'
  ** Unknown exception encountered with possibly-broken third-party extension throw
  ** which supports versions 2.1 of Mercurial.
  ** Please disable throw and try your action again.
  ** If that fixes the bug please report it to http://example.com/bts
  ** Python * (glob)
  ** Mercurial Distributed SCM (version 1.9.3)
  ** Extensions loaded: throw, older

Ability to point to a different point
  $ hg --config extensions.throw=throw.py --config extensions.older=older.py \
  >   --config ui.supportcontact='Your Local Goat Lenders' throw 2>&1 | egrep '^\*\*'
  ** unknown exception encountered, please report by visiting
  ** Your Local Goat Lenders
  ** Python * (glob)
  ** Mercurial Distributed SCM (*) (glob)
  ** Extensions loaded: throw, older

Declare the version as supporting this hg version, show regular bts link:
  $ hgver=`hg debuginstall -T '{hgver}'`
  $ echo 'testedwith = """'"$hgver"'"""' >> throw.py
  $ if [ -z "$hgver" ]; then
  >   echo "unable to fetch a mercurial version. Make sure __version__ is correct";
  > fi
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py throw 2>&1 | egrep '^\*\*'
  ** unknown exception encountered, please report by visiting
  ** https://mercurial-scm.org/wiki/BugTracker
  ** Python * (glob)
  ** Mercurial Distributed SCM (*) (glob)
  ** Extensions loaded: throw

Patch version is ignored during compatibility check
  $ echo "testedwith = b'3.2'" >> throw.py
  $ echo "util.version = lambda:b'3.2.2'" >> throw.py
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg --config extensions.throw=throw.py throw 2>&1 | egrep '^\*\*'
  ** unknown exception encountered, please report by visiting
  ** https://mercurial-scm.org/wiki/BugTracker
  ** Python * (glob)
  ** Mercurial Distributed SCM (*) (glob)
  ** Extensions loaded: throw

Test version number support in 'hg version':
  $ echo '__version__ = (1, 2, 3)' >> throw.py
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg version -v
  Mercurial Distributed SCM (version *) (glob)
  (see https://mercurial-scm.org for more information)
  
  Copyright (C) 2005-* Matt Mackall and others (glob)
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  
  Enabled extensions:
  

  $ hg version -v --config extensions.throw=throw.py
  Mercurial Distributed SCM (version *) (glob)
  (see https://mercurial-scm.org for more information)
  
  Copyright (C) 2005-* Matt Mackall and others (glob)
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  
  Enabled extensions:
  
    throw  external  1.2.3
  $ echo 'getversion = lambda: b"1.twentythree"' >> throw.py
  $ rm -f throw.pyc throw.pyo
  $ rm -Rf __pycache__
  $ hg version -v --config extensions.throw=throw.py --config extensions.strip=
  Mercurial Distributed SCM (version *) (glob)
  (see https://mercurial-scm.org for more information)
  
  Copyright (C) 2005-* Matt Mackall and others (glob)
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  
  Enabled extensions:
  
    throw  external  1.twentythree
    strip  internal  

  $ hg version -q --config extensions.throw=throw.py
  Mercurial Distributed SCM (version *) (glob)

Test template output:

  $ hg version --config extensions.strip= -T'{extensions}'
  strip

Test JSON output of version:

  $ hg version -Tjson
  [
   {
    "extensions": [],
    "ver": "*" (glob)
   }
  ]

  $ hg version --config extensions.throw=throw.py -Tjson
  [
   {
    "extensions": [{"bundled": false, "name": "throw", "ver": "1.twentythree"}],
    "ver": "3.2.2"
   }
  ]

  $ hg version --config extensions.strip= -Tjson
  [
   {
    "extensions": [{"bundled": true, "name": "strip", "ver": null}],
    "ver": "*" (glob)
   }
  ]

Test template output of version:

  $ hg version --config extensions.throw=throw.py --config extensions.strip= \
  > -T'{extensions % "{name}  {pad(ver, 16)}  ({if(bundled, "internal", "external")})\n"}'
  throw  1.twentythree     (external)
  strip                    (internal)

Refuse to load extensions with minimum version requirements

  $ cat > minversion1.py << EOF
  > from mercurial import util
  > util.version = lambda: b'3.5.2'
  > minimumhgversion = b'3.6'
  > EOF
  $ hg --config extensions.minversion=minversion1.py version
  (third party extension minversion requires version 3.6 or newer of Mercurial; disabling)
  Mercurial Distributed SCM (version 3.5.2)
  (see https://mercurial-scm.org for more information)
  
  Copyright (C) 2005-* Matt Mackall and others (glob)
  This is free software; see the source for copying conditions. There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

  $ cat > minversion2.py << EOF
  > from mercurial import util
  > util.version = lambda: b'3.6'
  > minimumhgversion = b'3.7'
  > EOF
  $ hg --config extensions.minversion=minversion2.py version 2>&1 | egrep '\(third'
  (third party extension minversion requires version 3.7 or newer of Mercurial; disabling)

Can load version that is only off by point release

  $ cat > minversion2.py << EOF
  > from mercurial import util
  > util.version = lambda: b'3.6.1'
  > minimumhgversion = b'3.6'
  > EOF
  $ hg --config extensions.minversion=minversion3.py version 2>&1 | egrep '\(third'
  [1]

Can load minimum version identical to current

  $ cat > minversion3.py << EOF
  > from mercurial import util
  > util.version = lambda: b'3.5'
  > minimumhgversion = b'3.5'
  > EOF
  $ hg --config extensions.minversion=minversion3.py version 2>&1 | egrep '\(third'
  [1]

Restore HGRCPATH

  $ HGRCPATH=$ORGHGRCPATH
  $ export HGRCPATH

Commands handling multiple repositories at a time should invoke only
"reposetup()" of extensions enabling in the target repository.

  $ mkdir reposetup-test
  $ cd reposetup-test

  $ cat > $TESTTMP/reposetuptest.py <<EOF
  > from mercurial import extensions
  > def reposetup(ui, repo):
  >     ui.write(b'reposetup() for %s\n' % (repo.root))
  >     ui.flush()
  > EOF
  $ hg init src
  $ echo a > src/a
  $ hg -R src commit -Am '#0 at src/a'
  adding a
  $ echo '[extensions]' >> src/.hg/hgrc
  $ echo '# enable extension locally' >> src/.hg/hgrc
  $ echo "reposetuptest = $TESTTMP/reposetuptest.py" >> src/.hg/hgrc
  $ hg -R src status
  reposetup() for $TESTTMP/reposetup-test/src
  reposetup() for $TESTTMP/reposetup-test/src (chg !)

#if no-extraextensions
  $ hg --cwd src debugextensions
  reposetup() for $TESTTMP/reposetup-test/src
  dodo (untested!)
  dudu (untested!)
  mq
  reposetuptest (untested!)
  strip
#endif

  $ hg clone -U src clone-dst1
  reposetup() for $TESTTMP/reposetup-test/src
  $ hg init push-dst1
  $ hg -q -R src push push-dst1
  reposetup() for $TESTTMP/reposetup-test/src
  $ hg init pull-src1
  $ hg -q -R pull-src1 pull src
  reposetup() for $TESTTMP/reposetup-test/src

  $ cat <<EOF >> $HGRCPATH
  > [extensions]
  > # disable extension globally and explicitly
  > reposetuptest = !
  > EOF
  $ hg clone -U src clone-dst2
  reposetup() for $TESTTMP/reposetup-test/src
  $ hg init push-dst2
  $ hg -q -R src push push-dst2
  reposetup() for $TESTTMP/reposetup-test/src
  $ hg init pull-src2
  $ hg -q -R pull-src2 pull src
  reposetup() for $TESTTMP/reposetup-test/src

  $ cat <<EOF >> $HGRCPATH
  > [extensions]
  > # enable extension globally
  > reposetuptest = $TESTTMP/reposetuptest.py
  > EOF
  $ hg clone -U src clone-dst3
  reposetup() for $TESTTMP/reposetup-test/src
  reposetup() for $TESTTMP/reposetup-test/clone-dst3
  $ hg init push-dst3
  reposetup() for $TESTTMP/reposetup-test/push-dst3
  $ hg -q -R src push push-dst3
  reposetup() for $TESTTMP/reposetup-test/src
  reposetup() for $TESTTMP/reposetup-test/push-dst3
  $ hg init pull-src3
  reposetup() for $TESTTMP/reposetup-test/pull-src3
  $ hg -q -R pull-src3 pull src
  reposetup() for $TESTTMP/reposetup-test/pull-src3
  reposetup() for $TESTTMP/reposetup-test/src

  $ echo '[extensions]' >> src/.hg/hgrc
  $ echo '# disable extension locally' >> src/.hg/hgrc
  $ echo 'reposetuptest = !' >> src/.hg/hgrc
  $ hg clone -U src clone-dst4
  reposetup() for $TESTTMP/reposetup-test/clone-dst4
  $ hg init push-dst4
  reposetup() for $TESTTMP/reposetup-test/push-dst4
  $ hg -q -R src push push-dst4
  reposetup() for $TESTTMP/reposetup-test/push-dst4
  $ hg init pull-src4
  reposetup() for $TESTTMP/reposetup-test/pull-src4
  $ hg -q -R pull-src4 pull src
  reposetup() for $TESTTMP/reposetup-test/pull-src4

disabling in command line overlays with all configuration
  $ hg --config extensions.reposetuptest=! clone -U src clone-dst5
  $ hg --config extensions.reposetuptest=! init push-dst5
  $ hg --config extensions.reposetuptest=! -q -R src push push-dst5
  $ hg --config extensions.reposetuptest=! init pull-src5
  $ hg --config extensions.reposetuptest=! -q -R pull-src5 pull src

  $ cat <<EOF >> $HGRCPATH
  > [extensions]
  > # disable extension globally and explicitly
  > reposetuptest = !
  > EOF
  $ hg init parent
  $ hg init parent/sub1
  $ echo 1 > parent/sub1/1
  $ hg -R parent/sub1 commit -Am '#0 at parent/sub1'
  adding 1
  $ hg init parent/sub2
  $ hg init parent/sub2/sub21
  $ echo 21 > parent/sub2/sub21/21
  $ hg -R parent/sub2/sub21 commit -Am '#0 at parent/sub2/sub21'
  adding 21
  $ cat > parent/sub2/.hgsub <<EOF
  > sub21 = sub21
  > EOF
  $ hg -R parent/sub2 commit -Am '#0 at parent/sub2'
  adding .hgsub
  $ hg init parent/sub3
  $ echo 3 > parent/sub3/3
  $ hg -R parent/sub3 commit -Am '#0 at parent/sub3'
  adding 3
  $ cat > parent/.hgsub <<EOF
  > sub1 = sub1
  > sub2 = sub2
  > sub3 = sub3
  > EOF
  $ hg -R parent commit -Am '#0 at parent'
  adding .hgsub
  $ echo '[extensions]' >> parent/.hg/hgrc
  $ echo '# enable extension locally' >> parent/.hg/hgrc
  $ echo "reposetuptest = $TESTTMP/reposetuptest.py" >> parent/.hg/hgrc
  $ cp parent/.hg/hgrc parent/sub2/.hg/hgrc
  $ hg -R parent status -S -A
  reposetup() for $TESTTMP/reposetup-test/parent
  reposetup() for $TESTTMP/reposetup-test/parent/sub2
  C .hgsub
  C .hgsubstate
  C sub1/1
  C sub2/.hgsub
  C sub2/.hgsubstate
  C sub2/sub21/21
  C sub3/3

  $ cd ..

Prohibit registration of commands that don't use @command (issue5137)

  $ hg init deprecated
  $ cd deprecated

  $ cat <<EOF > deprecatedcmd.py
  > def deprecatedcmd(repo, ui):
  >     pass
  > cmdtable = {
  >     b'deprecatedcmd': (deprecatedcmd, [], b''),
  > }
  > EOF
  $ cat <<EOF > .hg/hgrc
  > [extensions]
  > deprecatedcmd = `pwd`/deprecatedcmd.py
  > mq = !
  > hgext.mq = !
  > hgext/mq = !
  > EOF

  $ hg deprecatedcmd > /dev/null
  *** failed to import extension deprecatedcmd from $TESTTMP/deprecated/deprecatedcmd.py: missing attributes: norepo, optionalrepo, inferrepo
  *** (use @command decorator to register 'deprecatedcmd')
  hg: unknown command 'deprecatedcmd'
  (use 'hg help' for a list of commands)
  [255]

 the extension shouldn't be loaded at all so the mq works:

  $ hg qseries --config extensions.mq= > /dev/null
  *** failed to import extension deprecatedcmd from $TESTTMP/deprecated/deprecatedcmd.py: missing attributes: norepo, optionalrepo, inferrepo
  *** (use @command decorator to register 'deprecatedcmd')

  $ cd ..

Test synopsis and docstring extending

  $ hg init exthelp
  $ cat > exthelp.py <<EOF
  > from mercurial import commands, extensions
  > def exbookmarks(orig, *args, **opts):
  >     return orig(*args, **opts)
  > def uisetup(ui):
  >     synopsis = b' GREPME [--foo] [-x]'
  >     docstring = '''
  >     GREPME make sure that this is in the help!
  >     '''
  >     extensions.wrapcommand(commands.table, b'bookmarks', exbookmarks,
  >                            synopsis, docstring)
  > EOF
  $ abspath=`pwd`/exthelp.py
  $ echo '[extensions]' >> $HGRCPATH
  $ echo "exthelp = $abspath" >> $HGRCPATH
  $ cd exthelp
  $ hg help bookmarks | grep GREPME
  hg bookmarks [OPTIONS]... [NAME]... GREPME [--foo] [-x]
      GREPME make sure that this is in the help!
  $ cd ..

Show deprecation warning for the use of cmdutil.command

  $ cat > nonregistrar.py <<EOF
  > from mercurial import cmdutil
  > cmdtable = {}
  > command = cmdutil.command(cmdtable)
  > @command(b'foo', [], norepo=True)
  > def foo(ui):
  >     pass
  > EOF

Prohibit the use of unicode strings as the default value of options

  $ hg init $TESTTMP/opt-unicode-default

  $ cat > $TESTTMP/test_unicode_default_value.py << EOF
  > from __future__ import print_function
  > from mercurial import registrar
  > cmdtable = {}
  > command = registrar.command(cmdtable)
  > @command(b'dummy', [(b'', b'opt', u'value', u'help')], 'ext [OPTIONS]')
  > def ext(*args, **opts):
  >     print(opts[b'opt'], flush=True)
  > EOF
  $ $PYTHON $TESTTMP/unflush.py $TESTTMP/test_unicode_default_value.py
  $ cat > $TESTTMP/opt-unicode-default/.hg/hgrc << EOF
  > [extensions]
  > test_unicode_default_value = $TESTTMP/test_unicode_default_value.py
  > EOF
  $ hg -R $TESTTMP/opt-unicode-default dummy
  *** failed to import extension test_unicode_default_value from $TESTTMP/test_unicode_default_value.py: unicode *'value' found in cmdtable.dummy (glob)
  *** (use b'' to make it byte string)
  hg: unknown command 'dummy'
  (did you mean summary?)
  [255]