view mercurial/helptext/internals/extensions.txt @ 45095:8e04607023e5

procutil: ensure that procutil.std{out,err}.write() writes all bytes

Python 3 offers different kind of streams and it’s not guaranteed for all of
them that calling write() writes all bytes.

When Python is started in unbuffered mode, sys.std{out,err}.buffer are
instances of io.FileIO, whose write() can write less bytes for
platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
write less if interrupted by a signal; when writing to Windows consoles, it’s
limited to 32767 bytes to avoid the "not enough space" error). This can lead to
silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
not be a buffered stream) and when using the text streams sys.std{out,err}
(I’ve created a CPython bug report for that:
https://bugs.python.org/issue41221).

Python may fix the problem at some point. For now, we implement our own wrapper
for procutil.std{out,err} that calls the raw stream’s write() method until all
bytes have been written. We don’t use sys.std{out,err} for larger writes, so I
think it’s not worth the effort to patch them.

Extensions allow the creation of new features and using them directly from
the main hg command line as if they were built-in commands. The extensions
have full access to the *internal* API.

Use of Mercurial's internal API very likely makes your code subject to
Mercurial's license. Before going any further, read the License page.

There are NO guarantees that third-party code calling into Mercurial's
internals won't break from release to release. If you do use Mercurial's API
for published third-party code, we expect you to test your code before each
major Mercurial release. This will prevent various bug reports from your users
when they upgrade their copy of Mercurial.

File Layout
===========

Extensions are usually written as simple python modules. Larger ones are
better split into multiple modules of a single package (see the convert
extension). The package root module gives its name to the extension and
implements the ``cmdtable`` and optional callbacks described below.

Command table
=============

To write your own extension, your python module can provide an optional dict
named ``cmdtable`` with entries describing each command. A command should be
registered to the ``cmdtable`` by ``@command`` decorator.

Example using ``@command`` decorator (requires Mercurial 1.9)::

    from mercurial.i18n import _

    cmdtable = {}
    try:
        from mercurial import registrar
        command = registrar.command(cmdtable)
    except (AttributeError, ImportError):
        # Fallback to hg < 4.3 support
        from mercurial import cmdutil
        command = cmdutil.command(cmdtable)

    @command('print-parents',
        [('s', 'short', None, _('print short form')),
         ('l', 'long', None, _('print long form'))],
        _('[options] node'))
    def printparents(ui, repo, node, **opts):
        ...

The cmdtable dictionary
-----------------------

The ``cmdtable`` dictionary uses as key the new command names, and, as value,
a tuple containing:

1. the function to be called when the command is used.
2. a list of options the command can take.
3. a command line synopsis for the command (the function docstring is used for
   the full help).

List of options
---------------

All the command flag options are documented in the mercurial/fancyopts.py
sources.

The options list is a list of tuples containing:

1. the short option letter, or ``''`` if no short option is available
   (for example, ``o`` for a ``-o`` option).
2. the long option name (for example, ``option`` for a ``--option`` option).
3. a default value for the option.
4. a help string for the option (it's possible to omit the "hg newcommand"
   part and only the options and parameter substring is needed).

Command function signatures
---------------------------

Functions that implement new commands always receive a ``ui`` and usually
a ``repo`` parameter. The rest of parameters are taken from the command line
items that don't start with a dash and are passed in the same order they were
written. If no default value is given in the parameter list they are required.

If there is no repo to be associated with the command and consequently no
``repo`` passed, then ``norepo=True`` should be passed to the ``@command``
decorator::

    @command('mycommand', [], norepo=True)
    def mycommand(ui, **opts):
        ...

For examples of ``norepo``, see the convert extension.

Command function docstrings
===========================

The docstring of your function is used as the main help text, shown by
``hg help mycommand``. The docstring should be formatted using a simple
subset of reStructuredText markup. The supported constructs include:

Paragraphs::

    This is a paragraph.

    Paragraphs are separated
    by blank lines.

A verbatim block is introduced with a double colon followed by an indented
block. The double colon is turned into a single colon on display::

    Some text::

      verbatim
        text
         !!

We have field lists::

    :key1: value1
    :key2: value2

Bullet lists::

    - foo
    - bar

Enumerated lists::

    1. foo
    2. bar

Inline markup::

    ``*bold*``, ``monospace``, :hg:`command`

Mark Mercurial commands with ``:hg:`` to make a nice link to the corresponding
documentation. We'll expand the support if new constructs can be parsed
without too much trouble.

Communicating with the user
===========================

Besides the ``ui`` methods, like ``ui.write(*msg)`` or
``ui.prompt(msg, default="y")``, an extension can add help text for each
of its commands and the extension itself.

The module docstring will be used as help string when ``hg help extensionname``
is used and, similarly, the help string for a command and the docstring
belonging to the function that's wrapped by the command will be shown when
``hg help command`` is invoked.

Setup Callbacks
===============

Extensions are loaded in phases. All extensions are processed in a given phase
before the next phase begins. In the first phase, all extension modules are
loaded and registered with Mercurial. This means that you can find all enabled
extensions with ``extensions.find`` in the following phases.

Extension setup
---------------

There are two callbacks to be called when extensions are loaded, named
``uisetup`` and ``extsetup``. ``uisetup`` is called first for each extension,
then ``extsetup`` is called. This means ``extsetup`` can be useful in case
one extension optionally depends on another extension.

Both ``uisetup`` and ``extsetup`` receive a ui object with the local
repository configuration::

    def uisetup(ui):
        # ...

    def extsetup(ui):
        # ...

Be aware that ``uisetup`` in NOT the function to configure a ``ui`` instance.
It's called only once per process, not per ``ui`` instance. Also, any changes
to the ``ui`` may be discarded because the ``ui`` here temporarily loaded
local configuration. So, it's generally wrong to do `ui.setconfig()` in
these callbacks. Notable exception is setting ``pre/post-<command>`` hooks
and extending ``ui.__class__``.

In Mercurial 1.3.1 or earlier, ``extsetup`` takes no argument.

Command table setup
-------------------

After ``extsetup``, the ``cmdtable`` is copied into the global command table
in Mercurial.

Ui instance setup
-----------------

The optional ``uipopulate`` is called for each ``ui`` instance after
configuration is loaded, where extensions can set up additional ui members,
update configuration by ``ui.setconfig()``, and extend the class dynamically.

Typically there are three ``ui`` instances involved in command execution:

``req.ui`` (or ``repo.baseui``)
    Only system and user configurations are loaded into it.
``lui``
    Local repository configuration is loaded as well. This will be used at
    early dispatching stage where a repository isn't available.
``repo.ui``
    The fully-loaded ``ui`` used after a repository is instantiated. This
    will be created from the ``req.ui`` per repository.

In command server and hgweb, this may be called more than once for the same
``ui`` instance.

(New in Mercurial 4.9)

Repository setup
----------------

Extensions can implement an optional callback named ``reposetup``. It is
called after the main Mercurial repository initialization, and can be used
to setup any local state the extension might need.

As other command functions it receives an ``ui`` object and a ``repo`` object
(no additional parameters for this, though)::

    def reposetup(ui, repo):
        #do initialization here.

It is important to take into account that the ``ui`` object that is received
by the ``reposetup`` function is not the same as the one received by the
``uisetup`` and ``extsetup`` functions. This is particularly important when
setting up hooks as described in the following section, since not all hooks
use the same ``ui`` object and hence different hooks must be configured in
different setup functions.

Wrapping methods on the ui and repo classes
-------------------------------------------

Because extensions can be loaded *per repository*, you should avoid using
``extensions.wrapfunction()`` on methods of the ``ui`` and ``repo`` objects.
Instead, create a subclass of the specific class of the instance passed into
the ``*setup()`` hook; e.g. use ``ui.__class__`` as the base class, then
reassign your new class to ``ui.__class__`` again. Mercurial will then use
your updated ``ui`` or ``repo`` instance only for repositories where your
extension is enabled (or copies thereof, reusing your new class).

For example::

    def uisetup(ui):
        class echologui(ui.__class__):
            def log(self, service, *msg, **opts):
                if msg:
                    self.write('%s: %s\n' % (service, msg[0] % msg[1:]))
                super(echologui, self).log(service, *msg, **opts)

        ui.__class__ = echologui

Configuring Hooks
=================

Some extensions must use hooks to do their work. These required hooks can
be configured manually by the user by modifying the ``[hook]`` section of
their hgrc, but they can also be configured automatically by calling the
``ui.setconfig('hooks', ...)`` function in one of the setup functions
described above.

The main difference between manually modifying the hooks section in the hgrc
and using ``ui.setconfig()`` is that when using ``ui.setconfig()`` you have
access to the actual hook function object, which you can pass directly to
``ui.setconfig()``, while when you use the hooks section of the hgrc file
you must refer to the hook function by using the
``python:modulename.functioname`` idiom (e.g. ``python:hgext.notify.hook``).

For example::

    # Define hooks -- note that the actual function name it irrelevant.
    def preupdatehook(ui, repo, **kwargs):
        ui.write("Pre-update hook triggered\n")

    def updatehook(ui, repo, **kwargs):
        ui.write("Update hook triggered\n")

    def uisetup(ui):
        # When pre-<cmd> and post-<cmd> hooks are configured by means of
        # the ui.setconfig() function, you must use the ui object passed
        # to uisetup or extsetup.
        ui.setconfig("hooks", "pre-update.myextension", preupdatehook)

    def reposetup(ui, repo):
        # Repository-specific hooks can be configured here. These include
        # the update hook.
        ui.setconfig("hooks", "update.myextension", updatehook)

Note how different hooks may need to be configured in different setup
functions. In the example you can see that the ``update`` hook must be
configured in the ``reposetup`` function, while the ``pre-update`` hook
must be configured on the ``uisetup`` or the ``extsetup`` functions.

Marking compatible versions
===========================

Every extension should use the ``testedwith`` variable to specify Mercurial
releases it's known to be compatible with. This helps us and users diagnose
where problems are coming from::

    testedwith = '2.0 2.0.1 2.1 2.1.1 2.1.2'

Do not use the ``internal`` marker in third-party extensions; we will
immediately drop all bug reports mentioning your extension if we catch you
doing this.

Similarly, an extension can use the ``buglink`` variable to specify how users
should report issues with the extension.  This link will be included in the
error message if the extension produces errors::

    buglink = 'https://bitbucket.org/USER/REPO/issues'

If an extension requires a minimum version of Mercurial, it can be declared
with the ``minimumhgversion`` variable::

    minimumhgversion = '4.6'

Older clients will print a warning that the extension requires a new version,
instead of attempting to load it.

Wrap up: what belongs where?
============================

You will find here a list of most common tasks, based on setups from the
extensions included in Mercurial core.

uisetup
-------

* Changes to ``ui.__class__`` . The ``ui`` object that will be used to run
  the command has not yet been created. Changes made here will affect ``ui``
  objects created after this, and in particular the ``ui`` that will be passed
  to ``runcommand``
* Command wraps (``extensions.wrapcommand``)
* Changes that need to be visible by other extensions: because initialization
  occurs in phases (all extensions run ``uisetup``, then all run ``extsetup``),
  a change made here will be visible by other extensions during ``extsetup``.
* Monkeypatches or function wraps (``extensions.wrapfunction``) of ``dispatch``
  module members
* Set up ``pre-*`` and ``post-*`` hooks. (DEPRECATED. ``uipopulate`` is
  preferred on Mercurial 4.9 and later.)
* ``pushkey`` setup

extsetup
--------

* Changes depending on the status of other extensions. (``if extensions.find('mq')``)
* Add a global option to all commands
* Extend revsets

uipopulate
----------

* Modify ``ui`` instance attributes and configuration variables.
* Changes to ``ui.__class__`` per instance.
* Set up all hooks per scoped configuration.

reposetup
---------

* Set up all hooks but ``pre-*`` and ``post-*``. (DEPRECATED. ``uipopulate`` is
  preferred on Mercurial 4.9 and later.)
* Modify configuration variables
* Changes to ``repo.__class__``, ``repo.dirstate.__class__``