contrib/automation/README.rst

42311:195dcc10b3d7

Under normal operation, recurring costs are limited to:

* Storage costs for AMI / EBS snapshots. This should be just a few pennies
  per month.

When running EC2 instances, you'll be billed accordingly. By default, we
use *small* instances, like ``t3.medium``. This instance type costs ~$0.07 per
hour.

.. note::

   When running Windows EC2 instances, AWS bills at the full hourly cost, even
   if the instance doesn't run for a full hour (per-second billing doesn't

   $ automation.py terminate-ec2-instances

To purge all EC2 resources that we manage::

   $ automation.py purge-ec2-resources

42312:65b3ef162b39

Under normal operation, recurring costs are limited to:

* Storage costs for AMI / EBS snapshots. This should be just a few pennies
  per month.

When running EC2 instances, you'll be billed accordingly. Default instance
types vary by operation. We try to be respectful of your money when choosing
defaults. e.g. for Windows instances which are billed per hour, we use e.g.
``t3.medium`` instances, which cost ~$0.07 per hour. For operations that
scale well to many CPUs like running Linux tests, we may use a more powerful
instance like ``c5.9xlarge``. However, since Linux instances are billed
per second and the cost of running an e.g. ``c5.9xlarge`` for half the time
of a ``c5.4xlarge`` is roughly the same, the choice is justified.

.. note::

   When running Windows EC2 instances, AWS bills at the full hourly cost, even
   if the instance doesn't run for a full hour (per-second billing doesn't

   $ automation.py terminate-ec2-instances

To purge all EC2 resources that we manage::

   $ automation.py purge-ec2-resources

Remote Machine Interfaces
=========================

The code that connects to a remote machine and executes things is
theoretically machine agnostic as long as the remote machine conforms to
an *interface*. In other words, to perform actions like running tests
remotely or triggering packaging, it shouldn't matter if the remote machine
is an EC2 instance, a virtual machine, etc. This section attempts to document
the interface that remote machines need to provide in order to be valid
*targets* for remote execution. These interfaces are often not ideal nor
the most flexible. Instead, they have often evolved as the requirements of
our automation code have evolved.

Linux
-----

Remote Linux machines expose an SSH server on port 22. The SSH server
must allow the ``hg`` user to authenticate using the SSH key generated by
the automation code. The ``hg`` user should be part of the ``hg`` group
and it should have ``sudo`` access without password prompting.

The SSH channel must support SFTP to facilitate transferring files from
client to server.

``/bin/bash`` must be executable and point to a bash shell executable.

The ``/hgdev`` directory must exist and all its content owned by ``hg::hg``.

The ``/hgdev/pyenv`` directory should contain an installation of
``pyenv``. Various Python distributions should be installed. The exact
versions shouldn't matter. ``pyenv global`` should have been run so
``/hgdev/pyenv/shims/`` is populated with redirector scripts that point
to the appropriate Python executable.

The ``/hgdev/venv-bootstrap`` directory must contain a virtualenv
with Mercurial installed. The ``/hgdev/venv-bootstrap/bin/hg`` executable
is referenced by various scripts and the client.

The ``/hgdev/src`` directory MUST contain a clone of the Mercurial
source code. The state of the working directory is not important.

In order to run tests, the ``/hgwork`` directory will be created.
This may require running various ``mkfs.*`` executables and ``mount``
to provision a new filesystem. This will require elevated privileges
via ``sudo``.

Various dependencies to run the Mercurial test harness are also required.
Documenting them is beyond the scope of this document. Various tests
also require other optional dependencies and missing dependencies will
be printed by the test runner when a test is skipped.