Mercurial > hg
view mercurial/registrar.py @ 42285:65b3ef162b39
automation: initial support for running Linux tests
Building on top of our Windows automation support, this commit
implements support for performing automated tasks on remote Linux
machines. Specifically, we implement support for running tests
on ephemeral EC2 instances. This seems to be a worthwhile place
to start, as building packages on Linux is more or less a solved
problem because we already have facilities for building in Docker
containers, which provide "good enough" reproducibility guarantees.
The new `run-tests-linux` command works similarly to
`run-tests-windows`: it ensures an AMI with hg dependencies is
available, provisions a temporary EC2 instance with this AMI, pushes
local changes to that instance via SSH, then invokes `run-tests.py`.
Using this new command, I am able to run the entire test harness
substantially faster then I am on my local machine courtesy of
access to massive core EC2 instances:
wall: 16:20 ./run-tests.py -l (i7-6700K)
wall: 14:00 automation.py run-tests-linux --ec2-instance c5.2xlarge
wall: 8:30 automation.py run-tests-linux --ec2-instance m5.4xlarge
wall: 8:04 automation.py run-tests-linux --ec2-instance c5.4xlarge
wall: 4:30 automation.py run-tests-linux --ec2-instance c5.9xlarge
wall: 3:57 automation.py run-tests-linux --ec2-instance m5.12xlarge
wall: 3:05 automation.py run-tests-linux --ec2-instance m5.24xlarge
wall: 3:02 automation.py run-tests-linux --ec2-instance c5.18xlarge
~3 minute wall time to run pretty much the entire test harness is
not too bad!
The AMIs install multiple versions of Python. And the run-tests-linux
command specifies which one to use:
automation.py run-tests-linux --python system3
automation.py run-tests-linux --python 3.5
automation.py run-tests-linux --python pypy2.7
By default, the system Python 2.7 is used. Using this functionality,
I was able to identity some unexpected test failures on PyPy!
Included in the feature is support for running with alternate
filesystems. You can simply pass --filesystem to the command to
specify the type of filesystem to run tests on. When the ephemeral
instance is started, a new filesystem will be created and tests
will run from it:
wall: 4:30 automation.py run-tests-linux --ec2-instance c5.9xlarge
wall: 4:20 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem xfs
wall: 4:24 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem tmpfs
wall: 4:26 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem ext4
We also support multiple Linux distributions:
$ automation.py run-tests-linux --distro debian9
total time: 298.1s; setup: 60.7s; tests: 237.5s; setup overhead: 20.4%
$ automation.py run-tests-linux --distro ubuntu18.04
total time: 286.1s; setup: 61.3s; tests: 224.7s; setup overhead: 21.4%
$ automation.py run-tests-linux --distro ubuntu18.10
total time: 278.5s; setup: 58.2s; tests: 220.3s; setup overhead: 20.9%
$ automation.py run-tests-linux --distro ubuntu19.04
total time: 265.8s; setup: 42.5s; tests: 223.3s; setup overhead: 16.0%
Debian and Ubuntu are supported because those are what I use and am
most familiar with. It should be easy enough to add support for other
distros.
Unlike the Windows AMIs, Linux EC2 instances bill per second. So
the cost to instantiating an ephemeral instance isn't as severe.
That being said, there is some overhead, as it takes several dozen
seconds for the instance to boot, push local changes, and build
Mercurial. During this time, the instance is largely CPU idle and
wasting money. Even with this inefficiency, running tests is
relatively cheap: $0.15-$0.25 per full test run. A machine running
tests as efficiently as these EC2 instances would cost say $6,000, so
you can run the test harness a >20,000 times for the cost of an
equivalent machine. Running tests in EC2 is almost certainly cheaper
than buying a beefy machine for developers to use :)
# no-check-commit because foo_bar function names
Differential Revision: https://phab.mercurial-scm.org/D6319
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sat, 27 Apr 2019 11:48:26 -0700 |
parents | f8f61cf246f5 |
children | 832c59d1196e |
line wrap: on
line source
# registrar.py - utilities to register function for specific purpose # # Copyright FUJIWARA Katsunori <foozy@lares.dti.ne.jp> and others # # This software may be used and distributed according to the terms of the # GNU General Public License version 2 or any later version. from __future__ import absolute_import from . import ( configitems, error, pycompat, util, ) # unlike the other registered items, config options are neither functions or # classes. Registering the option is just small function call. # # We still add the official API to the registrar module for consistency with # the other items extensions want might to register. configitem = configitems.getitemregister class _funcregistrarbase(object): """Base of decorator to register a function for specific purpose This decorator stores decorated functions into own dict 'table'. The least derived class can be defined by overriding 'formatdoc', for example:: class keyword(_funcregistrarbase): _docformat = ":%s: %s" This should be used as below: keyword = registrar.keyword() @keyword('bar') def barfunc(*args, **kwargs): '''Explanation of bar keyword .... ''' pass In this case: - 'barfunc' is stored as 'bar' in '_table' of an instance 'keyword' above - 'barfunc.__doc__' becomes ":bar: Explanation of bar keyword" """ def __init__(self, table=None): if table is None: self._table = {} else: self._table = table def __call__(self, decl, *args, **kwargs): return lambda func: self._doregister(func, decl, *args, **kwargs) def _doregister(self, func, decl, *args, **kwargs): name = self._getname(decl) if name in self._table: msg = 'duplicate registration for name: "%s"' % name raise error.ProgrammingError(msg) if func.__doc__ and not util.safehasattr(func, '_origdoc'): doc = pycompat.sysbytes(func.__doc__).strip() func._origdoc = doc func.__doc__ = pycompat.sysstr(self._formatdoc(decl, doc)) self._table[name] = func self._extrasetup(name, func, *args, **kwargs) return func def _merge(self, registrarbase): """Merge the entries of the given registrar object into this one. The other registrar object must not contain any entries already in the current one, or a ProgrammmingError is raised. Additionally, the types of the two registrars must match. """ if not isinstance(registrarbase, type(self)): msg = "cannot merge different types of registrar" raise error.ProgrammingError(msg) dups = set(registrarbase._table).intersection(self._table) if dups: msg = 'duplicate registration for names: "%s"' % '", "'.join(dups) raise error.ProgrammingError(msg) self._table.update(registrarbase._table) def _parsefuncdecl(self, decl): """Parse function declaration and return the name of function in it """ i = decl.find('(') if i >= 0: return decl[:i] else: return decl def _getname(self, decl): """Return the name of the registered function from decl Derived class should override this, if it allows more descriptive 'decl' string than just a name. """ return decl _docformat = None def _formatdoc(self, decl, doc): """Return formatted document of the registered function for help 'doc' is '__doc__.strip()' of the registered function. """ return self._docformat % (decl, doc) def _extrasetup(self, name, func): """Execute exra setup for registered function, if needed """ class command(_funcregistrarbase): """Decorator to register a command function to table This class receives a command table as its argument. The table should be a dict. The created object can be used as a decorator for adding commands to that command table. This accepts multiple arguments to define a command. The first argument is the command name (as bytes). The `options` keyword argument is an iterable of tuples defining command arguments. See ``mercurial.fancyopts.fancyopts()`` for the format of each tuple. The `synopsis` argument defines a short, one line summary of how to use the command. This shows up in the help output. There are three arguments that control what repository (if any) is found and passed to the decorated function: `norepo`, `optionalrepo`, and `inferrepo`. The `norepo` argument defines whether the command does not require a local repository. Most commands operate against a repository, thus the default is False. When True, no repository will be passed. The `optionalrepo` argument defines whether the command optionally requires a local repository. If no repository can be found, None will be passed to the decorated function. The `inferrepo` argument defines whether to try to find a repository from the command line arguments. If True, arguments will be examined for potential repository locations. See ``findrepo()``. If a repository is found, it will be used and passed to the decorated function. The `intents` argument defines a set of intended actions or capabilities the command is taking. These intents can be used to affect the construction of the repository object passed to the command. For example, commands declaring that they are read-only could receive a repository that doesn't have any methods allowing repository mutation. Other intents could be used to prevent the command from running if the requested intent could not be fulfilled. If `helpcategory` is set (usually to one of the constants in the help module), the command will be displayed under that category in the help's list of commands. The following intents are defined: readonly The command is read-only The signature of the decorated function looks like this: def cmd(ui[, repo] [, <args>] [, <options>]) `repo` is required if `norepo` is False. `<args>` are positional args (or `*args`) arguments, of non-option arguments from the command line. `<options>` are keyword arguments (or `**options`) of option arguments from the command line. See the WritingExtensions and MercurialApi documentation for more exhaustive descriptions and examples. """ # Command categories for grouping them in help output. # These can also be specified for aliases, like: # [alias] # myalias = something # myalias:category = repo CATEGORY_REPO_CREATION = 'repo' CATEGORY_REMOTE_REPO_MANAGEMENT = 'remote' CATEGORY_COMMITTING = 'commit' CATEGORY_CHANGE_MANAGEMENT = 'management' CATEGORY_CHANGE_ORGANIZATION = 'organization' CATEGORY_FILE_CONTENTS = 'files' CATEGORY_CHANGE_NAVIGATION = 'navigation' CATEGORY_WORKING_DIRECTORY = 'wdir' CATEGORY_IMPORT_EXPORT = 'import' CATEGORY_MAINTENANCE = 'maintenance' CATEGORY_HELP = 'help' CATEGORY_MISC = 'misc' CATEGORY_NONE = 'none' def _doregister(self, func, name, options=(), synopsis=None, norepo=False, optionalrepo=False, inferrepo=False, intents=None, helpcategory=None, helpbasic=False): func.norepo = norepo func.optionalrepo = optionalrepo func.inferrepo = inferrepo func.intents = intents or set() func.helpcategory = helpcategory func.helpbasic = helpbasic if synopsis: self._table[name] = func, list(options), synopsis else: self._table[name] = func, list(options) return func INTENT_READONLY = b'readonly' class revsetpredicate(_funcregistrarbase): """Decorator to register revset predicate Usage:: revsetpredicate = registrar.revsetpredicate() @revsetpredicate('mypredicate(arg1, arg2[, arg3])') def mypredicatefunc(repo, subset, x): '''Explanation of this revset predicate .... ''' pass The first string argument is used also in online help. Optional argument 'safe' indicates whether a predicate is safe for DoS attack (False by default). Optional argument 'takeorder' indicates whether a predicate function takes ordering policy as the last argument. Optional argument 'weight' indicates the estimated run-time cost, useful for static optimization, default is 1. Higher weight means more expensive. Usually, revsets that are fast and return only one revision has a weight of 0.5 (ex. a symbol); revsets with O(changelog) complexity and read only the changelog have weight 10 (ex. author); revsets reading manifest deltas have weight 30 (ex. adds); revset reading manifest contents have weight 100 (ex. contains). Note: those values are flexible. If the revset has a same big-O time complexity as 'contains', but with a smaller constant, it might have a weight of 90. 'revsetpredicate' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'revsetpredicate' is used for decorating in extension. Otherwise, explicit 'revset.loadpredicate()' is needed. """ _getname = _funcregistrarbase._parsefuncdecl _docformat = "``%s``\n %s" def _extrasetup(self, name, func, safe=False, takeorder=False, weight=1): func._safe = safe func._takeorder = takeorder func._weight = weight class filesetpredicate(_funcregistrarbase): """Decorator to register fileset predicate Usage:: filesetpredicate = registrar.filesetpredicate() @filesetpredicate('mypredicate()') def mypredicatefunc(mctx, x): '''Explanation of this fileset predicate .... ''' pass The first string argument is used also in online help. Optional argument 'callstatus' indicates whether a predicate implies 'matchctx.status()' at runtime or not (False, by default). Optional argument 'weight' indicates the estimated run-time cost, useful for static optimization, default is 1. Higher weight means more expensive. There are predefined weights in the 'filesetlang' module. ====== ============================================================= Weight Description and examples ====== ============================================================= 0.5 basic match patterns (e.g. a symbol) 10 computing status (e.g. added()) or accessing a few files 30 reading file content for each (e.g. grep()) 50 scanning working directory (ignored()) ====== ============================================================= 'filesetpredicate' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'filesetpredicate' is used for decorating in extension. Otherwise, explicit 'fileset.loadpredicate()' is needed. """ _getname = _funcregistrarbase._parsefuncdecl _docformat = "``%s``\n %s" def _extrasetup(self, name, func, callstatus=False, weight=1): func._callstatus = callstatus func._weight = weight class _templateregistrarbase(_funcregistrarbase): """Base of decorator to register functions as template specific one """ _docformat = ":%s: %s" class templatekeyword(_templateregistrarbase): """Decorator to register template keyword Usage:: templatekeyword = registrar.templatekeyword() # new API (since Mercurial 4.6) @templatekeyword('mykeyword', requires={'repo', 'ctx'}) def mykeywordfunc(context, mapping): '''Explanation of this template keyword .... ''' pass # old API (DEPRECATED) @templatekeyword('mykeyword') def mykeywordfunc(repo, ctx, templ, cache, revcache, **args): '''Explanation of this template keyword .... ''' pass The first string argument is used also in online help. Optional argument 'requires' should be a collection of resource names which the template keyword depends on. This also serves as a flag to switch to the new API. If 'requires' is unspecified, all template keywords and resources are expanded to the function arguments. 'templatekeyword' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'templatekeyword' is used for decorating in extension. Otherwise, explicit 'templatekw.loadkeyword()' is needed. """ def _extrasetup(self, name, func, requires=None): func._requires = requires class templatefilter(_templateregistrarbase): """Decorator to register template filer Usage:: templatefilter = registrar.templatefilter() @templatefilter('myfilter', intype=bytes) def myfilterfunc(text): '''Explanation of this template filter .... ''' pass The first string argument is used also in online help. Optional argument 'intype' defines the type of the input argument, which should be (bytes, int, templateutil.date, or None for any.) 'templatefilter' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'templatefilter' is used for decorating in extension. Otherwise, explicit 'templatefilters.loadkeyword()' is needed. """ def _extrasetup(self, name, func, intype=None): func._intype = intype class templatefunc(_templateregistrarbase): """Decorator to register template function Usage:: templatefunc = registrar.templatefunc() @templatefunc('myfunc(arg1, arg2[, arg3])', argspec='arg1 arg2 arg3', requires={'ctx'}) def myfuncfunc(context, mapping, args): '''Explanation of this template function .... ''' pass The first string argument is used also in online help. If optional 'argspec' is defined, the function will receive 'args' as a dict of named arguments. Otherwise 'args' is a list of positional arguments. Optional argument 'requires' should be a collection of resource names which the template function depends on. 'templatefunc' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'templatefunc' is used for decorating in extension. Otherwise, explicit 'templatefuncs.loadfunction()' is needed. """ _getname = _funcregistrarbase._parsefuncdecl def _extrasetup(self, name, func, argspec=None, requires=()): func._argspec = argspec func._requires = requires class internalmerge(_funcregistrarbase): """Decorator to register in-process merge tool Usage:: internalmerge = registrar.internalmerge() @internalmerge('mymerge', internalmerge.mergeonly, onfailure=None, precheck=None, binary=False, symlink=False): def mymergefunc(repo, mynode, orig, fcd, fco, fca, toolconf, files, labels=None): '''Explanation of this internal merge tool .... ''' return 1, False # means "conflicted", "no deletion needed" The first string argument is used to compose actual merge tool name, ":name" and "internal:name" (the latter is historical one). The second argument is one of merge types below: ========== ======== ======== ========= merge type precheck premerge fullmerge ========== ======== ======== ========= nomerge x x x mergeonly o x o fullmerge o o o ========== ======== ======== ========= Optional argument 'onfailure' is the format of warning message to be used at failure of merging (target filename is specified at formatting). Or, None or so, if warning message should be suppressed. Optional argument 'precheck' is the function to be used before actual invocation of internal merge tool itself. It takes as same arguments as internal merge tool does, other than 'files' and 'labels'. If it returns false value, merging is aborted immediately (and file is marked as "unresolved"). Optional argument 'binary' is a binary files capability of internal merge tool. 'nomerge' merge type implies binary=True. Optional argument 'symlink' is a symlinks capability of inetrnal merge function. 'nomerge' merge type implies symlink=True. 'internalmerge' instance in example above can be used to decorate multiple functions. Decorated functions are registered automatically at loading extension, if an instance named as 'internalmerge' is used for decorating in extension. Otherwise, explicit 'filemerge.loadinternalmerge()' is needed. """ _docformat = "``:%s``\n %s" # merge type definitions: nomerge = None mergeonly = 'mergeonly' # just the full merge, no premerge fullmerge = 'fullmerge' # both premerge and merge def _extrasetup(self, name, func, mergetype, onfailure=None, precheck=None, binary=False, symlink=False): func.mergetype = mergetype func.onfailure = onfailure func.precheck = precheck binarycap = binary or mergetype == self.nomerge symlinkcap = symlink or mergetype == self.nomerge # actual capabilities, which this internal merge tool has func.capabilities = {"binary": binarycap, "symlink": symlinkcap}