contrib/benchmarks/__init__.py
author Yuya Nishihara <yuya@tcha.org>
Tue, 31 Oct 2017 22:37:30 +0900
branchstable
changeset 34947 ff2110eadbfa
parent 30588 be0e7af80543
child 43076 2372284d9457
permissions -rw-r--r--
configitems: relax warning about unwanted default value The original condition was a bit harsh for extension authors since third-party extensions need to preserve compatibility with older Mercurial versions, where no defaults would be loaded from the configtable. So let's silence the warning if the given default value matches, which should be harmless.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     1
# __init__.py - asv benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     2
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     3
# Copyright 2016 Logilab SA <contact@logilab.fr>
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     4
#
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
     7
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     8
# "historical portability" policy of contrib/benchmarks:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
     9
#
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    10
# We have to make this code work correctly with current mercurial stable branch
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    11
# and if possible with reasonable cost with early Mercurial versions.
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    12
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    13
'''ASV (https://asv.readthedocs.io) benchmark suite
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    14
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    15
Benchmark are parameterized against reference repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    16
directory pointed by the REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    17
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    18
Invocation example:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    19
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    20
    $ export REPOS_DIR=~/hgperf/repos
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    21
    # run suite on given revision
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    22
    $ asv --config contrib/asv.conf.json run REV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    23
    # run suite on new changesets found in stable and default branch
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    24
    $ asv --config contrib/asv.conf.json run NEW
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    25
    # display a comparative result table of benchmark results between two given
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    26
    # revisions
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    27
    $ asv --config contrib/asv.conf.json compare REV1 REV2
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    28
    # compute regression detection and generate ASV static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    29
    $ asv --config contrib/asv.conf.json publish
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    30
    # serve the static website
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    31
    $ asv --config contrib/asv.conf.json preview
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    32
'''
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    33
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    34
from __future__ import absolute_import
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    35
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    36
import functools
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    37
import os
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    38
import re
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    39
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    40
from mercurial import (
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    41
    extensions,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    42
    hg,
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    43
    ui as uimod,
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    44
    util,
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    45
)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    46
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    47
basedir = os.path.abspath(os.path.join(os.path.dirname(__file__),
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    48
                          os.path.pardir, os.path.pardir))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    49
reposdir = os.environ['REPOS_DIR']
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    50
reposnames = [name for name in os.listdir(reposdir)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    51
              if os.path.isdir(os.path.join(reposdir, name, ".hg"))]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    52
if not reposnames:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    53
    raise ValueError("No repositories found in $REPO_DIR")
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    54
outputre = re.compile((r'! wall (\d+.\d+) comb \d+.\d+ user \d+.\d+ sys '
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    55
                       r'\d+.\d+ \(best of \d+\)'))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    56
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    57
def runperfcommand(reponame, command, *args, **kwargs):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    58
    os.environ["HGRCPATH"] = os.environ.get("ASVHGRCPATH", "")
30588
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    59
    # for "historical portability"
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    60
    # ui.load() has been available since d83ca85
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    61
    if util.safehasattr(uimod.ui, "load"):
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    62
        ui = uimod.ui.load()
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    63
    else:
be0e7af80543 perf: add historical support of ui.load()
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents: 30559
diff changeset
    64
        ui = uimod.ui()
30406
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    65
    repo = hg.repository(ui, os.path.join(reposdir, reponame))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    66
    perfext = extensions.load(ui, 'perfext',
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    67
                              os.path.join(basedir, 'contrib', 'perf.py'))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    68
    cmd = getattr(perfext, command)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    69
    ui.pushbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    70
    cmd(ui, repo, *args, **kwargs)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    71
    output = ui.popbuffer()
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    72
    match = outputre.search(output)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    73
    if not match:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    74
        raise ValueError("Invalid output {0}".format(output))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    75
    return float(match.group(1))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    76
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    77
def perfbench(repos=reposnames, name=None, params=None):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    78
    """decorator to declare ASV benchmark based on contrib/perf.py extension
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    79
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    80
    An ASV benchmark is a python function with the given attributes:
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    81
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    82
    __name__: should start with track_, time_ or mem_ to be collected by ASV
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    83
    params and param_name: parameter matrix to display multiple graphs on the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    84
    same page.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    85
    pretty_name: If defined it's displayed in web-ui instead of __name__
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    86
    (useful for revsets)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    87
    the module name is prepended to the benchmark name and displayed as
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    88
    "category" in webui.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    89
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    90
    Benchmarks are automatically parameterized with repositories found in the
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    91
    REPOS_DIR environment variable.
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    92
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    93
    `params` is the param matrix in the form of a list of tuple
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    94
    (param_name, [value0, value1])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    95
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    96
    For example [(x, [a, b]), (y, [c, d])] declare benchmarks for
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    97
    (a, c), (a, d), (b, c) and (b, d).
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    98
    """
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
    99
    params = list(params or [])
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   100
    params.insert(0, ("repo", repos))
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   101
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   102
    def decorator(func):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   103
        @functools.wraps(func)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   104
        def wrapped(repo, *args):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   105
            def perf(command, *a, **kw):
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   106
                return runperfcommand(repo, command, *a, **kw)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   107
            return func(perf, *args)
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   108
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   109
        wrapped.params = [p[1] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   110
        wrapped.param_names = [p[0] for p in params]
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   111
        wrapped.pretty_name = name
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   112
        return wrapped
cff0f5926797 perf: add asv benchmarks
Philippe Pepiot <philippe.pepiot@logilab.fr>
parents:
diff changeset
   113
    return decorator