view contrib/relnotes @ 41773:37ead13fb3d4

rust-cpython: using rustext.dagop.headrevs in revlog

As with the previous oxidation series, revlog plays the role
of the factory, either using its parents function, or passing the
index.

We include below results of revsetbenchmarks.py taken on the
PyPy repository on those of contrib/all-revsets.tx that involve
`heads()`.

In most of the cases, this seems to be either neutral or an improvement.
In the cases where it's actually a bit slower, we suspect that differences
in `heads()` performance is actually burried in variance on the incoming
revset (probably several orders of magnitude slower).

The precheck for filtered revisions of parent changeset has a significative
performance benefit, too.

Result by revset
================

Revision:
0) 0c7b353ce100; rust-cpython: binding for headrevs()
1) Parent of this changeset; changelog: prefilter in headrevs()
2) This changeset


revset #0: heads(commonancestors(last(head(), 2)))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.001379      0.001361      0.001381      0.001410      0.001393      0.001372      0.001414      0.001387      0.001411      0.001429      0.001415
1) 0.001351      0.001373      0.001383      0.001392      0.001401      0.001385      0.001405      0.001406      0.001385      0.001424      0.001399
2) 0.001365      0.001362      0.001375      0.001393      0.001370      0.001365      0.001413      0.001386      0.001377      0.001415      0.001411

revset #1: heads(commonancestors(head()))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.047578      0.048578      0.047764      0.048065      0.047289      0.047305      0.047729      0.047370      0.047611      0.048005      0.047755
1) 0.048072      0.047471      0.048351      0.048193      0.048380      0.047968      0.047683      0.047355      0.048587      0.047044      0.048299
2) 0.047124      0.046699      0.046896      0.047250      0.046920      0.047379      0.046855      0.047753      0.047289      0.047219      0.046991

revset #2: heads(all())
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.037654      0.037814      0.037149      0.037457      0.037609      0.037053      0.036825      0.037054      0.037739      0.036816      0.037604
1) 0.021845  58% 0.022172  58% 0.022148  59% 0.022059  58% 0.022261  59% 0.022246  60% 0.021691  58% 0.021967  59% 0.022156  58% 0.021820  59% 0.023141  61%
2) 0.014459  66% 0.014470  65% 0.014420  65% 0.014413  65% 0.014421  64% 0.014492  65% 0.014512  66% 0.014579  66% 0.014500  65% 0.014501  66% 0.014537  62%

revset #3: heads(-10000:-1)
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.003696      0.003681      0.003719      0.003746      0.003725      0.003750      0.003692      0.003747      0.003712      0.003754      0.003763
1) 0.002131  57% 0.002142  58% 0.002147  57% 0.002203  58% 0.002143  57% 0.002208  58% 0.002158  58% 0.002182  58% 0.002169  58% 0.002209  58% 0.002201  58%
2) 0.001490  69% 0.001524  71% 0.001515  70% 0.001528  69% 0.001531  71% 0.001520  68% 0.001549  71% 0.001542  70% 0.001560  71% 0.001559  70% 0.001544  70%

revset #4: (-5000:-1000) and heads(-10000:-1)
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.003832      0.003816      0.003747      0.003814      0.003749      0.003894      0.003784      0.003796      0.003915      0.003829      0.003795
1) 0.002282  59% 0.002208  57% 0.002220  59% 0.002240  58% 0.002210  58% 0.002276  58% 0.002250  59% 0.002250  59% 0.002311  59% 0.002230  58% 0.002241  59%
2) 0.001658  72% 0.001662  75% 0.001568  70% 0.001599  71% 0.001588  71% 0.001696  74% 0.001615  71% 0.001593  70% 0.001710  73% 0.001622  72% 0.001616  72%

revset #5: heads(matching(tip, "author"))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 7.826449      7.563260      7.581034      7.688493      7.634001      7.777860      7.768228      8.026097      7.767422      7.565254      7.938643
1) 7.750766      7.562555      7.660426      7.574089      7.492220      7.438582      7.562015      7.530635  93% 7.636343      7.636712      7.645113
2) 7.617941      7.519601      7.584922      7.507653      7.547440      7.524436      7.575291      7.883991      7.792142      7.709622      7.868595

revset #6: heads(matching(tip, "author")) and -10000:-1
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 7.744489      7.728684      7.734065      7.928513      7.875949      7.883727      7.815492      7.791335      7.784793      7.761218      7.815731
1) 7.808956      7.480446      7.618759      7.920270      7.676343      7.803613      7.770210      7.713100      7.584420      7.767335      7.825140
2) 7.519987      7.938748 106% 7.805328      7.694162      7.750129      7.714229      7.603825      7.580734      7.555291      7.524207      7.504580

revset #7: (-10000:-1) and heads(matching(tip, "author"))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 7.909321      7.694357      7.666021      7.538686      7.771821      7.876217      7.852103      7.812727      7.545919      7.788860      7.764585
1) 7.749232      7.683715      7.968393      7.895257      7.764160      8.314884 105% 7.921697      7.882613      7.867209      7.684707      7.544501
2) 7.824903      7.784605      7.727846      7.566613      7.581994      7.539205  90% 7.555316      7.535572      7.581786      7.901795      7.662832

#!/usr/bin/env python3
"""Generate release notes from our commit log.

This uses the relnotes extension directives when they're available,
and falls back to our old pre-relnotes logic that used to live in the
release-tools repo.
"""
import argparse
import re
import subprocess

rules = {
    # keep
    r"\(issue": 100,
    r"\(BC\)": 100,
    r"\(API\)": 100,
    r"\(SEC\)": 100,
    # core commands, bump up
    r"(commit|files|log|pull|push|patch|status|tag|summary)(|s|es):": 20,
    r"(annotate|alias|branch|bookmark|clone|graft|import|verify).*:": 20,
    # extensions, bump up
    r"(mq|shelve|rebase):": 20,
    # newsy
    r": deprecate": 20,
    r": new.*(extension|flag|module)": 10,
    r"( ability|command|feature|option|support)": 10,
    # experimental
    r"hg-experimental": 20,
    r"(from|graduate).*experimental": 15,
    r"(hide|mark).*experimental": -10,
    # bug-like?
    r"(fix|don't break|improve)": 7,
    r"(not|n't|avoid|fix|prevent).*crash": 10,
    r"vulnerab": 10,
    # boring stuff, bump down
    r"^contrib": -5,
    r"debug": -5,
    r"help": -5,
    r"minor": -5,
    r"(doc|metavar|bundle2|obsolete|obsmarker|rpm|setup|debug\S+:)": -15,
    r"(check-code|check-commit|check-config|import-checker)": -20,
    r"(flake8|lintian|pyflakes|pylint)": -20,
    # cleanups and refactoring
    r"(clean ?up|white ?space|spelling|quoting)": -20,
    r"(flatten|dedent|indent|nesting|unnest)": -20,
    r"(typo|hint|note|comment|TODO|FIXME)": -20,
    r"(style:|convention|one-?liner)": -20,
    r"(argument|absolute_import|attribute|assignment|mutable)": -15,
    r"(scope|True|False)": -10,
    r"(unused|useless|unnecessar|superfluous|duplicate|deprecated)": -10,
    r"(redundant|pointless|confusing|uninitialized|meaningless|dead)": -10,
    r": (drop|remove|delete|rip out)": -10,
    r": (inherit|rename|simplify|naming|inline)": -10,
    r"(correct doc|docstring|document .* method)": -20,
    r"(abstract|factor|extract|prepare|split|replace| import)": -20,
    r": add.*(function|method|implementation|example)": -10,
    r": (move|extract) .* (to|into|from|out of)": -20,
    r": implement ": -5,
    r": use .* implementation": -20,
    r": use .* instead of": -20,
    # code
    r"_": -10,
    r"__": -5,
    r"\(\)": -5,
    r"\S\S\S+\.\S\S\S\S+": -5,
    # dumb keywords
    r"\S+/\S+:": -10,
    r"\S+\.\S+:": -10,
    # python compatibility
    r"[Pp]y(|thon) ?[23]": -20,
    r"pycompat": -20,
    r"(coerce|convert|encode) .*to (byte|sys|)(s|str|string)": -20,
    # tests
    r"^test(|s|ing|runner|-\S+):": -20,
    r"^(f|hghave|run-tests):": -20,
    r"add.* tests?": -20,
    r"(buildbot|fuzz|mock|ratchet)": -10,
    # drop
    r"^i18n-": -50,
    r"^i18n:.*(hint|comment)": -50,
    r"perf:": -50,
    r"Added.*for changeset": -50,
    r"^_": -50,
}

cutoff = 10
commits = []

groupings = [
    (r"util|parsers|repo|ctx|context|revlog|filelog|alias|cmdutil", "core"),
    (r"revset|template|ui|dirstate|hook|i18n|transaction|wire|vfs", "core"),
    (r"dispatch|exchange|localrepo|streamclone|color|pager", "core"),
    (r"hgweb|paper|coal|gitweb|monoblue|spartan", "hgweb"),
    (r"pull|push|revert|resolve|annotate|bookmark|branch|clone", "commands"),
    (r"commands|commit|config|files|graft|import|log|merge|patch", "commands"),
    (r"phases|status|summary|amend|tag|help|verify", "commands"),
    (r"rebase|mq|convert|eol|histedit|largefiles", "extensions"),
    (r"shelve|unshelve", "extensions"),
]

def wikify(desc):
    desc = desc.replace("(issue", "(Bts:issue")
    desc = re.sub(r"\b([0-9a-f]{12})\b", r"Cset:\1", desc)
    # stop ParseError from being recognized as a (nonexistent) wiki page
    desc = re.sub(r" ([A-Z][a-z]+[A-Z][a-z]+)\b", r" !\1", desc)
    # prevent wiki markup of magic methods
    desc = re.sub(r"\b(\S*__\S*)\b", r"`\1`", desc)
    return desc

def main():
    desc = "example: %(prog)s 4.7.2 --stoprev 4.8rc0"
    ap = argparse.ArgumentParser(description=desc)
    ap.add_argument(
        "startrev",
        metavar="REV",
        type=str,
        help=(
            "Starting revision for the release notes. This revision "
            "won't be included, but later revisions will."
        ),
    )
    ap.add_argument(
        "--stoprev",
        metavar="REV",
        type=str,
        default="@",
        help=(
            "Stop revision for release notes. This revision will be included,"
            " but no later revisions will. This revision needs to be "
            "a descendant of startrev."
        ),
    )
    args = ap.parse_args()
    fromext = subprocess.check_output(
        [
            "hg",
            "--config",
            "extensions.releasenotes=",
            "releasenotes",
            "-r",
            "only(%s, %s)" % (args.stoprev, args.startrev),
        ]
    ).decode("utf-8")
    # Find all release notes from un-relnotes-flagged commits.
    for entry in sorted(
        subprocess.check_output(
            [
                "hg",
                "log",
                "-r",
                "only(%s, %s) - merge()" % (args.stoprev, args.startrev),
                "-T",
                r"{desc|firstline}\n",
            ]
        )
        .decode("utf-8")
        .splitlines()
    ):
        desc = entry.replace("`", "'")

        score = 0
        for rule, val in rules.items():
            if re.search(rule, desc):
                score += val

        if score >= cutoff:
            commits.append(wikify(desc))
    # Group unflagged notes.
    groups = {}
    bcs = []
    apis = []

    for d in commits:
        if "(BC)" in d:
            bcs.append(d)
        if "(API)" in d:
            apis.append(d)
        for rule, g in groupings:
            if re.match(rule, d):
                groups.setdefault(g, []).append(d)
                break
        else:
            groups.setdefault("unsorted", []).append(d)
    print(fromext)
    # print legacy release notes sections
    for g in sorted(groups):
        print("\n=== %s ===" % g)
        for d in sorted(groups[g]):
            print(" * %s" % d)

    if bcs:
        print("\n=== Behavior Changes ===\n")

    for d in sorted(bcs):
        print(" * %s" % d)

    if apis:
        print("\n=== Internal API Changes ===\n")

    for d in sorted(apis):
        print(" * %s" % d)

if __name__ == "__main__":
    main()