tests/filterpyflakes.py

annotate: do not construct attr.s object per line while computing history

Unfortunately, good abstraction has a cost. It's way slower to construct
an annotateline() object than creating a plain tuple or a list. This patch
changes the internal data structure from row-based to columnar, so the
decorate() function can be instant (i.e. no Python in hot loop.)

For code readability, the outermost tuple is switched to an attr.s object
instead.

  (original, row-based attr.s)
  $ hg annot mercurial/commands.py --time > /dev/null
  time: real 11.470 secs (user 11.400+0.000 sys 0.070+0.000)
  $ hg annot mercurial/commands.py --time --line-number > /dev/null
  time: real 39.590 secs (user 39.500+0.000 sys 0.080+0.000)

  (this patch, columnar)
  $ hg annot mercurial/commands.py --time > /dev/null
  time: real 11.780 secs (user 11.710+0.000 sys 0.070+0.000)
  $ hg annot mercurial/commands.py --time --line-number > /dev/null
  time: real 12.240 secs (user 12.170+0.000 sys 0.090+0.000)

  (cf. 4.3.3, row-based tuple)
  $ hg annot mercurial/commands.py --time --line-number > /dev/null
  time: real 19.540 secs (user 19.460+0.000 sys 0.080+0.000)

#!/usr/bin/env python

# Filter output by pyflakes to control which warnings we check

from __future__ import absolute_import, print_function

import re
import sys

lines = []
for line in sys.stdin:
    # We blacklist tests that are too noisy for us
    pats = [
        r"undefined name 'WindowsError'",
        r"redefinition of unused '[^']+' from line",
        # for cffi, allow re-exports from pure.*
        r"cffi/[^:]*:.*\bimport \*' used",
        r"cffi/[^:]*:.*\*' imported but unused",
    ]

    keep = True
    for pat in pats:
        if re.search(pat, line):
            keep = False
            break # pattern matches
    if keep:
        fn = line.split(':', 1)[0]
        f = open(fn)
        data = f.read()
        f.close()
        if 'no-' 'check-code' in data:
            continue
        lines.append(line)

for line in lines:
    sys.stdout.write(line)
print()