tests/filterpyflakes.py

scmutil: add file object wrapper class to check ambiguity at closing

In Mercurial source tree, opening a file in "a"/"a+" mode like below
doesn't specify atomictemp=True for vfs, and this avoids file stat
ambiguity check by atomictempfile.

  - writing changes out in revlog layer uses "a+" mode
  - truncation in repair.strip() uses "a" mode
  - truncation in transaction._playback() uses "a" mode

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to revlog-style file (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to revlog-style file again

Therefore, cache validation doesn't work after (3) as expected.

This patch adds file object wrapper class checkambigatclosing to check
(and get rid of) ambiguity at closing. It is used by vfs in subsequent
patch.

This is a part of ExactCacheValidationPlan.

    https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

BTW, checkambigatclosing is tested in test-filecache.py, even though
it doesn't use filecache itself, because filecache assumes that file
stat ambiguity never occurs (and there is no another test-*.py related
to filecache).

#!/usr/bin/env python

# Filter output by pyflakes to control which warnings we check

from __future__ import absolute_import, print_function

import re
import sys

def makekey(typeandline):
    """
    for sorting lines by: msgtype, path/to/file, lineno, message

    typeandline is a sequence of a message type and the entire message line
    the message line format is path/to/file:line: message

    >>> makekey((3, 'example.py:36: any message'))
    (3, 'example.py', 36, ' any message')
    >>> makekey((7, 'path/to/file.py:68: dummy message'))
    (7, 'path/to/file.py', 68, ' dummy message')
    >>> makekey((2, 'fn:88: m')) > makekey((2, 'fn:9: m'))
    True
    """

    msgtype, line = typeandline
    fname, line, message = line.split(":", 2)
    # line as int for ordering 9 before 88
    return msgtype, fname, int(line), message


lines = []
for line in sys.stdin:
    # We whitelist tests (see more messages in pyflakes.messages)
    pats = [
            (r"imported but unused", None),
            (r"local variable '.*' is assigned to but never used", None),
            (r"unable to detect undefined names", None),
            (r"undefined name '.*'",
             r"undefined name '(WindowsError|memoryview)'")
           ]

    for msgtype, (pat, excl) in enumerate(pats):
        if re.search(pat, line) and (not excl or not re.search(excl, line)):
            break # pattern matches
    else:
        continue # no pattern matched, next line
    fn = line.split(':', 1)[0]
    f = open(fn)
    data = f.read()
    f.close()
    if 'no-' 'check-code' in data:
        continue
    lines.append((msgtype, line))

for msgtype, line in sorted(lines, key=makekey):
    sys.stdout.write(line)
print()

# self test of "undefined name" detection for other than 'memoryview'
if False:
    print(undefinedname)