view tests/filterpyflakes.py @ 24710:909ee6b2a024

lazymanifest: prevent leak when updating an entry more than once

__setitem__ on the lazymanifest C type wasn't checking to see if a
line had previously been malloced before replacing it, leading to
leaks if files got updated multiple times in the course of a task.

I was able to reproduce the leak with this change to test-manifest.py:

diff --git a/tests/test-manifest.py b/tests/test-manifest.py
--- a/tests/test-manifest.py
+++ b/tests/test-manifest.py
@@ -456,6 +456,16 @@ class basemanifesttests(object):
                 ['a/b/c/bar.txt', 'a/b/c/foo.txt', 'a/b/d/ten.txt'],
                 m2.keys())

+    def testManifestSetItem(self):
+        m = self.parsemanifest('')
+        for x in range(3):
+            m['file%d' % x] = BIN_HASH_1
+        for x in range(3):
+            m['file%d' % x] = BIN_HASH_2
+        import time
+        time.sleep(4)
+
+

along with the commands:

 $ make local
 $ PYTHONPATH=. SILENT_BE_NOISY=1 python tests/test-manifest.py testmanifestdict.testManifestSetItem &
 $ sleep 4
 $ leaks $(jobs -p | tee /dev/stderr | awk '{print $3}')
 $ wait

in an interactive shell on OS X. As far as I can tell, it had to be an
interactive shell so that I could get the pid of the test run using
the jobs builtin. Prior to this change, I was leaking several strings,
and after this change leaks reports no leaks.

I thought there was a bug filed for this in bugzilla, but I can't find
it either in bugzilla or by searching my email.

#!/usr/bin/env python

# Filter output by pyflakes to control which warnings we check

import sys, re, os

def makekey(typeandline):
    """
    for sorting lines by: msgtype, path/to/file, lineno, message

    typeandline is a sequence of a message type and the entire message line
    the message line format is path/to/file:line: message

    >>> makekey((3, 'example.py:36: any message'))
    (3, 'example.py', 36, ' any message')
    >>> makekey((7, 'path/to/file.py:68: dummy message'))
    (7, 'path/to/file.py', 68, ' dummy message')
    >>> makekey((2, 'fn:88: m')) > makekey((2, 'fn:9: m'))
    True
    """

    msgtype, line = typeandline
    fname, line, message = line.split(":", 2)
    # line as int for ordering 9 before 88
    return msgtype, fname, int(line), message


lines = []
for line in sys.stdin:
    # We whitelist tests (see more messages in pyflakes.messages)
    pats = [
            (r"imported but unused", None),
            (r"local variable '.*' is assigned to but never used", None),
            (r"unable to detect undefined names", None),
            (r"undefined name '.*'",
             r"undefined name '(WindowsError|memoryview)'")
           ]

    for msgtype, (pat, excl) in enumerate(pats):
        if re.search(pat, line) and (not excl or not re.search(excl, line)):
            break # pattern matches
    else:
        continue # no pattern matched, next line
    fn = line.split(':', 1)[0]
    f = open(os.path.join(os.path.dirname(os.path.dirname(__file__)), fn))
    data = f.read()
    f.close()
    if 'no-' 'check-code' in data:
        continue
    lines.append((msgtype, line))

for msgtype, line in sorted(lines, key=makekey):
    sys.stdout.write(line)
print

# self test of "undefined name" detection for other than 'memoryview'
if False:
    print undefinedname