view tests/heredoctest.py @ 17962:4c29668ca316 stable

util: make chunkbuffer non-quadratic on Windows

The old str-based += collector performed very nicely on Linux, but
turns out to be quadratically expensive on Windows, causing
chunkbuffer to dominate in profiles.

This list-based version has been measured to significantly improve
performance with large chunks on Windows, with negligible overall
overhead on Linux (though microbenchmarks show it to be about 50% slower).

This may increase memory overhead where += didn't behave quadratically. If we
want to gather up 1G of data to join, we temporarily have 1G in our
list and 1G in our string.

import sys

globalvars = {}
localvars = {}
lines = sys.stdin.readlines()
while lines:
    l = lines.pop(0)
    if l.startswith('SALT'):
        print l[:-1]
    elif l.startswith('>>> '):
        snippet = l[4:]
        while lines and lines[0].startswith('... '):
            l = lines.pop(0)
            snippet += "\n" + l[4:]
        c = compile(snippet, '<heredoc>', 'single')
        try:
            exec c in globalvars, localvars
        except Exception, inst:
            print repr(inst)