Mon, 14 Nov 2016 23:16:05 +0000 match: migrate to util.iterfile
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:16:05 +0000] rev 30409
match: migrate to util.iterfile
Mon, 14 Nov 2016 23:15:01 +0000 store: migrate to util.iterfile
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:15:01 +0000] rev 30408
store: migrate to util.iterfile
Mon, 14 Nov 2016 23:14:06 +0000 patch: migrate to util.iterfile
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:14:06 +0000] rev 30407
patch: migrate to util.iterfile
Mon, 14 Nov 2016 23:12:11 +0000 worker: migrate to util.iterfile
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:12:11 +0000] rev 30406
worker: migrate to util.iterfile
Mon, 14 Nov 2016 23:32:54 +0000 util: add iterfile to workaround a fileobj.__iter__ issue with EINTR
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:32:54 +0000] rev 30405
util: add iterfile to workaround a fileobj.__iter__ issue with EINTR The fileobj.__iter__ implementation in Python 2.7.12 (hg changeset 45d4cea97b04) is buggy: it cannot handle EINTR correctly. In Objects/fileobject.c: size_t Py_UniversalNewlineFread(....) { .... if (!f->f_univ_newline) return fread(buf, 1, n, stream); .... } According to the "fread" man page: If an error occurs, or the end of the file is reached, the return value is a short item count (or zero). Therefore it's possible for "fread" (and "Py_UniversalNewlineFread") to return a positive value while errno is set to EINTR and ferror(stream) changes from zero to non-zero. There are multiple "Py_UniversalNewlineFread": "file_read", "file_readinto", "file_readlines", "readahead". While the first 3 have code to handle the EINTR case, the last one "readahead" doesn't: static int readahead(PyFileObject *f, Py_ssize_t bufsize) { .... chunksize = Py_UniversalNewlineFread( f->f_buf, bufsize, f->f_fp, (PyObject *)f); .... if (chunksize == 0) { if (ferror(f->f_fp)) { PyErr_SetFromErrno(PyExc_IOError); .... } } .... } It means "readahead" could ignore EINTR, if "Py_UniversalNewlineFread" returns a non-zero value. And at the next time "readahead" got executed, if "Py_UniversalNewlineFread" returns 0, "readahead" would raise a Python error without a incorrect errno - could be 0 - thus "IOError: [Errno 0] Error". The only user of "readahead" is "readahead_get_line_skip". The only user of "readahead_get_line_skip" is "file_iternext", aka. "fileobj.__iter__", which should be avoided. There are multiple places where the pattern "for x in fp" is used. This patch adds a "iterfile" method in "util.py" so we can migrate our code from "for x in fp" to "fox x in util.iterfile(fp)".
Thu, 10 Nov 2016 16:37:18 -0500 filterpyflakes: whitelist listcomp aliasing checking
Augie Fackler <augie@google.com> [Thu, 10 Nov 2016 16:37:18 -0500] rev 30404
filterpyflakes: whitelist listcomp aliasing checking The test change is because of how filterpyflakes is organized - a line number changed.
(0) -30000 -10000 -3000 -1000 -300 -100 -30 -10 -6 +6 +10 +30 +100 +300 +1000 +3000 +10000 tip