Mon, 14 Nov 2016 23:12:11 +0000 worker: migrate to util.iterfile
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:12:11 +0000] rev 30406
worker: migrate to util.iterfile
Mon, 14 Nov 2016 23:32:54 +0000 util: add iterfile to workaround a fileobj.__iter__ issue with EINTR
Jun Wu <quark@fb.com> [Mon, 14 Nov 2016 23:32:54 +0000] rev 30405
util: add iterfile to workaround a fileobj.__iter__ issue with EINTR The fileobj.__iter__ implementation in Python 2.7.12 (hg changeset 45d4cea97b04) is buggy: it cannot handle EINTR correctly. In Objects/fileobject.c: size_t Py_UniversalNewlineFread(....) { .... if (!f->f_univ_newline) return fread(buf, 1, n, stream); .... } According to the "fread" man page: If an error occurs, or the end of the file is reached, the return value is a short item count (or zero). Therefore it's possible for "fread" (and "Py_UniversalNewlineFread") to return a positive value while errno is set to EINTR and ferror(stream) changes from zero to non-zero. There are multiple "Py_UniversalNewlineFread": "file_read", "file_readinto", "file_readlines", "readahead". While the first 3 have code to handle the EINTR case, the last one "readahead" doesn't: static int readahead(PyFileObject *f, Py_ssize_t bufsize) { .... chunksize = Py_UniversalNewlineFread( f->f_buf, bufsize, f->f_fp, (PyObject *)f); .... if (chunksize == 0) { if (ferror(f->f_fp)) { PyErr_SetFromErrno(PyExc_IOError); .... } } .... } It means "readahead" could ignore EINTR, if "Py_UniversalNewlineFread" returns a non-zero value. And at the next time "readahead" got executed, if "Py_UniversalNewlineFread" returns 0, "readahead" would raise a Python error without a incorrect errno - could be 0 - thus "IOError: [Errno 0] Error". The only user of "readahead" is "readahead_get_line_skip". The only user of "readahead_get_line_skip" is "file_iternext", aka. "fileobj.__iter__", which should be avoided. There are multiple places where the pattern "for x in fp" is used. This patch adds a "iterfile" method in "util.py" so we can migrate our code from "for x in fp" to "fox x in util.iterfile(fp)".
Thu, 10 Nov 2016 16:37:18 -0500 filterpyflakes: whitelist listcomp aliasing checking
Augie Fackler <augie@google.com> [Thu, 10 Nov 2016 16:37:18 -0500] rev 30404
filterpyflakes: whitelist listcomp aliasing checking The test change is because of how filterpyflakes is organized - a line number changed.
Thu, 10 Nov 2016 16:35:54 -0500 verify: avoid shadowing two variables with a list comprehension
Augie Fackler <augie@google.com> [Thu, 10 Nov 2016 16:35:54 -0500] rev 30403
verify: avoid shadowing two variables with a list comprehension The variable names are clearly worse now, but since we're really just transposing key and value I'm not too worried about the clarity loss.
Thu, 10 Nov 2016 16:35:10 -0500 revset: avoid shadowing a variable with a list comprehension
Augie Fackler <augie@google.com> [Thu, 10 Nov 2016 16:35:10 -0500] rev 30402
revset: avoid shadowing a variable with a list comprehension
Thu, 10 Nov 2016 16:34:43 -0500 revlog: avoid shadowing several variables using list comprehensions
Augie Fackler <augie@google.com> [Thu, 10 Nov 2016 16:34:43 -0500] rev 30401
revlog: avoid shadowing several variables using list comprehensions
(0) -30000 -10000 -3000 -1000 -300 -100 -30 -10 -6 +6 +10 +30 +100 +300 +1000 +3000 +10000 tip