util: add iterfile to workaround a fileobj.__iter__ issue with EINTR
The fileobj.__iter__ implementation in Python 2.7.12 (hg changeset
45d4cea97b04) is buggy: it cannot handle EINTR correctly.
In Objects/fileobject.c:
size_t Py_UniversalNewlineFread(....) {
....
if (!f->f_univ_newline)
return fread(buf, 1, n, stream);
....
}
According to the "fread" man page:
If an error occurs, or the end of the file is reached, the return value
is a short item count (or zero).
Therefore it's possible for "fread" (and "Py_UniversalNewlineFread") to
return a positive value while errno is set to EINTR and ferror(stream)
changes from zero to non-zero.
There are multiple "Py_UniversalNewlineFread": "file_read", "file_readinto",
"file_readlines", "readahead". While the first 3 have code to handle the
EINTR case, the last one "readahead" doesn't:
static int readahead(PyFileObject *f, Py_ssize_t bufsize) {
....
chunksize = Py_UniversalNewlineFread(
f->f_buf, bufsize, f->f_fp, (PyObject *)f);
....
if (chunksize == 0) {
if (ferror(f->f_fp)) {
PyErr_SetFromErrno(PyExc_IOError);
....
}
}
....
}
It means "readahead" could ignore EINTR, if "Py_UniversalNewlineFread"
returns a non-zero value. And at the next time "readahead" got executed, if
"Py_UniversalNewlineFread" returns 0, "readahead" would raise a Python error
without a incorrect errno - could be 0 - thus "IOError: [Errno 0] Error".
The only user of "readahead" is "readahead_get_line_skip".
The only user of "readahead_get_line_skip" is "file_iternext", aka.
"fileobj.__iter__", which should be avoided.
There are multiple places where the pattern "for x in fp" is used. This
patch adds a "iterfile" method in "util.py" so we can migrate our code from
"for x in fp" to "fox x in util.iterfile(fp)".
Tests for the automv extension; detect moved files at commit time.
$ cat >> $HGRCPATH << EOF
> [extensions]
> automv=
> rebase=
> EOF
Setup repo
$ hg init repo
$ cd repo
Test automv command for commit
$ printf 'foo\nbar\nbaz\n' > a.txt
$ hg add a.txt
$ hg commit -m 'init repo with a'
mv/rm/add
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit -m 'msg'
detected move of 1 files
$ hg status --change . -C
A b.txt
a.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
mv/rm/add/modif
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit -m 'msg'
detected move of 1 files
created new head
$ hg status --change . -C
A b.txt
a.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
mv/rm/add/modif
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\nfoo\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit -m 'msg'
created new head
$ hg status --change . -C
A b.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
mv/rm/add/modif/changethreshold
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\nfoo\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --config automv.similarity='60' -m 'msg'
detected move of 1 files
created new head
$ hg status --change . -C
A b.txt
a.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
mv
$ mv a.txt b.txt
$ hg status -C
! a.txt
? b.txt
$ hg commit -m 'msg'
nothing changed (1 missing files, see 'hg status')
[1]
$ hg status -C
! a.txt
? b.txt
$ hg revert -aqC
$ rm b.txt
mv/rm/add/notincommitfiles
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ echo 'bar' > c.txt
$ hg add c.txt
$ hg status -C
A b.txt
A c.txt
R a.txt
$ hg commit c.txt -m 'msg'
created new head
$ hg status --change . -C
A c.txt
$ hg status -C
A b.txt
R a.txt
$ hg up -r 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg rm a.txt
$ echo 'bar' > c.txt
$ hg add c.txt
$ hg commit -m 'msg'
detected move of 1 files
created new head
$ hg status --change . -C
A b.txt
a.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv/rm/add/--no-automv
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --no-automv -m 'msg'
created new head
$ hg status --change . -C
A b.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
Test automv command for commit --amend
mv/rm/add
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend -m 'amended'
detected move of 1 files
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
a.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv/rm/add/modif
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend -m 'amended'
detected move of 1 files
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
a.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv/rm/add/modif
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\nfoo\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend -m 'amended'
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv/rm/add/modif/changethreshold
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ printf '\nfoo\n' >> b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend --config automv.similarity='60' -m 'amended'
detected move of 1 files
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
a.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg status -C
! a.txt
? b.txt
$ hg commit --amend -m 'amended'
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status -C
! a.txt
? b.txt
$ hg up -Cr 0
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
mv/rm/add/notincommitfiles
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ echo 'bar' > d.txt
$ hg add d.txt
$ hg status -C
A b.txt
A d.txt
R a.txt
$ hg commit --amend -m 'amended' d.txt
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A c.txt
A d.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend -m 'amended'
detected move of 1 files
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
a.txt
A c.txt
A d.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 3 files removed, 0 files unresolved
mv/rm/add/--no-automv
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg add b.txt
$ hg status -C
A b.txt
R a.txt
$ hg commit --amend -m 'amended' --no-automv
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
A c.txt
R a.txt
$ hg up -r 0
1 files updated, 0 files merged, 2 files removed, 0 files unresolved
mv/rm/commit/add/amend
$ echo 'c' > c.txt
$ hg add c.txt
$ hg commit -m 'revision to amend to'
created new head
$ mv a.txt b.txt
$ hg rm a.txt
$ hg status -C
R a.txt
? b.txt
$ hg commit -m "removed a"
$ hg add b.txt
$ hg commit --amend -m 'amended'
saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-amend-backup.hg (glob)
$ hg status --change . -C
A b.txt
R a.txt
error conditions
$ cat >> $HGRCPATH << EOF
> [automv]
> similarity=110
> EOF
$ hg commit -m 'revision to amend to'
abort: automv.similarity must be between 0 and 100
[255]