worker: avoid potential partial write of pickled data
Previously, the code wrote the pickled data using os.write(). However,
os.write() can write less bytes than passed to it. To trigger the problem, the
pickled data had to be larger than
2147479552 bytes on my system.
Instead, open a file object and pass it to pickle.dump(). This also has the
advantage that it doesn’t buffer the whole pickled data in memory.
Note that the opened file must be buffered because pickle doesn’t support
unbuffered streams because unbuffered streams’ write() method might write less
bytes than passed to it (like os.write()) but pickle.dump() relies on that all
bytes are written (see https://github.com/python/cpython/issues/93050).
The side effect of using a file object and a with statement is that wfd is
explicitly closed now while it seems like before it was implicitly closed by
process exit.
$ hg init rep
$ cd rep
$ mkdir dir
$ touch foo dir/bar
$ hg -v addremove
adding dir/bar
adding foo
$ hg -v commit -m "add 1"
committing files:
dir/bar
foo
committing manifest
committing changelog
committed changeset 0:6f7f953567a2
$ cd dir/
$ touch ../foo_2 bar_2
$ hg -v addremove
adding dir/bar_2
adding foo_2
$ hg -v commit -m "add 2"
committing files:
dir/bar_2
foo_2
committing manifest
committing changelog
committed changeset 1:e65414bf35c5
$ cd ..
$ hg forget foo
$ hg -v addremove
adding foo
$ hg forget foo
$ hg -v addremove nonexistent
nonexistent: $ENOENT$
[1]
$ cd ..
$ hg init subdir
$ cd subdir
$ mkdir dir
$ cd dir
$ touch a.py
$ hg addremove 'glob:*.py'
adding a.py
$ hg forget a.py
$ hg addremove -I 'glob:*.py'
adding a.py
$ hg forget a.py
$ hg addremove
adding dir/a.py
$ cd ..
$ hg init sim
$ cd sim
$ echo a > a
$ echo a >> a
$ echo a >> a
$ echo c > c
$ hg commit -Ama
adding a
adding c
$ mv a b
$ rm c
$ echo d > d
$ hg addremove -n -s 50 # issue 1696
removing a
adding b
removing c
adding d
recording removal of a as rename to b (100% similar)
$ hg addremove -ns 50 --color debug
[ui.addremove.removed ui.status|removing a]
[ui.addremove.added ui.status|adding b]
[ui.addremove.removed ui.status|removing c]
[ui.addremove.added ui.status|adding d]
[ ui.status|recording removal of a as rename to b (100% similar)]
$ hg addremove -s 50
removing a
adding b
removing c
adding d
recording removal of a as rename to b (100% similar)
$ hg commit -mb
$ cp b c
$ hg forget b
$ hg addremove -s 50
adding b
adding c
$ rm c
$ hg ci -A -m "c" nonexistent
nonexistent: $ENOENT$
abort: failed to mark all new/missing files as added/removed
[255]
$ hg st
! c
$ cd ..