view mercurial/cffi/mpatch.py @ 45095:8e04607023e5

procutil: ensure that procutil.std{out,err}.write() writes all bytes

Python 3 offers different kind of streams and it’s not guaranteed for all of
them that calling write() writes all bytes.

When Python is started in unbuffered mode, sys.std{out,err}.buffer are
instances of io.FileIO, whose write() can write less bytes for
platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
write less if interrupted by a signal; when writing to Windows consoles, it’s
limited to 32767 bytes to avoid the "not enough space" error). This can lead to
silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
not be a buffered stream) and when using the text streams sys.std{out,err}
(I’ve created a CPython bug report for that:
https://bugs.python.org/issue41221).

Python may fix the problem at some point. For now, we implement our own wrapper
for procutil.std{out,err} that calls the raw stream’s write() method until all
bytes have been written. We don’t use sys.std{out,err} for larger writes, so I
think it’s not worth the effort to patch them.

# mpatch.py - CFFI implementation of mpatch.c
#
# Copyright 2016 Maciej Fijalkowski <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from ..pure.mpatch import *
from ..pure.mpatch import mpatchError  # silence pyflakes
from . import _mpatch

ffi = _mpatch.ffi
lib = _mpatch.lib


@ffi.def_extern()
def cffi_get_next_item(arg, pos):
    all, bins = ffi.from_handle(arg)
    container = ffi.new(b"struct mpatch_flist*[1]")
    to_pass = ffi.new(b"char[]", str(bins[pos]))
    all.append(to_pass)
    r = lib.mpatch_decode(to_pass, len(to_pass) - 1, container)
    if r < 0:
        return ffi.NULL
    return container[0]


def patches(text, bins):
    lgt = len(bins)
    all = []
    if not lgt:
        return text
    arg = (all, bins)
    patch = lib.mpatch_fold(ffi.new_handle(arg), lib.cffi_get_next_item, 0, lgt)
    if not patch:
        raise mpatchError(b"cannot decode chunk")
    outlen = lib.mpatch_calcsize(len(text), patch)
    if outlen < 0:
        lib.mpatch_lfree(patch)
        raise mpatchError(b"inconsistency detected")
    buf = ffi.new(b"char[]", outlen)
    if lib.mpatch_apply(buf, text, len(text), patch) < 0:
        lib.mpatch_lfree(patch)
        raise mpatchError(b"error applying patches")
    res = ffi.buffer(buf, outlen)[:]
    lib.mpatch_lfree(patch)
    return res