view hgext/remotefilelog/constants.py @ 45095:8e04607023e5

procutil: ensure that procutil.std{out,err}.write() writes all bytes

Python 3 offers different kind of streams and it’s not guaranteed for all of
them that calling write() writes all bytes.

When Python is started in unbuffered mode, sys.std{out,err}.buffer are
instances of io.FileIO, whose write() can write less bytes for
platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
write less if interrupted by a signal; when writing to Windows consoles, it’s
limited to 32767 bytes to avoid the "not enough space" error). This can lead to
silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
not be a buffered stream) and when using the text streams sys.std{out,err}
(I’ve created a CPython bug report for that:
https://bugs.python.org/issue41221).

Python may fix the problem at some point. For now, we implement our own wrapper
for procutil.std{out,err} that calls the raw stream’s write() method until all
bytes have been written. We don’t use sys.std{out,err} for larger writes, so I
think it’s not worth the effort to patch them.

from __future__ import absolute_import

import struct

from mercurial.i18n import _

NETWORK_CAP_LEGACY_SSH_GETFILES = b'exp-remotefilelog-ssh-getfiles-1'

SHALLOWREPO_REQUIREMENT = b"exp-remotefilelog-repo-req-1"

BUNDLE2_CAPABLITY = b"exp-remotefilelog-b2cap-1"

FILENAMESTRUCT = b'!H'
FILENAMESIZE = struct.calcsize(FILENAMESTRUCT)

NODESIZE = 20
PACKREQUESTCOUNTSTRUCT = b'!I'

NODECOUNTSTRUCT = b'!I'
NODECOUNTSIZE = struct.calcsize(NODECOUNTSTRUCT)

PATHCOUNTSTRUCT = b'!I'
PATHCOUNTSIZE = struct.calcsize(PATHCOUNTSTRUCT)

FILEPACK_CATEGORY = b""
TREEPACK_CATEGORY = b"manifests"

ALL_CATEGORIES = [FILEPACK_CATEGORY, TREEPACK_CATEGORY]

# revision metadata keys. must be a single character.
METAKEYFLAG = b'f'  # revlog flag
METAKEYSIZE = b's'  # full rawtext size


def getunits(category):
    if category == FILEPACK_CATEGORY:
        return _(b"files")
    if category == TREEPACK_CATEGORY:
        return _(b"trees")


# Repack options passed to ``markledger``.
OPTION_PACKSONLY = b'packsonly'