contrib/packaging/build-linux-wheels.sh

store: don't read the whole fncache in memory

In large repositories with lot of files, the fncache grows more than 100 MB and
reading that whole thing into memory slows things down. Let's not read the whole
thing into memory.

This patch changes fncache loading code to read 1 MB at once. Loading 1 MB at
once saves ~1 sec on perffncacheload for our internal repository. I tried
various values such as 0.5 MB, 5 MB, 10 MB but best results were produced using
1 MB as the chunksize.

On a narrow clone with fncache around 40 MB, this patch saves ~0.04 seconds on
average on perffncacheload.

To test the code, I have coded an extension in test-fncache.t which set
chunksize to 1 byte, and the test passes with that.

Differential Revision: https://phab.mercurial-scm.org/D5296

#!/bin/bash
# This file is directly inspired by
# https://github.com/pypa/python-manylinux-demo/blob/master/travis/build-wheels.sh
set -e -x

PYTHON_TARGETS=$(ls -d /opt/python/cp27*/bin)

# Create an user for the tests
useradd hgbuilder

# Bypass uid/gid problems
cp -R /src /io && chown -R hgbuilder:hgbuilder /io

# Compile wheels for Python 2.X
for PYBIN in $PYTHON_TARGETS; do
    "${PYBIN}/pip" wheel /io/ -w wheelhouse/
done

# Bundle external shared libraries into the wheels with
# auditwheel (https://github.com/pypa/auditwheel) repair.
# It also fix the ABI tag on the wheel making it pip installable.
for whl in wheelhouse/*.whl; do
    auditwheel repair "$whl" -w /src/wheelhouse/
done

# Install packages and run the tests for all Python versions
cd /io/tests/

for PYBIN in $PYTHON_TARGETS; do
    # Install mercurial wheel as root
    "${PYBIN}/pip" install mercurial --no-index -f /src/wheelhouse
    # But run tests as hgbuilder user (non-root)
    su hgbuilder -c "\"${PYBIN}/python\" /io/tests/run-tests.py --with-hg=\"${PYBIN}/hg\" --blacklist=/io/contrib/packaging/linux-wheel-centos5-blacklist"
done