contrib/packaging/inno/build.py

store: don't read the whole fncache in memory

In large repositories with lot of files, the fncache grows more than 100 MB and
reading that whole thing into memory slows things down. Let's not read the whole
thing into memory.

This patch changes fncache loading code to read 1 MB at once. Loading 1 MB at
once saves ~1 sec on perffncacheload for our internal repository. I tried
various values such as 0.5 MB, 5 MB, 10 MB but best results were produced using
1 MB as the chunksize.

On a narrow clone with fncache around 40 MB, this patch saves ~0.04 seconds on
average on perffncacheload.

To test the code, I have coded an extension in test-fncache.t which set
chunksize to 1 byte, and the test passes with that.

Differential Revision: https://phab.mercurial-scm.org/D5296

#!/usr/bin/env python3
# build.py - Inno installer build script.
#
# Copyright 2019 Gregory Szorc <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

# This script automates the building of the Inno MSI installer for Mercurial.

# no-check-code because Python 3 native.

import argparse
import os
import pathlib
import sys


if __name__ == '__main__':
    parser = argparse.ArgumentParser()

    parser.add_argument('--python',
                        required=True,
                        help='path to python.exe to use')
    parser.add_argument('--iscc',
                        help='path to iscc.exe to use')
    parser.add_argument('--version',
                        help='Mercurial version string to use '
                             '(detected from __version__.py if not defined')

    args = parser.parse_args()

    if args.iscc:
        iscc = pathlib.Path(args.iscc)
    else:
        iscc = (pathlib.Path(os.environ['ProgramFiles(x86)']) / 'Inno Setup 5' /
            'ISCC.exe')

    here = pathlib.Path(os.path.abspath(os.path.dirname(__file__)))
    source_dir = here.parent.parent.parent
    build_dir = source_dir / 'build'

    sys.path.insert(0, str(source_dir / 'contrib' / 'packaging'))

    from hgpackaging.inno import build

    build(source_dir, build_dir, pathlib.Path(args.python), iscc,
          version=args.version)