Mercurial > hg
view tests/test-largefiles-small-disk.t @ 31467:08ecec297521
bdiff: use Python memory allocator in fixws
Python has its own memory allocation APIs. For allocations
<= 512 bytes, it allocates memory from arenas. This means that
average small allocations don't call the system allocator, which
makes them faster. Also, arena allocations cut down on memory
fragmentation, which can matter for performance in long-running
processes.
Another advantage of using the Python memory allocator is that
allocations are tracked by Python. This is a bigger deal in
Python 3, as modern versions of Python have some decent built-in
tools for examining memory usage, leaks, etc.
This patch converts a trivial malloc() + free() in the bdiff code
to use the Python allocator APIs. Since the object being
operated on is a line, chances are it will use an arena. So,
this could have a net positive impact on performance (although
I didn't measure it).
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 09 Mar 2017 11:54:25 -0800 |
parents | 1ef37b16b8e8 |
children | 991134261057 |
line wrap: on
line source
Test how largefiles abort in case the disk runs full $ cat > criple.py <<EOF > import os, errno, shutil > from mercurial import util > # > # this makes the original largefiles code abort: > _origcopyfileobj = shutil.copyfileobj > def copyfileobj(fsrc, fdst, length=16*1024): > # allow journal files (used by transaction) to be written > if 'journal.' in fdst.name: > return _origcopyfileobj(fsrc, fdst, length) > fdst.write(fsrc.read(4)) > raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC)) > shutil.copyfileobj = copyfileobj > # > # this makes the rewritten code abort: > def filechunkiter(f, size=131072, limit=None): > yield f.read(4) > raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC)) > util.filechunkiter = filechunkiter > # > def oslink(src, dest): > raise OSError("no hardlinks, try copying instead") > util.oslink = oslink > EOF $ echo "[extensions]" >> $HGRCPATH $ echo "largefiles =" >> $HGRCPATH $ hg init alice $ cd alice $ echo "this is a very big file" > big $ hg add --large big $ hg commit --config extensions.criple=$TESTTMP/criple.py -m big abort: No space left on device [255] The largefile is not created in .hg/largefiles: $ ls .hg/largefiles dirstate The user cache is not even created: >>> import os; os.path.exists("$HOME/.cache/largefiles/") False Make the commit with space on the device: $ hg commit -m big Now make a clone with a full disk, and make sure lfutil.link function makes copies instead of hardlinks: $ cd .. $ hg --config extensions.criple=$TESTTMP/criple.py clone --pull alice bob requesting all changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files updating to branch default getting changed largefiles abort: No space left on device [255] The largefile is not created in .hg/largefiles: $ ls bob/.hg/largefiles dirstate