Mercurial > hg
view contrib/Makefile.python @ 42469:4a3abb33380a
deltas: set estimated compression upper bound to "3x" instead of "10x"
In pratice, we very rarely observer compression better than "3x" on manifest
deltas. Having a more aggressive estimate significantly helps our pathological
use case on a private repository. Here are a comparison of timings using
different upper bound.
Estimated compression | ø | ×10 | ×5 | ×3 |
timing | 14.11 | 2.61 | 1.96 | 1.53 |
We also tested the impact of this series on an array of public repositories.
This shown no impact in either size nor timing.
Full data set below for those interested.
Size
----
Regarding size, not significant impact have been noticed on neither public nor
private repositories. Here are the number we gathered on public repositories:
zlib/upperbound | no | 10x | 5x | 3x
mercurial | 5 875 730 | 5 875 730 | 5 875 730 | 5 875 730
pypy | 27 782 913 | 27 782 913 | 27 782 913 | 27 782 913
netbeans | 159 161 207 | 159 161 207 | 159 161 207 | 159 959 879 (+0.5%)
mozilla-central | 323 841 642 | 323 841 642 | 323 841 642 | 319 867 519 (-2.5%)
mozilla-try | 746 649 123 | 746 649 123 | 746 649 123 | 741 155 568 (-0.7%)
private-repo | 1 485 287 294 | 1 485 287 294 | 1 485 287 294 | 1 409 248 382 (-5.1%)
zstd/upperbound | no | 10x | 5x | 3x
mercurial | 5 895 206 | 5 895 206 | 5 895 206 | 5 895 206
pypy | 28 689 230 | 28 689 230 | 28 689 230 | 28 689 230
netbeans | 157 636 387 | 157 636 387 | 157 636 387 | 159 692 678 (+1.3%)
mozilla-central | 317 650 281 | 317 650 281 | 317 650 281 | 319 613 603 (+0.6%)
mozilla-try | 737 555 275 | 737 555 275 | 737 555 275 | 738 079 473 (+0.1%)
private-repo | 1 352 362 982 | 1 352 362 982 | 1 346 961 880 | 1 361 327 384 (+0.7%)
Speed
------
Timing gathered using `hg perfrevlogwrite -m`. Value are in seconds.
mercurial
zlib | no | 10x | 5x | 3x |
total | 65.551783 | 65.388887 | 65.260658 | 65.321199 |
max | 0.034544 | 0.034571 | 0.034659 | 0.034521 |
99.99% | 0.034544 | 0.034571 | 0.034659 | 0.034521 |
zstd | no | 10x | 5x | 3x |
total | 49.118449 | 49.054062 | 48.753588 | 48.740230 |
max | 0.009338 | 0.009239 | 0.009202 | 0.009178 |
99.99% | 0.007618 | 0.007639 | 0.007626 | 0.007621 |
pypy
zlib | no | 10x | 5x | 3x |
total | 560.865984 | 558.983817 | 559.083815 | 559.349152 |
max | 0.219614 | 0.215922 | 0.218112 | 0.218107 |
99.99% | 0.219614 | 0.215922 | 0.218112 | 0.218107 |
zstd | no | 10x | 5x | 3x |
total | 349.393280 | 347.395819 | 347.185407 | 345.643985 |
max | 0.084143 | 0.083536 | 0.081834 | 0.082178 |
99.99% | 0.039445 | 0.039639 | 0.039612 | 0.039175 |
netbeans
zlib | no | 10x | 5x | 3x |
total | 33103.327727 | 33314.932260 | 33211.745233 | 33345.891778 |
max | 2.666852 | 2.672059 | 2.662453 | 2.662936 |
99.99% | 2.058772 | 2.070429 | 2.069569 | 2.064653 |
zstd | no | 10x | 5x | 3x |
total | 20112.102708 | 20095.879719 | 20083.390300 | 20123.221859 |
max | 2.063482 | 2.062851 | 2.065229 | 2.060147 |
99.99% | 1.146647 | 1.143794 | 1.142933 | 1.146529 |
mozilla
zlib | no | 10x | 5x | 3x |
total | 41374.102138 | 41418.816773 | 41381.956370 | 41334.280732 |
max | 3.383474 | 3.387400 | 3.405711 | 3.387316 |
99.99% | 1.006755 | 1.005954 | 1.007700 | 1.007373 |
zstd | no | 10x | 5x | 3x |
total | 24689.691520 | 24643.939662 | 24664.630027 | 24664.512714 |
max | 1.460822 | 1.449640 | 1.439747 | 1.465304 |
99.99% | 0.527111 | 0.527377 | 0.527807 | 0.527226 |
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Fri, 26 Apr 2019 00:48:12 +0200 |
parents | ae68418cc3a1 |
children |
line wrap: on
line source
PYTHONVER=2.7.16 PYTHONNAME=python- PREFIX=$(HOME)/bin/prefix-$(PYTHONNAME)$(PYTHONVER) SYMLINKDIR=$(HOME)/bin help: @echo @echo 'Make a custom installation of a Python version' @echo @echo 'Common make parameters:' @echo ' PYTHONVER=... [$(PYTHONVER)]' @echo ' PREFIX=... [$(PREFIX)]' @echo ' SYMLINKDIR=... [$(SYMLINKDIR) creating $(PYTHONNAME)$(PYTHONVER)]' @echo @echo 'Common make targets:' @echo ' python - install Python $$PYTHONVER in $$PREFIX' @echo ' symlink - create a $$SYMLINKDIR/$(PYTHONNAME)$$PYTHONVER symlink' @echo @echo 'Example: create a temporary Python installation:' @echo ' $$ make -f Makefile.python python PYTHONVER=${PYTHONVER} PREFIX=/tmp/p27' @echo ' $$ /tmp/p27/bin/python -V' @echo ' Python 2.7' @echo @echo 'Some external libraries are required for building Python: zlib bzip2 openssl.' @echo 'Make sure their development packages are installed systemwide.' # fedora: yum install zlib-devel bzip2-devel openssl-devel # debian: apt-get install zlib1g-dev libbz2-dev libssl-dev @echo @echo 'To build a nice collection of interesting Python versions:' @echo ' $$ for v in 2.{6{,.1,.2,.9},7{,.8,.10}}; do' @echo ' make -f Makefile.python symlink PYTHONVER=$$v || break; done' @echo 'To run a Mercurial test on all these Python versions:' @echo ' $$ for py in `cd ~/bin && ls $(PYTHONNAME)2.*`; do' @echo ' echo $$py; $$py run-tests.py test-http.t; echo; done' @echo export LANGUAGE=C export LC_ALL=C python: $(PREFIX)/bin/python docutils printf 'import sys, zlib, bz2, docutils, ssl' | $(PREFIX)/bin/python PYTHON_SRCDIR=Python-$(PYTHONVER) PYTHON_SRCFILE=$(PYTHON_SRCDIR).tgz $(PREFIX)/bin/python: [ -f $(PYTHON_SRCFILE) ] || wget http://www.python.org/ftp/python/$(PYTHONVER)/$(PYTHON_SRCFILE) || curl -OL http://www.python.org/ftp/python/$(PYTHONVER)/$(PYTHON_SRCFILE) || [ -f $(PYTHON_SRCFILE) ] rm -rf $(PYTHON_SRCDIR) tar xf $(PYTHON_SRCFILE) # Debian/Ubuntu disables SSLv2,3 the hard way, disable it on old Pythons too -sed -i 's,self.*SSLv[23]_method(),0;//\0,g' $(PYTHON_SRCDIR)/Modules/_ssl.c # Find multiarch system libraries on Ubuntu and disable fortify error when setting argv LDFLAGS="-L/usr/lib/`dpkg-architecture -qDEB_HOST_MULTIARCH`"; \ BASECFLAGS=-U_FORTIFY_SOURCE; \ export LDFLAGS BASECFLAGS; \ cd $(PYTHON_SRCDIR) && ./configure --prefix=$(PREFIX) && make all SVNVERSION=pwd && make install printf 'import sys, zlib, bz2, ssl' | $(PREFIX)/bin/python rm -rf $(PYTHON_SRCDIR) DOCUTILSVER=0.12 DOCUTILS_SRCDIR=docutils-$(DOCUTILSVER) DOCUTILS_SRCFILE=$(DOCUTILS_SRCDIR).tar.gz docutils: $(PREFIX)/bin/python @$(PREFIX)/bin/python -c 'import docutils' || ( set -ex; \ [ -f $(DOCUTILS_SRCFILE) ] || wget http://downloads.sourceforge.net/project/docutils/docutils/$(DOCUTILSVER)/$(DOCUTILS_SRCFILE) || [ -f $(DOCUTILS_SRCFILE) ]; \ rm -rf $(DOCUTILS_SRCDIR); \ tar xf $(DOCUTILS_SRCFILE); \ cd $(DOCUTILS_SRCDIR) && $(PREFIX)/bin/python setup.py install --prefix=$(PREFIX); \ $(PREFIX)/bin/python -c 'import docutils'; \ rm -rf $(DOCUTILS_SRCDIR); ) symlink: python $(SYMLINKDIR) ln -sf $(PREFIX)/bin/python $(SYMLINKDIR)/$(PYTHONNAME)$(PYTHONVER) .PHONY: help python docutils symlink