Makefile

aggressivemergedeltas: enabled the option by default

The option has been around for a while (August 2015) but was never turned on by
default. In-depth testing shows large wins for having that on with no
significant drawbacks.

When enabled, revlog consider delta against both p1 and p2 at the same time when
storing a revision. Selecting a delta against "p2" can produce better deltas and
chain. This raise large benefit for all repositories, especially if they have a
lot of merges.

Comparison of `.hg/store/` size:

    mercurial (6.74% merges):
        before:    54,225,348 bytes
        after:     47,279,959 bytes -13%
    pypy (8.30% merges):
        before:   459,041,759 bytes
        after:    346,090,067 bytes -25%
    netbeans (34.21% merges):
        before: 2,468,041,333 bytes
        after:  1,364,077,645 bytes -45%
    mozilla-central (4.84% merges):
        before: 2,731,799,546 bytes
        after:  2,157,718,019 bytes -21%

Comparison of `00manifest.d` size:

    mercurial (6.74% merges):
        before:    11,682,516 bytes
        after:      6,143,044 bytes -47%
    pypy (8.30% merges):
        before:   156,447,163 bytes
        after:     52,941,780 bytes -66%
    netbeans (34.21% merges):
        before: 1,250,363,851 bytes
        after:    130,088,982 bytes -90%
    mozilla-central (4.84% merges):
        before:   468,202,733 bytes
        after:    215,096,339 bytes -54%

In addition, the better deltas help with the performance of multiple core
operations. However, better chains mean longer chains, which can affect
performance negatively (mostly manifest revision retrieval time). Chains length
is a deeper problem that also affects linear repository too. Overall we think
the benefits of using p2 as a diff target are bigger than the downsizes. In
addition, we are also working on ways to improve the performance impact of chain
length, so theses downsizes get fixed in the future.

Below are interesting items from the full benchmark run:

    bundling 100 revisions from pypy:
        before: 670ms
        after:  480ms -28%
    bundle 10000 revisions from pypy:
        before: 1.38s
        after:  1.10s -54%
    bundle 10000 revisions from pypy:
        before: 16.1s
        after:  7.81s -52%
    bundle 10000 revisions from netbeans:
        before: 19.3s
        after:  15.5s -19%

    unbundle 1000 revisions to pypy:
        before: 641ms
        after:  315ms - 51%

    clone mercurial (http):
        before: 26.0s
        after:  22.6s -23%

    pulling 1000 revisions from pypy (shh):
        before: 2.07s
        after:  1.36s -44%

    pushing 1000 revision through http (pypy repository)
        before: 2.18s
        after:  1.35s -48%

    diff time in mozilla-central:
        before: 1.420s
        after:  0.983s -31%

    status time in mozilla-central:
        before: 1.260s
        after:  0.828s -34%

Impact in other cases seems minimal (within a couple of percent in worse cases)
and can be seen in both direction:

Timing for a simple `hg commit`:

    mozilla-central:
        before: 3.37s
        after:  3.22s -4%
    pypy:
        before:  194ms
        after:   197ms +2%

Timing for status (from tip to parent of tip):
    mercurial:
        before: 52.4ms
        after:  52.4ms (same)
    pypy:
        before: 55.2
        after:  56.9 +3%

Timing for `hg update`

    mozilla-central, across 10 revisions:
        before: 4.82s
        after:  4.59s -5%
    mozilla-central, across 10000 revisions:
        before: 49.1s
        after:  49.9s +2%
    pypy, across 10 revisions:
        before: 213ms
        after:  216ms +1%
    pypy, across 10000 revisions:
        before: 5.31ms
        after:  5.24ms -1%

The negative consequences are related to manifest fetch time:
(timing for the tip revision tested by the benchmark)

    pypy-2018:
        before: 2.60ms
        after:  3.88ms +50%
    mozilla-central-2018:
        before: 565ms
        after:  652ms +15% (~+100ms)
    netbeans-2018:
        before: 101ms
        after:  250ms +48% (~+150ms)

This shows up as a fixed overhead on some command we benchmarked:

    no-op push of mozilla-central:
        before:  945ms
        after:  1040ms +10% (~+100ms)
    pushing 10 changeset in netbeabs over ssh:
        before:  557ms
        after:   712ms +28% (+155ms)
    pushing 100 changeset in netbeabs over ssh:
        before: 592ms
        after:  771ms +30% (+179ms)

# If you want to change PREFIX, do not just edit it below. The changed
# value wont get passed on to recursive make calls. You should instead
# override the variable on the command like:
#
# % make PREFIX=/opt/ install

export PREFIX=/usr/local
PYTHON=python
$(eval HGROOT := $(shell pwd))
HGPYTHONS ?= $(HGROOT)/build/pythons
PURE=
PYFILES:=$(shell find mercurial hgext doc -name '*.py')
DOCFILES=mercurial/help/*.txt
export LANGUAGE=C
export LC_ALL=C
TESTFLAGS ?= $(shell echo $$HGTESTFLAGS)
OSXVERSIONFLAGS ?= $(shell echo $$OSXVERSIONFLAGS)

# Set this to e.g. "mingw32" to use a non-default compiler.
COMPILER=

COMPILERFLAG_tmp_ =
COMPILERFLAG_tmp_${COMPILER} ?= -c $(COMPILER)
COMPILERFLAG=${COMPILERFLAG_tmp_${COMPILER}}

help:
	@echo 'Commonly used make targets:'
	@echo '  all          - build program and documentation'
	@echo '  install      - install program and man pages to $$PREFIX ($(PREFIX))'
	@echo '  install-home - install with setup.py install --home=$$HOME ($(HOME))'
	@echo '  local        - build for inplace usage'
	@echo '  tests        - run all tests in the automatic test suite'
	@echo '  test-foo     - run only specified tests (e.g. test-merge1.t)'
	@echo '  dist         - run all tests and create a source tarball in dist/'
	@echo '  clean        - remove files created by other targets'
	@echo '                 (except installed files or dist source tarball)'
	@echo '  update-pot   - update i18n/hg.pot'
	@echo
	@echo 'Example for a system-wide installation under /usr/local:'
	@echo '  make all && su -c "make install" && hg version'
	@echo
	@echo 'Example for a local installation (usable in this directory):'
	@echo '  make local && ./hg version'

all: build doc

local:
	$(PYTHON) setup.py $(PURE) \
	  build_py -c -d . \
	  build_ext $(COMPILERFLAG) -i \
	  build_hgexe $(COMPILERFLAG) -i \
	  build_mo
	env HGRCPATH= $(PYTHON) hg version

build:
	$(PYTHON) setup.py $(PURE) build $(COMPILERFLAG)

wheel:
	FORCE_SETUPTOOLS=1 $(PYTHON) setup.py $(PURE) bdist_wheel $(COMPILERFLAG)

doc:
	$(MAKE) -C doc

cleanbutpackages:
	-$(PYTHON) setup.py clean --all # ignore errors from this command
	find contrib doc hgext hgext3rd i18n mercurial tests hgdemandimport \
		\( -name '*.py[cdo]' -o -name '*.so' \) -exec rm -f '{}' ';'
	rm -f MANIFEST MANIFEST.in hgext/__index__.py tests/*.err
	rm -f mercurial/__modulepolicy__.py
	if test -d .hg; then rm -f mercurial/__version__.py; fi
	rm -rf build mercurial/locale
	$(MAKE) -C doc clean
	$(MAKE) -C contrib/chg distclean

clean: cleanbutpackages
	rm -rf packages

install: install-bin install-doc

install-bin: build
	$(PYTHON) setup.py $(PURE) install --root="$(DESTDIR)/" --prefix="$(PREFIX)" --force

install-doc: doc
	cd doc && $(MAKE) $(MFLAGS) install

install-home: install-home-bin install-home-doc

install-home-bin: build
	$(PYTHON) setup.py $(PURE) install --home="$(HOME)" --prefix="" --force

install-home-doc: doc
	cd doc && $(MAKE) $(MFLAGS) PREFIX="$(HOME)" install

MANIFEST-doc:
	$(MAKE) -C doc MANIFEST

MANIFEST.in: MANIFEST-doc
	hg manifest | sed -e 's/^/include /' > MANIFEST.in
	echo include mercurial/__version__.py >> MANIFEST.in
	sed -e 's/^/include /' < doc/MANIFEST >> MANIFEST.in

dist:	tests dist-notests

dist-notests:	doc MANIFEST.in
	TAR_OPTIONS="--owner=root --group=root --mode=u+w,go-w,a+rX-s" $(PYTHON) setup.py -q sdist

check: tests

tests:
	cd tests && $(PYTHON) run-tests.py $(TESTFLAGS)

test-%:
	cd tests && $(PYTHON) run-tests.py $(TESTFLAGS) $@

testpy-%:
	@echo Looking for Python $* in $(HGPYTHONS)
	[ -e $(HGPYTHONS)/$*/bin/python ] || ( \
	cd $$(mktemp --directory --tmpdir) && \
        $(MAKE) -f $(HGROOT)/contrib/Makefile.python PYTHONVER=$* PREFIX=$(HGPYTHONS)/$* python )
	cd tests && $(HGPYTHONS)/$*/bin/python run-tests.py $(TESTFLAGS)

check-code:
	hg manifest | xargs python contrib/check-code.py

format-c:
	clang-format --style file -i \
	  `hg files 'set:(**.c or **.cc or **.h) and not "listfile:contrib/clang-format-ignorelist"'`

update-pot: i18n/hg.pot

i18n/hg.pot: $(PYFILES) $(DOCFILES) i18n/posplit i18n/hggettext
	$(PYTHON) i18n/hggettext mercurial/commands.py \
	  hgext/*.py hgext/*/__init__.py \
	  mercurial/fileset.py mercurial/revset.py \
	  mercurial/templatefilters.py \
	  mercurial/templatefuncs.py \
	  mercurial/templatekw.py \
	  mercurial/filemerge.py \
	  mercurial/hgweb/webcommands.py \
	  mercurial/util.py \
	  $(DOCFILES) > i18n/hg.pot.tmp
        # All strings marked for translation in Mercurial contain
        # ASCII characters only. But some files contain string
        # literals like this '\037\213'. xgettext thinks it has to
        # parse them even though they are not marked for translation.
        # Extracting with an explicit encoding of ISO-8859-1 will make
        # xgettext "parse" and ignore them.
	echo $(PYFILES) | xargs \
	  xgettext --package-name "Mercurial" \
	  --msgid-bugs-address "<[e-mail]>" \
	  --copyright-holder "Matt Mackall <[e-mail]> and others" \
	  --from-code ISO-8859-1 --join --sort-by-file --add-comments=i18n: \
	  -d hg -p i18n -o hg.pot.tmp
	$(PYTHON) i18n/posplit i18n/hg.pot.tmp
        # The target file is not created before the last step. So it never is in
        # an intermediate state.
	mv -f i18n/hg.pot.tmp i18n/hg.pot

%.po: i18n/hg.pot
        # work on a temporary copy for never having a half completed target
	cp $@ $@.tmp
	msgmerge --no-location --update $@.tmp $^
	mv -f $@.tmp $@

# Packaging targets

packaging_targets := \
  centos5 \
  centos6 \
  centos7 \
  deb \
  docker-centos5 \
  docker-centos6 \
  docker-centos7 \
  docker-debian-jessie \
  docker-debian-stretch \
  docker-fedora20 \
  docker-fedora21 \
  docker-fedora28 \
  docker-ubuntu-trusty \
  docker-ubuntu-trusty-ppa \
  docker-ubuntu-xenial \
  docker-ubuntu-xenial-ppa \
  docker-ubuntu-artful \
  docker-ubuntu-artful-ppa \
  docker-ubuntu-bionic \
  docker-ubuntu-bionic-ppa \
  fedora20 \
  fedora21 \
  fedora28 \
  linux-wheels \
  linux-wheels-x86_64 \
  linux-wheels-i686 \
  ppa

# Forward packaging targets for convenience.
$(packaging_targets):
	$(MAKE) -C contrib/packaging $@

osx:
	rm -rf build/mercurial
	/usr/bin/python2.7 setup.py install --optimize=1 \
	  --root=build/mercurial/ --prefix=/usr/local/ \
	  --install-lib=/Library/Python/2.7/site-packages/
	make -C doc all install DESTDIR="$(PWD)/build/mercurial/"
        # Place a bogon .DS_Store file in the target dir so we can be
        # sure it doesn't get included in the final package.
	touch build/mercurial/.DS_Store
        # install zsh completions - this location appears to be
        # searched by default as of macOS Sierra.
	install -d build/mercurial/usr/local/share/zsh/site-functions/
	install -m 0644 contrib/zsh_completion build/mercurial/usr/local/share/zsh/site-functions/_hg
        # install bash completions - there doesn't appear to be a
        # place that's searched by default for bash, so we'll follow
        # the lead of Apple's git install and just put it in a
        # location of our own.
	install -d build/mercurial/usr/local/hg/contrib/
	install -m 0644 contrib/bash_completion build/mercurial/usr/local/hg/contrib/hg-completion.bash
	make -C contrib/chg \
	  HGPATH=/usr/local/bin/hg \
	  PYTHON=/usr/bin/python2.7 \
	  HGEXTDIR=/Library/Python/2.7/site-packages/hgext \
	  DESTDIR=../../build/mercurial \
	  PREFIX=/usr/local \
	  clean install
	mkdir -p $${OUTPUTDIR:-dist}
	HGVER=$$(python contrib/genosxversion.py $(OSXVERSIONFLAGS) build/mercurial/Library/Python/2.7/site-packages/mercurial/__version__.py) && \
	OSXVER=$$(sw_vers -productVersion | cut -d. -f1,2) && \
	pkgbuild --filter \\.DS_Store --root build/mercurial/ \
	  --identifier org.mercurial-scm.mercurial \
	  --version "$${HGVER}" \
	  build/mercurial.pkg && \
	productbuild --distribution contrib/packaging/macosx/distribution.xml \
	  --package-path build/ \
	  --version "$${HGVER}" \
	  --resources contrib/packaging/macosx/ \
	  "$${OUTPUTDIR:-dist/}"/Mercurial-"$${HGVER}"-macosx"$${OSXVER}".pkg

.PHONY: help all local build doc cleanbutpackages clean install install-bin \
	install-doc install-home install-home-bin install-home-doc \
	dist dist-notests check tests check-code format-c update-pot \
	$(packaging_targets) \
	osx