Mercurial > hg
view tests/test-blackbox.t @ 43198:c16fe77e340a
pathcopies: give up any optimization based on `introrev`
Between 8a0136f69027 and d98fb3f42f33, we sped up the search for the
introduction revision during path copies. However, further checking show that
finding the introduction revision is still expensive and that we are better off
without it. So we simply drop it and only rely on the linkrev optimisation.
I ran `perfpathcopies` on 6989 pair of revision in the pypy
repository (`hg perfhelper-pathcopies`. The result is massively in favor of
dropping this condition. The result of the copy tracing are unchanged.
Attempt to use a smaller changes preserving linkrev usage were unsuccessful, it
can return wrong result. The following changesets broke test-mv-cp-st-diff.t
- if not f.isintroducedafter(limit):
+ if limit >= 0 and f.linkrev() < limit:
return None
Here are various numbers (before this changeset/after this changesets)
source destination before after saved-time ratio
worth cases e66f24650daf 695dfb0f493b 1.062843 1.246369 -0.183526 1.172675
c979853a3b6a 8d60fe293e79 1.036985 1.196414 -0.159429 1.153743
22349fa2fc33 fbb1c9fd86c0 0.879926 1.038682 -0.158756 1.180420
682b98f3e672 a4878080a536 0.909952 1.063801 -0.153849 1.169074
5adabc9b9848 920958a93997 0.993622 1.147452 -0.153830 1.154817
worse 1% dbfbfcf077e9 aea8f2fd3593 1.016595 1.082999 -0.066404 1.065320
worse 5% c95f1ced15f2 7d29d5e39734 0.453694 0.471156 -0.017462 1.038488
worse 10% 3e144ed1d5b7 2aef0e942480 0.035140 0.037535 -0.002395 1.068156
worse 25% 321fc60db035 801748ba582a 0.009267 0.009325 -0.000058 1.006259
median 2088ce763fc2 e6991321d78b 0.000665 0.000651 0.000014 0.978947
best 25% 915631a97de6 385b31354be6 0.040743 0.040363 0.000380 0.990673
best 10% ad495c36a765 19c10384d3e7 0.431658 0.411490 0.020168 0.953278
best 5% d13ae7d283ae 813c99f810ac 1.141404 1.075346 0.066058 0.942126
best 1% 81593cb4a496 99ae11866969 1.833297 0.063823 1.769474 0.034813
best cases c3b14617fbd7 743a0fcaa4eb 1101.811740 2.735970 1099.075770 0.002483
c3b14617fbd7 9ba6ab77fd29 1116.753953 2.800729 1113.953224 0.002508
058b99d6e81f 57e249b7a3ea 1246.128485 3.042762 1243.085723 0.002442
9a8c361aab49 0354a250d371 1253.111894 3.085796 1250.026098 0.002463
442dbbc53c68 3ec1002a818c 1261.786294 3.138607 1258.647687 0.002487
As one can see, the average case is not really impacted. However, the worth case
we get after this changeset are much better than the one we had before it. We
have 30 pairs where improvements are above 10 minutes.
This reflect in the combined time for all pairs
before: 26256s
after: 1300s (-95%)
If we remove these pathological 30 cases, we still see a significant improvements:
before: 1631s
after: 1245s (-24%)
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Thu, 10 Oct 2019 03:49:33 +0200 |
parents | 775224e26d74 |
children | c6b4c348360e |
line wrap: on
line source
setup $ cat > myextension.py <<EOF > from mercurial import error, registrar > cmdtable = {} > command = registrar.command(cmdtable) > @command(b'crash', [], b'hg crash') > def crash(ui, *args, **kwargs): > raise Exception("oops") > @command(b'abortcmd', [], b'hg abortcmd') > def abort(ui, *args, **kwargs): > raise error.Abort(b"oops") > EOF $ abspath=`pwd`/myextension.py $ cat >> $HGRCPATH <<EOF > [extensions] > blackbox= > mock=$TESTDIR/mockblackbox.py > mq= > myextension=$TESTTMP/myextension.py > [alias] > confuse = log --limit 3 > so-confusing = confuse --style compact > EOF $ hg init blackboxtest $ cd blackboxtest command, exit codes, and duration $ echo a > a $ hg add a $ hg blackbox --config blackbox.dirty=True 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> init blackboxtest exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> add a 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> add a exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000+ (5000)> blackbox --config *blackbox.dirty=True* (glob) failure exit code $ rm ./.hg/blackbox.log $ hg add non-existent non-existent: $ENOENT$ [1] $ hg blackbox 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> add non-existent 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> add non-existent exited 1 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox abort exit code $ rm ./.hg/blackbox.log $ hg abortcmd 2> /dev/null [255] $ hg blackbox -l 2 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> abortcmd exited 255 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox -l 2 unhandled exception $ rm ./.hg/blackbox.log $ hg crash 2> /dev/null [1] $ hg blackbox -l 2 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> crash exited 1 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox -l 2 alias expansion is logged $ rm ./.hg/blackbox.log $ hg confuse $ hg blackbox 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> confuse 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> alias 'confuse' expands to 'log --limit 3' 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> confuse exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox recursive aliases work correctly $ rm ./.hg/blackbox.log $ hg so-confusing $ hg blackbox 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> so-confusing 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> alias 'so-confusing' expands to 'confuse --style compact' 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> alias 'confuse' expands to 'log --limit 3' 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> so-confusing exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox custom date format $ rm ./.hg/blackbox.log $ hg --config blackbox.date-format='%Y-%m-%d @ %H:%M:%S' \ > --config devel.default-date='1334347993 0' --traceback status A a $ hg blackbox 2012-04-13 @ 20:13:13 bob @0000000000000000000000000000000000000000 (5000)> --config *blackbox.date-format=%Y-%m-%d @ %H:%M:%S* --config *devel.default-date=1334347993 0* --traceback status (glob) 2012-04-13 @ 20:13:13 bob @0000000000000000000000000000000000000000 (5000)> --config *blackbox.date-format=%Y-%m-%d @ %H:%M:%S* --config *devel.default-date=1334347993 0* --traceback status exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox incoming change tracking create two heads to verify that we only see one change in the log later $ hg commit -ma $ hg up null 0 files updated, 0 files merged, 1 files removed, 0 files unresolved $ echo b > b $ hg commit -Amb adding b created new head clone, commit, pull $ hg clone . ../blackboxtest2 updating to branch default 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ echo c > c $ hg commit -Amc adding c $ cd ../blackboxtest2 $ hg pull pulling from $TESTTMP/blackboxtest searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files new changesets d02f48003e62 (run 'hg update' to get a working copy) $ hg blackbox -l 6 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> pull 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (served) in * seconds (glob) 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (served) with 1 labels and 2 nodes 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> 1 incoming changes - new heads: d02f48003e62 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> pull exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6 we must not cause a failure if we cannot write to the log $ hg rollback repository tip rolled back to revision 1 (undo pull) $ mv .hg/blackbox.log .hg/blackbox.log- $ mkdir .hg/blackbox.log $ hg --debug incoming warning: cannot write to blackbox.log: * (glob) comparing with $TESTTMP/blackboxtest query 1; heads searching for changes all local changesets known remotely changeset: 2:d02f48003e62c24e2659d97d30f2a83abe5d5d51 tag: tip phase: draft parent: 1:6563da9dcf87b1949716e38ff3e3dfaa3198eb06 parent: -1:0000000000000000000000000000000000000000 manifest: 2:ab9d46b053ebf45b7996f2922b9893ff4b63d892 user: test date: Thu Jan 01 00:00:00 1970 +0000 files+: c extra: branch=default description: c $ hg pull pulling from $TESTTMP/blackboxtest searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files new changesets d02f48003e62 (run 'hg update' to get a working copy) a failure reading from the log is fatal $ hg blackbox -l 3 abort: *$TESTTMP/blackboxtest2/.hg/blackbox.log* (glob) [255] $ rmdir .hg/blackbox.log $ mv .hg/blackbox.log- .hg/blackbox.log backup bundles get logged $ touch d $ hg commit -Amd adding d created new head $ hg strip tip 0 files updated, 0 files merged, 1 files removed, 0 files unresolved saved backup bundle to $TESTTMP/blackboxtest2/.hg/strip-backup/*-backup.hg (glob) $ hg blackbox -l 6 1970/01/01 00:00:00 bob @73f6ee326b27d820b0472f1a825e3a50f3dc489b (5000)> strip tip 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> saved backup bundle to $TESTTMP/blackboxtest2/.hg/strip-backup/73f6ee326b27-7612e004-backup.hg 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (base) in * seconds (glob) 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (base) with 1 labels and 2 nodes 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> strip tip exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6 extension and python hooks - use the eol extension for a pythonhook $ echo '[extensions]' >> .hg/hgrc $ echo 'eol=' >> .hg/hgrc $ echo '[hooks]' >> .hg/hgrc $ echo 'update = echo hooked' >> .hg/hgrc $ hg update The fsmonitor extension is incompatible with the eol extension and has been disabled. (fsmonitor !) hooked 1 files updated, 0 files merged, 0 files removed, 0 files unresolved updated to "d02f48003e62: c" 1 other heads for branch "default" $ cat >> .hg/hgrc <<EOF > [extensions] > # disable eol, because it is not needed for subsequent tests > # (in addition, keeping it requires extra care for fsmonitor) > eol=! > EOF $ hg blackbox -l 5 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> update (no-chg !) 1970/01/01 00:00:00 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> pythonhook-preupdate: hgext.eol.preupdate finished in * seconds (glob) 1970/01/01 00:00:00 bob @d02f48003e62c24e2659d97d30f2a83abe5d5d51 (5000)> exthook-update: echo hooked finished in * seconds (glob) 1970/01/01 00:00:00 bob @d02f48003e62c24e2659d97d30f2a83abe5d5d51 (5000)> update exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @d02f48003e62c24e2659d97d30f2a83abe5d5d51 (5000)> serve --cmdserver chgunix --address $TESTTMP.chgsock/server.* --daemon-postexec 'chdir:/' (glob) (chg !) 1970/01/01 00:00:00 bob @d02f48003e62c24e2659d97d30f2a83abe5d5d51 (5000)> blackbox -l 5 log rotation $ echo '[blackbox]' >> .hg/hgrc $ echo 'maxsize = 20 b' >> .hg/hgrc $ echo 'maxfiles = 3' >> .hg/hgrc $ hg status $ hg status $ hg status $ hg tip -q 2:d02f48003e62 $ ls .hg/blackbox.log* .hg/blackbox.log .hg/blackbox.log.1 .hg/blackbox.log.2 $ cd .. $ hg init blackboxtest3 $ cd blackboxtest3 $ hg blackbox 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> init blackboxtest3 exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> blackbox $ mv .hg/blackbox.log .hg/blackbox.log- $ mkdir .hg/blackbox.log $ sed -e 's/\(.*test1.*\)/#\1/; s#\(.*commit2.*\)#os.rmdir(".hg/blackbox.log")\ > os.rename(".hg/blackbox.log-", ".hg/blackbox.log")\ > \1#' $TESTDIR/test-dispatch.py > ../test-dispatch.py $ "$PYTHON" $TESTDIR/blackbox-readonly-dispatch.py running: --debug add foo warning: cannot write to blackbox.log: Is a directory (no-windows !) warning: cannot write to blackbox.log: $TESTTMP/blackboxtest3/.hg/blackbox.log: Access is denied (windows !) adding foo result: 0 running: --debug commit -m commit1 -d 2000-01-01 foo warning: cannot write to blackbox.log: Is a directory (no-windows !) warning: cannot write to blackbox.log: $TESTTMP/blackboxtest3/.hg/blackbox.log: Access is denied (windows !) committing files: foo committing manifest committing changelog updating the branch cache committed changeset 0:0e46349438790c460c5c9f7546bfcd39b267bbd2 result: 0 running: --debug commit -m commit2 -d 2000-01-02 foo committing files: foo committing manifest committing changelog updating the branch cache committed changeset 1:45589e459b2edfbf3dbde7e01f611d2c1e7453d7 result: 0 running: --debug log -r 0 changeset: 0:0e46349438790c460c5c9f7546bfcd39b267bbd2 phase: draft parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000 manifest: 0:9091aa5df980aea60860a2e39c95182e68d1ddec user: test date: Sat Jan 01 00:00:00 2000 +0000 files+: foo extra: branch=default description: commit1 result: 0 running: --debug log -r tip changeset: 1:45589e459b2edfbf3dbde7e01f611d2c1e7453d7 tag: tip phase: draft parent: 0:0e46349438790c460c5c9f7546bfcd39b267bbd2 parent: -1:0000000000000000000000000000000000000000 manifest: 1:895aa9b7886f89dd017a6d62524e1f9180b04df9 user: test date: Sun Jan 02 00:00:00 2000 +0000 files: foo extra: branch=default description: commit2 result: 0 $ hg blackbox 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> updating the branch cache 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> updated branch cache (served) in * seconds (glob) 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> wrote branch cache (served) with 1 labels and 1 nodes 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> --debug commit -m commit2 -d 2000-01-02 foo exited 0 after *.?? seconds (glob) 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> --debug log -r 0 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> writing .hg/cache/tags2-visible with 0 tags 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> --debug log -r 0 exited 0 after *.?? seconds (glob) 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> --debug log -r tip 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> --debug log -r tip exited 0 after *.?? seconds (glob) 1970/01/01 00:00:00 bob @45589e459b2edfbf3dbde7e01f611d2c1e7453d7 (5000)> blackbox Test log recursion from dirty status check $ cat > ../r.py <<EOF > from mercurial import context, error, extensions > x=[False] > def status(orig, *args, **opts): > args[0].repo().ui.log(b"broken", b"recursion?") > return orig(*args, **opts) > def reposetup(ui, repo): > extensions.wrapfunction(context.basectx, 'status', status) > EOF $ hg id --config extensions.x=../r.py --config blackbox.dirty=True 45589e459b2e tip cleanup $ cd .. Test missing log directory, which shouldn't be created automatically $ cat <<'EOF' > closeremove.py > def reposetup(ui, repo): > class rmrepo(repo.__class__): > def close(self): > super(rmrepo, self).close() > self.ui.debug(b'removing %s\n' % self.vfs.base) > self.vfs.rmtree() > repo.__class__ = rmrepo > EOF $ hg init gone $ cd gone $ cat <<'EOF' > .hg/hgrc > [extensions] > closeremove = ../closeremove.py > EOF $ hg log --debug removing $TESTTMP/gone/.hg warning: cannot write to blackbox.log: $ENOENT$ (no-windows !) warning: cannot write to blackbox.log: $TESTTMP/gone/.hg/blackbox.log: $ENOTDIR$ (windows !) $ cd .. blackbox should disable itself if track is empty $ hg --config blackbox.track= init nothing_tracked $ cd nothing_tracked $ cat >> .hg/hgrc << EOF > [blackbox] > track = > EOF $ hg blackbox $ cd $TESTTMP a '*' entry in blackbox.track is interpreted as log everything $ hg --config blackbox.track='*' \ > --config blackbox.logsource=True \ > init track_star $ cd track_star $ cat >> .hg/hgrc << EOF > [blackbox] > logsource = True > track = * > EOF (only look for entries with specific logged sources, otherwise this test is pretty brittle) $ hg blackbox | egrep '\[command(finish)?\]' 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000) [commandfinish]> --config *blackbox.track=* --config *blackbox.logsource=True* init track_star exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000) [command]> blackbox $ cd $TESTTMP #if chg when using chg, blackbox.log should get rotated correctly $ cat > $TESTTMP/noop.py << EOF > from __future__ import absolute_import > import time > from mercurial import registrar, scmutil > cmdtable = {} > command = registrar.command(cmdtable) > @command('noop') > def noop(ui, repo): > pass > EOF $ hg init blackbox-chg $ cd blackbox-chg $ cat > .hg/hgrc << EOF > [blackbox] > maxsize = 500B > [extensions] > # extension change forces chg to restart > noop=$TESTTMP/noop.py > EOF $ "$PYTHON" -c 'print("a" * 400)' > .hg/blackbox.log $ chg noop $ chg noop $ chg noop $ chg noop $ chg noop $ cat > showsize.py << 'EOF' > import os > import sys > limit = 500 > for p in sys.argv[1:]: > size = os.stat(p).st_size > if size >= limit: > desc = '>=' > else: > desc = '<' > print('%s: %s %d' % (p, desc, limit)) > EOF $ "$PYTHON" showsize.py .hg/blackbox* .hg/blackbox.log: < 500 .hg/blackbox.log.1: >= 500 .hg/blackbox.log.2: >= 500 $ cd .. With chg, blackbox should not create the log file if the repo is gone $ hg init repo1 $ hg --config extensions.a=! -R repo1 log $ rm -rf $TESTTMP/repo1 $ hg --config extensions.a=! init repo1 #endif blackbox should work if repo.ui.log is not called (issue5518) $ cat > $TESTTMP/raise.py << EOF > from __future__ import absolute_import > from mercurial import registrar, scmutil > cmdtable = {} > command = registrar.command(cmdtable) > @command(b'raise') > def raisecmd(*args): > raise RuntimeError('raise') > EOF $ cat >> $HGRCPATH << EOF > [blackbox] > track = commandexception > [extensions] > raise=$TESTTMP/raise.py > EOF $ hg init $TESTTMP/blackbox-exception-only $ cd $TESTTMP/blackbox-exception-only #if chg (chg exits 255 because it fails to receive an exit code) $ hg raise 2>/dev/null [255] #else (hg exits 1 because Python default exit code for uncaught exception is 1) $ hg raise 2>/dev/null [1] #endif $ head -1 .hg/blackbox.log 1970/01/01 00:00:00 bob @0000000000000000000000000000000000000000 (5000)> ** Unknown exception encountered with possibly-broken third-party extension mock $ tail -2 .hg/blackbox.log RuntimeError: raise