Mercurial > hg-stable
view tests/test-bundle-r.t @ 15141:16dc9a32ca04
mdiff: speed up showfunc for large diffs
This addresses the following issues with showfunc:
- Silly usage of regular expressions.
- Doing str.rstrip() needlessly in an inner loop.
- Doing catastrophic backtracking when trying to find a function line.
Finding function text is now at worst O(n lines in the old file), and
at best close to O(n hunks).
Given a diff like this[1]:
src/main/antlr3/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunker.g | 4 +-
src/main/java/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunkerLexer.java | 2 +-
src/main/java/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunkerParser.java | 29189 +++++----
3 files changed, 14741 insertions(+), 14454 deletions(-)
[1]: https://bitbucket.org/wwmm/chemicaltagger/changeset/d2bfbaecd4fc/raw
Without this change, hg log --stat --config diff.showfunc=1 takes an
absurdly long time to complete:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
32813 0 80.3546 40.6086 mercurial.mdiff:160(yieldhunk)
+65062746 0 25.7227 25.7227 +<method 'match' of '_sre.SRE_Pattern' objects>
+65062746 0 14.0221 14.0221 +<method 'rstrip' of 'str' objects>
+1809 0 0.0009 0.0009 +mercurial.mdiff:148(contextend)
+1809 0 0.0003 0.0003 +<len>
65062746 0 25.7227 25.7227 <method 'match' of '_sre.SRE_Pattern' objects>
65062763 0 14.0221 14.0221 <method 'rstrip' of 'str' objects>
543 0 0.1631 0.1631 <zlib.decompress>
3 0 0.0505 0.0505 <mercurial.bdiff.blocks>
31007 0 80.4564 0.0477 mercurial.mdiff:147(_unidiff)
+32813 0 80.3546 40.6086 +mercurial.mdiff:160(yieldhunk)
+3 0 0.0505 0.0505 +<mercurial.bdiff.blocks>
+3618 0 0.0022 0.0022 +mercurial.mdiff:154(contextstart)
+5427 0 0.0013 0.0013 +<len>
+3 0 0.0001 0.0000 +re:188(compile)
1 0 80.8381 0.0322 mercurial.patch:1777(diffstatdata)
+107499 0 0.0235 0.0235 +<method 'startswith' of 'str' objects>
+31014 0 80.7820 0.0071 +mercurial.util:1284(iterlines)
+3 0 0.0000 0.0000 +<method 'search' of '_sre.SRE_Pattern' objects>
+4 0 0.0000 0.0000 +mercurial.patch:1783(addresult)
+3 0 0.0000 0.0000 +<method 'group' of '_sre.SRE_Match' objects>
6 0 0.0444 0.0283 mercurial.mdiff:12(splitnewlines)
+6 0 0.0160 0.0160 +<method 'split' of 'str' objects>
32 0 0.0246 0.0246 <method 'update' of '_hashlib.HASH' objects>
11 0 0.0236 0.0236 <method 'read' of 'file' objects>
Time: real 80.880 secs (user 80.200+0.000 sys 0.380+0.000)
With this change, it's almost as fast as not using showfunc at all:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
543 0 0.1699 0.1699 <zlib.decompress>
3 0 0.0501 0.0501 <mercurial.bdiff.blocks>
32813 0 0.0415 0.0348 mercurial.mdiff:161(yieldhunk)
+70837 0 0.0058 0.0058 +<method 'isalnum' of 'str' objects>
+1809 0 0.0006 0.0006 +mercurial.mdiff:148(contextend)
+1809 0 0.0002 0.0002 +<len>
1 0 0.4879 0.0310 mercurial.patch:1777(diffstatdata)
+107499 0 0.0230 0.0230 +<method 'startswith' of 'str' objects>
+31014 0 0.4335 0.0065 +mercurial.util:1284(iterlines)
+3 0 0.0000 0.0000 +<method 'search' of '_sre.SRE_Pattern' objects>
+4 0 0.0000 0.0000 +mercurial.patch:1783(addresult)
+1 0 0.0004 0.0000 +re:188(compile)
32 0 0.0293 0.0293 <method 'update' of '_hashlib.HASH' objects>
6 0 0.0427 0.0279 mercurial.mdiff:12(splitnewlines)
+6 0 0.0147 0.0147 +<method 'split' of 'str' objects>
31007 0 0.1169 0.0235 mercurial.mdiff:147(_unidiff)
+3 0 0.0501 0.0501 +<mercurial.bdiff.blocks>
+32813 0 0.0415 0.0348 +mercurial.mdiff:161(yieldhunk)
+3618 0 0.0012 0.0012 +mercurial.mdiff:154(contextstart)
+5427 0 0.0006 0.0006 +<len>
107597 0 0.0230 0.0230 <method 'startswith' of 'str' objects>
16 0 0.0213 0.0213 <mercurial.mpatch.patches>
194 0 0.0149 0.0149 <method 'split' of 'str' objects>
Time: real 0.530 secs (user 0.450+0.000 sys 0.070+0.000)
| author | Brodie Rao <brodie@bitheap.org> |
|---|---|
| date | Mon, 19 Sep 2011 15:58:03 -0700 |
| parents | 7c26ce9edbd2 |
| children | 4f795f5fbb0b |
line wrap: on
line source
$ hg init test $ cd test $ hg unbundle $TESTDIR/bundles/remote.hg adding changesets adding manifests adding file changes added 9 changesets with 7 changes to 4 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) $ hg up tip 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd .. $ for i in 0 1 2 3 4 5 6 7 8; do > mkdir test-"$i" > hg --cwd test-"$i" init > hg -R test bundle -r "$i" test-"$i".hg test-"$i" > cd test-"$i" > hg unbundle ../test-"$i".hg > hg verify > hg tip -q > cd .. > done searching for changes 1 changesets found adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 1 changesets, 1 total revisions 0:bfaf4b5cbf01 searching for changes 2 changesets found adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 2 changesets, 2 total revisions 1:21f32785131f searching for changes 3 changesets found adding changesets adding manifests adding file changes added 3 changesets with 3 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 3 changesets, 3 total revisions 2:4ce51a113780 searching for changes 4 changesets found adding changesets adding manifests adding file changes added 4 changesets with 4 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 4 changesets, 4 total revisions 3:93ee6ab32777 searching for changes 2 changesets found adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 2 changesets, 2 total revisions 1:c70afb1ee985 searching for changes 3 changesets found adding changesets adding manifests adding file changes added 3 changesets with 3 changes to 1 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 1 files, 3 changesets, 3 total revisions 2:f03ae5a9b979 searching for changes 4 changesets found adding changesets adding manifests adding file changes added 4 changesets with 5 changes to 2 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 2 files, 4 changesets, 5 total revisions 3:095cb14b1b4d searching for changes 5 changesets found adding changesets adding manifests adding file changes added 5 changesets with 6 changes to 3 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 3 files, 5 changesets, 6 total revisions 4:faa2e4234c7a searching for changes 5 changesets found adding changesets adding manifests adding file changes added 5 changesets with 5 changes to 2 files (run 'hg update' to get a working copy) checking changesets checking manifests crosschecking files in changesets and manifests checking files 2 files, 5 changesets, 5 total revisions 4:916f1afdef90 $ cd test-8 $ hg pull ../test-7 pulling from ../test-7 searching for changes adding changesets adding manifests adding file changes added 4 changesets with 2 changes to 3 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 4 files, 9 changesets, 7 total revisions $ hg rollback repository tip rolled back to revision 4 (undo pull) $ cd .. should fail $ hg -R test bundle --base 2 -r tip test-bundle-branch1.hg test-3 abort: --base is incompatible with specifying a destination [255] $ hg -R test bundle -r tip test-bundle-branch1.hg abort: repository default-push not found! [255] $ hg -R test bundle --base 2 -r tip test-bundle-branch1.hg 2 changesets found $ hg -R test bundle --base 2 -r 7 test-bundle-branch2.hg 4 changesets found $ hg -R test bundle --base 2 test-bundle-all.hg 6 changesets found $ hg -R test bundle --base 3 -r tip test-bundle-should-fail.hg 1 changesets found empty bundle $ hg -R test bundle --base 7 --base 8 test-bundle-empty.hg no changes found [1] issue76 msg2163 $ hg -R test bundle --base 3 -r 3 -r 3 test-bundle-cset-3.hg no changes found [1] Issue1910: 'hg bundle --base $head' does not exclude $head from result $ hg -R test bundle --base 7 test-bundle-cset-7.hg 4 changesets found $ hg clone test-2 test-9 updating to branch default 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd test-9 revision 2 $ hg tip -q 2:4ce51a113780 $ hg unbundle ../test-bundle-should-fail.hg adding changesets transaction abort! rollback completed abort: 00changelog.i@93ee6ab32777: unknown parent! [255] revision 2 $ hg tip -q 2:4ce51a113780 $ hg unbundle ../test-bundle-all.hg adding changesets adding manifests adding file changes added 6 changesets with 4 changes to 4 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) revision 8 $ hg tip -q 8:916f1afdef90 $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 4 files, 9 changesets, 7 total revisions $ hg rollback repository tip rolled back to revision 2 (undo unbundle) revision 2 $ hg tip -q 2:4ce51a113780 $ hg unbundle ../test-bundle-branch1.hg adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 2 files (run 'hg update' to get a working copy) revision 4 $ hg tip -q 4:916f1afdef90 $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 2 files, 5 changesets, 5 total revisions $ hg rollback repository tip rolled back to revision 2 (undo unbundle) $ hg unbundle ../test-bundle-branch2.hg adding changesets adding manifests adding file changes added 4 changesets with 3 changes to 3 files (+1 heads) (run 'hg heads' to see heads, 'hg merge' to merge) revision 6 $ hg tip -q 6:faa2e4234c7a $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 3 files, 7 changesets, 6 total revisions $ hg rollback repository tip rolled back to revision 2 (undo unbundle) $ hg unbundle ../test-bundle-cset-7.hg adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 2 files (run 'hg update' to get a working copy) revision 4 $ hg tip -q 4:916f1afdef90 $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 2 files, 5 changesets, 5 total revisions $ cd ../test $ hg merge 7 note: possible conflict - afile was renamed multiple times to: anotherfile adifferentfile 2 files updated, 0 files merged, 0 files removed, 0 files unresolved (branch merge, don't forget to commit) $ hg ci -m merge $ cd .. $ hg -R test bundle --base 2 test-bundle-head.hg 7 changesets found $ hg clone test-2 test-10 updating to branch default 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd test-10 $ hg unbundle ../test-bundle-head.hg adding changesets adding manifests adding file changes added 7 changesets with 4 changes to 4 files (run 'hg update' to get a working copy) revision 9 $ hg tip -q 9:03fc0b0e347c $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files 4 files, 10 changesets, 7 total revisions