Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 14:04:07 +0200] rev 16721
branches: improve performance by removing redundant operations
This refactors the branches command so it collects all the information it needs
about a branch in one pass over the branch map.
In particular, it fixes an issue where the command called repo.branchtags() to
get branch tips, and then used repo.branchheads() to get the closed/open
status. Both repo methods read the changelog to determine if the branch is
closed, resulting in extra, redundant I/O.
For the PyPy repo with 744 branches and 843 branch heads, this brings
hg branches over NFS from:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
2427 0 0.9057 0.9057 <open>
2424 0 0.4096 0.4096 <method 'close' of 'file' objects>
2424 0 0.0476 0.0476 <method 'read' of 'file' objects>
1 0 0.0468 0.0468 mercurial.revlog:637(headrevs)
+1 0 0.0000 0.0000 +<len>
2422 0 0.0443 0.0443 <method 'seek' of 'file' objects>
2962 0 0.0337 0.0337 <zlib.decompress>
2491 0 1.8008 0.0322 mercurial.changelog:182(read)
+2491 0 1.6982 0.0315 +mercurial.revlog:881(revision)
+2488 0 0.0269 0.0134 +mercurial.changelog:28(decodeextra)
+4982 0 0.0085 0.0085 +<method 'split' of 'str' objects>
+4982 0 0.0274 0.0049 +mercurial.encoding:61(tolocal)
+2491 0 0.0039 0.0039 +<method 'index' of 'str' objects>
2491 0 1.6982 0.0315 mercurial.revlog:881(revision)
+2413 0 0.0112 0.0112 +mercurial.revlog:305(rev)
+2491 0 1.5315 0.0068 +mercurial.revlog:847(_chunkraw)
+2491 0 0.0414 0.0058 +mercurial.revlog:945(_checkhash)
+2491 0 0.0028 0.0028 +mercurial.revlog:349(flags)
+2491 0 0.0025 0.0025 +<mercurial.mpatch.patches>
2422 0 1.5089 0.0286 mercurial.revlog:818(_loadchunk)
+2422 0 0.4093 0.4093 +<method 'close' of 'file' objects>
+2422 0 0.0451 0.0451 +<method 'read' of 'file' objects>
+2422 0 0.0443 0.0443 +<method 'seek' of 'file' objects>
+2422 0 0.9703 0.0096 +mercurial.store:374(__call__)
+2422 0 0.0079 0.0069 +mercurial.revlog:810(_addchunk)
5804 0 0.0204 0.0204 mercurial.revlog:305(rev)
2426 0 0.9552 0.0177 mercurial.scmutil:218(__call__)
+2426 0 0.9057 0.9057 +<open>
+2426 0 0.0120 0.0083 +os.path:80(split)
+2426 0 0.0061 0.0049 +mercurial.scmutil:92(__call__)
Time: real 1.950 secs (user 0.560+0.000 sys 0.220+0.000)
down to:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
1545 0 0.6035 0.6035 <open>
1542 0 0.2697 0.2697 <method 'close' of 'file' objects>
1 0 0.0547 0.0547 mercurial.revlog:637(headrevs)
+1 0 0.0000 0.0000 +<len>
1542 0 0.0389 0.0389 <method 'read' of 'file' objects>
1540 0 0.0316 0.0316 <method 'seek' of 'file' objects>
1853 0 0.0227 0.0227 <zlib.decompress>
1557 0 1.2131 0.0226 mercurial.changelog:182(read)
+1557 0 1.1398 0.0221 +mercurial.revlog:881(revision)
+1555 0 0.0199 0.0094 +mercurial.changelog:28(decodeextra)
+3114 0 0.0058 0.0058 +<method 'split' of 'str' objects>
+3114 0 0.0196 0.0035 +mercurial.encoding:61(tolocal)
+1557 0 0.0026 0.0026 +<method 'index' of 'str' objects>
1557 0 1.1398 0.0221 mercurial.revlog:881(revision)
+1557 0 1.0307 0.0047 +mercurial.revlog:847(_chunkraw)
+1557 0 0.0287 0.0040 +mercurial.revlog:945(_checkhash)
+1557 0 0.0018 0.0018 +<mercurial.mpatch.patches>
+1557 0 0.0018 0.0018 +mercurial.revlog:326(node)
+1557 0 0.0417 0.0013 +mercurial.revlog:857(_chunkbase)
1540 0 1.0147 0.0210 mercurial.revlog:818(_loadchunk)
+1540 0 0.2693 0.2693 +<method 'close' of 'file' objects>
+1540 0 0.0360 0.0360 +<method 'read' of 'file' objects>
+1540 0 0.0316 0.0316 +<method 'seek' of 'file' objects>
+1540 0 0.6487 0.0070 +mercurial.store:374(__call__)
+1540 0 0.0059 0.0052 +mercurial.revlog:810(_addchunk)
3192 0 0.0173 0.0173 mercurial.revlog:305(rev)
8184 0 0.0300 0.0147 <method 'decode' of 'str' objects>
+5204 0 0.0149 0.0048 +encodings.utf_8:15(decode)
+1 0 0.0004 0.0000 +encodings:71(search_function)
43 26 0.0147 0.0129 <__import__>
Time: real 1.390 secs (user 0.450+0.000 sys 0.170+0.000)
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 14:04:06 +0200] rev 16720
context: add changectx.closesbranch() method
This removes the duplicated code for inspecting the 'close' extra field in
a changeset.
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 14:04:04 +0200] rev 16719
localrepo: add branchtip() method for faster single-branch lookups
For the PyPy repo with 744 branches and 843 branch heads, this brings
hg log -r default over NFS from:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
3249 0 1.3222 1.3222 <open>
3244 0 0.6211 0.6211 <method 'close' of 'file' objects>
3243 0 0.0800 0.0800 <method 'read' of 'file' objects>
3241 0 0.0660 0.0660 <method 'seek' of 'file' objects>
3905 0 0.0476 0.0476 <zlib.decompress>
3281 0 2.6756 0.0472 mercurial.changelog:182(read)
+3281 0 2.5256 0.0453 +mercurial.revlog:881(revision)
+3276 0 0.0389 0.0196 +mercurial.changelog:28(decodeextra)
+6562 0 0.0123 0.0123 +<method 'split' of 'str' objects>
+6562 0 0.0408 0.0073 +mercurial.encoding:61(tolocal)
+3281 0 0.0054 0.0054 +<method 'index' of 'str' objects>
3241 0 2.2464 0.0456 mercurial.revlog:818(_loadchunk)
+3241 0 0.6205 0.6205 +<method 'close' of 'file' objects>
+3241 0 0.0765 0.0765 +<method 'read' of 'file' objects>
+3241 0 0.0660 0.0660 +<method 'seek' of 'file' objects>
+3241 0 1.4209 0.0135 +mercurial.store:374(__call__)
+3241 0 0.0122 0.0107 +mercurial.revlog:810(_addchunk)
3281 0 2.5256 0.0453 mercurial.revlog:881(revision)
+3280 0 0.0175 0.0175 +mercurial.revlog:305(rev)
+3281 0 2.2819 0.0119 +mercurial.revlog:847(_chunkraw)
+3281 0 0.0603 0.0083 +mercurial.revlog:945(_checkhash)
+3281 0 0.0051 0.0051 +mercurial.revlog:349(flags)
+3281 0 0.0040 0.0040 +<mercurial.mpatch.patches>
13682 0 0.0479 0.0248 <method 'decode' of 'str' objects>
+7418 0 0.0228 0.0076 +encodings.utf_8:15(decode)
+1 0 0.0003 0.0000 +encodings:71(search_function)
3248 0 1.3995 0.0246 mercurial.scmutil:218(__call__)
+3248 0 1.3222 1.3222 +<open>
+3248 0 0.0235 0.0184 +os.path:80(split)
+3248 0 0.0084 0.0068 +mercurial.scmutil:92(__call__)
Time: real 2.750 secs (user 0.680+0.000 sys 0.360+0.000)
down to:
CallCount Recursive Total(ms) Inline(ms) module:lineno(function)
55 31 0.0197 0.0163 <__import__>
+1 0 0.0006 0.0002 +mercurial.context:8(<module>)
+1 0 0.0042 0.0001 +mercurial.revlog:12(<module>)
+1 0 0.0002 0.0001 +mercurial.match:8(<module>)
+1 0 0.0003 0.0001 +mercurial.dirstate:7(<module>)
+1 0 0.0057 0.0001 +mercurial.changelog:8(<module>)
1 0 0.0117 0.0032 mercurial.localrepo:525(_readbranchcache)
+844 0 0.0015 0.0015 +<binascii.unhexlify>
+845 0 0.0010 0.0010 +<method 'split' of 'str' objects>
+843 0 0.0045 0.0009 +mercurial.encoding:61(tolocal)
+843 0 0.0004 0.0004 +<method 'setdefault' of 'dict' objects>
+1 0 0.0003 0.0003 +<method 'close' of 'file' objects>
3 0 0.0029 0.0029 <method 'read' of 'file' objects>
9 0 0.0018 0.0018 <open>
990 0 0.0017 0.0017 <binascii.unhexlify>
53 0 0.0016 0.0016 mercurial.demandimport:43(__init__)
862 0 0.0015 0.0015 <_codecs.utf_8_decode>
862 0 0.0037 0.0014 <method 'decode' of 'str' objects>
+862 0 0.0023 0.0008 +encodings.utf_8:15(decode)
981 0 0.0011 0.0011 <method 'split' of 'str' objects>
861 0 0.0046 0.0009 mercurial.encoding:61(tolocal)
+861 0 0.0037 0.0014 +<method 'decode' of 'str' objects>
862 0 0.0023 0.0008 encodings.utf_8:15(decode)
+862 0 0.0015 0.0015 +<_codecs.utf_8_decode>
4 0 0.0008 0.0008 <method 'close' of 'file' objects>
179 154 0.0202 0.0004 mercurial.demandimport:83(__getattribute__)
+36 11 0.0199 0.0003 +mercurial.demandimport:55(_load)
+72 0 0.0001 0.0001 +mercurial.demandimport:83(__getattribute__)
+36 0 0.0000 0.0000 +<getattr>
1 0 0.0015 0.0004 mercurial.tags:148(_readtagcache)
Time: real 0.060 secs (user 0.030+0.000 sys 0.010+0.000)
David Soria Parra <dsp@php.net> [Sun, 13 May 2012 16:39:40 +0200] rev 16718
strip: introduce -B option to remove a bookmark
Add a -B option to remove a bookmark. All revisions are unreachable
from a different head or a different bookmark will be removed too.
This helps with topic branch workflow. You can create a topic branch
and remove it if not needed anymore with hg strip -B topic/xyz.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 May 2012 17:34:55 +0200] rev 16717
largefiles: use wlock for lfconvert (issue3444)
largefiles and lfconvert do dirty hacks with dirstate, so to avoid writing that
as a side effect of the wlock release we clear dirstate first.
To avoid confusing lock validation algorithms in error situations we unlock
_before_ removing the target directory.
Joshua Redstone <joshua.redstone@fb.com> [Fri, 11 May 2012 10:35:54 -0700] rev 16716
localrepo: strip now incrementally updates the branchheads cache
Destroying history via strip used to invalidate the branchheads cache,
causing it to be regenerated the next time it is read. This is
expensive in large repos. This change converts strip to pass info to
localrepo.destroyed() to enable to it to incrementally update the
cache, improving the performance of strip and other operations that
depend on it (e.g., rebase).
This change also strengthens a bit the integrity checking of the
branchheads cache when it is read, by rejecting the cache if it has
nodes in it that no longer exist.
Patrick Mezard <patrick@mezard.eu> [Sat, 12 May 2012 16:53:36 +0200] rev 16715
phase: remove useless test, olddata is never None
Patrick Mezard <patrick@mezard.eu> [Sun, 13 May 2012 14:29:05 +0200] rev 16714
test-rebase-collapse: fix Windows path separator
Matt Mackall <mpm@selenic.com> [Sun, 13 May 2012 15:26:06 +0200] rev 16713
help: tweak keyword format output
Augie Fackler <raf@durin42.com> [Sun, 13 May 2012 05:22:23 -0500] rev 16712
help: link back to the help topics page from a specific topic page
Augie Fackler <raf@durin42.com> [Sun, 13 May 2012 06:03:11 -0500] rev 16711
help: add --keyword (-k) for searching help
Augie Fackler <raf@durin42.com> [Sun, 13 May 2012 04:27:08 -0500] rev 16710
help: introduce topicmatch for finding topics matching a keyword
Augie Fackler <raf@durin42.com> [Sun, 13 May 2012 04:06:07 -0500] rev 16709
extensions.disabled: return {} instead of None no extensions are disabled
David Soria Parra <dsp@php.net> [Sun, 13 May 2012 11:55:42 +0200] rev 16708
merge: respect bookmarks during merge
Bookmarks will behave more like named branches when merge tries to pick
a revision to merge.
Bookmarks now to respect the current bookmarks. Bookmarks will not
accidentally merged with unnamed heads or other bookmarks. However merge
can pick heads with diverging bookmarks and pick those automatically.
We end up with two cases for picking a revision to merge:
(1) In case of an current bookmark, merge can pick a branch head that has a
diverged bookmark
(2) In case of no current bookmark, merge can pick a branch head that does not
have a bookmark.
David Soria Parra <dsp@php.net> [Sun, 13 May 2012 11:54:58 +0200] rev 16707
localrepo: introduce bookmarkheads
Similar to branch heads we introduce the notion of bookmarkheads.
Bookmarkheads are changests that are bookmarked with the given bookmark
or a diverged version
David Soria Parra <dsp@php.net> [Sat, 12 May 2012 21:28:10 +0200] rev 16706
bookmarks: delete divergent bookmarks on merge
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 13:18:06 +0200] rev 16705
check-code: ignore naked excepts with a "re-raise" comment
This also promotes the naked except check from a warning to an error.
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 13:17:50 +0200] rev 16704
check-code: improve detection of naked except clauses
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 13:17:31 +0200] rev 16703
cleanup: replace more naked excepts with more specific ones
Brodie Rao <brodie@sf.io> [Sun, 13 May 2012 13:17:27 +0200] rev 16702
check-code: promote 80+ character line warning to an error
Patrick Mezard <patrick@mezard.eu> [Sun, 13 May 2012 13:19:46 +0200] rev 16701
cmdutil: use filectx.cmp() in amend()
Na'Tosha Bard <natosha@unity3d.com> [Sat, 12 May 2012 15:41:27 +0200] rev 16700
largefiles: optimize performance when updating (issue3440)
Previously, when updating, cachelfiles was called blindly on all largefiles
in the repository at the revision being updated to, despite the fact that
a list of which largefiles needs to be updated has already been collected. This
optimization constrains the cachelfiles call to only the largefiles that need
to be updated.
On a repository with about 80 largefiles, updating between two revisions that
only change one largefile goes from approximately 6.7 seconds to 3.3 seconds.
Matt Mackall <mpm@selenic.com> [Sun, 13 May 2012 12:52:24 +0200] rev 16699
merge with stable
Bryan O'Sullivan <bryano@fb.com> [Sun, 13 May 2012 11:47:55 +0200] rev 16698
tests: make test-convert-darcs.t happier with darcs 2.8.0
Levi Bard <levi@unity3d.com> [Sat, 12 May 2012 17:00:01 +0200] rev 16697
bookmarks: pull new bookmarks from remote by default (BC)
Patrick Mezard <patrick@mezard.eu> [Thu, 03 May 2012 15:14:58 +0200] rev 16696
rebase: allow collapsing branches in place (issue3111)
We allow rebase plus collapse, but not collapse only? I imagine people would
rebase first then collapse once they are sure the rebase is correct and it is
the right time to finish it.
I was reluctant to submit this patch for reasons detailed below, but it
improves rebase --collapse usefulness so much it is worth the ugliness.
The fix is ugly because we should be fixing the collapse code path rather than
the merge. Collapsing by merging changesets repeatedly is inefficient compared
to what commit --amend does: commitctx(), update, strip. The problem with the
latter is, to generate the synthetic changeset, copy records are gathered with
copies.pathcopies(). copies.pathcopies() is still implemented with merging in
mind and discards information like file replaced by the copy of another,
criss-cross copies and so forth. I believe this information should not be lost,
even if we decide not to interpret it fully later, at merge time.
The second issue with improving rebase --collapse is the option should not be
there to begin with. Rebasing and collapsing are orthogonal and a dedicated
command would probably enable a better, simpler ui. We should avoid advertizing
rebase --collapse, but with this fix it becomes the best shipped solution to
collapse changesets.
And for the record, available techniques are:
- revert + commit + strip: lose copies
- mq/qfold: repeated patching() (mostly correct, fragile)
- rebase: repeated merges (mostly correct, fragile)
- collapse: revert + tag rewriting wizardry, lose copies
- histedit: repeated patching() (mostly correct, fragile)
- amend: copies.pathcopies() + commitctx() + update + strip
Adrian Buehlmann <adrian@cadifra.com> [Sat, 12 May 2012 14:00:51 +0200] rev 16695
diffhelpers: use Py_ssize_t in testhunk()
Eliminates
mercurial/diffhelpers.c(143) : warning C4244: '=' : conversion from
'Py_ssize_t' to 'int', possible loss of data
mercurial/diffhelpers.c(144) : warning C4244: '=' : conversion from
'Py_ssize_t' to 'int', possible loss of data
when compiling for Windows x64 target using the Microsoft compiler.
Adrian Buehlmann <adrian@cadifra.com> [Sat, 12 May 2012 13:21:08 +0200] rev 16694
diffhelpers: use Py_ssize_t in addlines()
Eliminates
mercurial/diffhelpers.c(81) : warning C4244: '=' : conversion from
'Py_ssize_t' to 'int', possible loss of data
mercurial/diffhelpers.c(82) : warning C4244: '=' : conversion from
'Py_ssize_t' to 'int', possible loss of data
when compiling for Windows x64 target using the Microsoft compiler.
Adrian Buehlmann <adrian@cadifra.com> [Sat, 12 May 2012 12:07:18 +0200] rev 16693
diffhelpers: use Py_ssize_t in _fix_newline()
Eliminates
mercurial/diffhelpers.c(23) : warning C4244: 'initializing' : conversion from
'Py_ssize_t' to 'int', possible loss of data
mercurial/diffhelpers.c(26) : warning C4244: 'initializing' : conversion from
'Py_ssize_t' to 'int', possible loss of data
mercurial/diffhelpers.c(27) : warning C4244: 'initializing' : conversion from
'Py_ssize_t' to 'int', possible loss of data
mercurial/diffhelpers.c(30) : warning C4244: 'initializing' : conversion from
'Py_ssize_t' to 'int', possible loss of data
when compiling for Windows x64 target using the Microsoft compiler.
Na'Tosha Bard <natosha@unity3d.com> [Sat, 12 May 2012 11:45:22 +0200] rev 16692
largefiles: add --all-largefiles flag to pull
Na'Tosha Bard <natosha@unity3d.com> [Sat, 12 May 2012 09:59:01 +0200] rev 16691
largefiles: refactor downloading of all largefiles to generic function
Mads Kiilerich <mads@kiilerich.com> [Sat, 12 May 2012 20:06:04 +0200] rev 16690
unbundle: move bookmark update outside the lock
d16c99f16f00 introduced updatecurrentbookmark after unbundle, but that path
might end up taking a wlock after lock when writing the bookmarks.