annotate tests/test-init.t @ 15141:16dc9a32ca04

mdiff: speed up showfunc for large diffs

This addresses the following issues with showfunc:

- Silly usage of regular expressions.
- Doing str.rstrip() needlessly in an inner loop.
- Doing catastrophic backtracking when trying to find a function line.

Finding function text is now at worst O(n lines in the old file), and
at best close to O(n hunks).

Given a diff like this[1]:

 src/main/antlr3/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunker.g        |      4 +-
 src/main/java/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunkerLexer.java  |      2 +-
 src/main/java/uk/ac/cam/ch/wwmm/pregenerated/ChemicalChunkerParser.java |  29189 +++++----
 3 files changed, 14741 insertions(+), 14454 deletions(-)

[1]: https://bitbucket.org/wwmm/chemicaltagger/changeset/d2bfbaecd4fc/raw

Without this change, hg log --stat --config diff.showfunc=1 takes an
absurdly long time to complete:

   CallCount    Recursive    Total(ms)   Inline(ms) module:lineno(function)
       32813            0     80.3546     40.6086   mercurial.mdiff:160(yieldhunk)
   [phone]            0     25.7227     25.7227   +<method 'match' of '_sre.SRE_Pattern' objects>
   [phone]            0     14.0221     14.0221   +<method 'rstrip' of 'str' objects>
       +1809            0      0.0009      0.0009   +mercurial.mdiff:148(contextend)
       +1809            0      0.0003      0.0003   +<len>
    65062746            0     25.7227     25.7227   <method 'match' of '_sre.SRE_Pattern' objects>
    65062763            0     14.0221     14.0221   <method 'rstrip' of 'str' objects>
         543            0      0.1631      0.1631   <zlib.decompress>
           3            0      0.0505      0.0505   <mercurial.bdiff.blocks>
       31007            0     80.4564      0.0477   mercurial.mdiff:147(_unidiff)
      +32813            0     80.3546     40.6086   +mercurial.mdiff:160(yieldhunk)
          +3            0      0.0505      0.0505   +<mercurial.bdiff.blocks>
       +3618            0      0.0022      0.0022   +mercurial.mdiff:154(contextstart)
       +5427            0      0.0013      0.0013   +<len>
          +3            0      0.0001      0.0000   +re:188(compile)
           1            0     80.8381      0.0322   mercurial.patch:1777(diffstatdata)
     +107499            0      0.0235      0.0235   +<method 'startswith' of 'str' objects>
      +31014            0     80.7820      0.0071   +mercurial.util:1284(iterlines)
          +3            0      0.0000      0.0000   +<method 'search' of '_sre.SRE_Pattern' objects>
          +4            0      0.0000      0.0000   +mercurial.patch:1783(addresult)
          +3            0      0.0000      0.0000   +<method 'group' of '_sre.SRE_Match' objects>
           6            0      0.0444      0.0283   mercurial.mdiff:12(splitnewlines)
          +6            0      0.0160      0.0160   +<method 'split' of 'str' objects>
          32            0      0.0246      0.0246   <method 'update' of '_hashlib.HASH' objects>
          11            0      0.0236      0.0236   <method 'read' of 'file' objects>
Time: real 80.880 secs (user 80.200+0.000 sys 0.380+0.000)

With this change, it's almost as fast as not using showfunc at all:

   CallCount    Recursive    Total(ms)   Inline(ms) module:lineno(function)
         543            0      0.1699      0.1699   <zlib.decompress>
           3            0      0.0501      0.0501   <mercurial.bdiff.blocks>
       32813            0      0.0415      0.0348   mercurial.mdiff:161(yieldhunk)
      +70837            0      0.0058      0.0058   +<method 'isalnum' of 'str' objects>
       +1809            0      0.0006      0.0006   +mercurial.mdiff:148(contextend)
       +1809            0      0.0002      0.0002   +<len>
           1            0      0.4879      0.0310   mercurial.patch:1777(diffstatdata)
     +107499            0      0.0230      0.0230   +<method 'startswith' of 'str' objects>
      +31014            0      0.4335      0.0065   +mercurial.util:1284(iterlines)
          +3            0      0.0000      0.0000   +<method 'search' of '_sre.SRE_Pattern' objects>
          +4            0      0.0000      0.0000   +mercurial.patch:1783(addresult)
          +1            0      0.0004      0.0000   +re:188(compile)
          32            0      0.0293      0.0293   <method 'update' of '_hashlib.HASH' objects>
           6            0      0.0427      0.0279   mercurial.mdiff:12(splitnewlines)
          +6            0      0.0147      0.0147   +<method 'split' of 'str' objects>
       31007            0      0.1169      0.0235   mercurial.mdiff:147(_unidiff)
          +3            0      0.0501      0.0501   +<mercurial.bdiff.blocks>
      +32813            0      0.0415      0.0348   +mercurial.mdiff:161(yieldhunk)
       +3618            0      0.0012      0.0012   +mercurial.mdiff:154(contextstart)
       +5427            0      0.0006      0.0006   +<len>
      107597            0      0.0230      0.0230   <method 'startswith' of 'str' objects>
          16            0      0.0213      0.0213   <mercurial.mpatch.patches>
         194            0      0.0149      0.0149   <method 'split' of 'str' objects>
Time: real 0.530 secs (user 0.450+0.000 sys 0.070+0.000)

This test tries to exercise the ssh functionality with a dummy script

  $ checknewrepo()
  > {
  >    name=$1
  >    if [ -d "$name"/.hg/store ]; then
  >    echo store created
  >    fi
  >    if [ -f "$name"/.hg/00changelog.i ]; then
  >    echo 00changelog.i created
  >    fi
  >    cat "$name"/.hg/requires
  > }

creating 'local'

  $ hg init local
  $ checknewrepo local
  store created
  00changelog.i created
  revlogv1
  fncache
  store
  dotencode
  $ echo this > local/foo
  $ hg ci --cwd local -A -m "init"
  adding foo

creating repo with format.usestore=false

  $ hg --config format.usestore=false init old
  $ checknewrepo old
  revlogv1

creating repo with format.usefncache=false

  $ hg --config format.usefncache=false init old2
  $ checknewrepo old2
  store created
  00changelog.i created
  revlogv1
  store

creating repo with format.dotencode=false

  $ hg --config format.dotencode=false init old3
  $ checknewrepo old3
  store created
  00changelog.i created
  revlogv1
  fncache
  store

test failure

  $ hg init local
  abort: repository local already exists!
  [255]

init+push to remote2

  $ hg init -e "python $TESTDIR/dummyssh" ssh://user@dummy/remote2
  $ hg incoming -R remote2 local
  comparing with local
  changeset:   0:08b9e9f63b32
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     init
  

  $ hg push -R local -e "python $TESTDIR/dummyssh" ssh://user@dummy/remote2
  pushing to ssh://user@dummy/remote2
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 1 changesets with 1 changes to 1 files

clone to remote1

  $ hg clone -e "python $TESTDIR/dummyssh" local ssh://user@dummy/remote1
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 1 changesets with 1 changes to 1 files

init to existing repo

  $ hg init -e "python $TESTDIR/dummyssh" ssh://user@dummy/remote1
  abort: repository remote1 already exists!
  abort: could not create remote repo!
  [255]

clone to existing repo

  $ hg clone -e "python $TESTDIR/dummyssh" local ssh://user@dummy/remote1
  abort: repository remote1 already exists!
  abort: could not create remote repo!
  [255]

output of dummyssh

  $ cat dummylog
  Got arguments 1:user@dummy 2:hg init remote2
  Got arguments 1:user@dummy 2:hg -R remote2 serve --stdio
  Got arguments 1:user@dummy 2:hg -R remote2 serve --stdio
  Got arguments 1:user@dummy 2:hg init remote1
  Got arguments 1:user@dummy 2:hg -R remote1 serve --stdio
  Got arguments 1:user@dummy 2:hg init remote1
  Got arguments 1:user@dummy 2:hg init remote1

comparing repositories

  $ hg tip -q -R local
  0:08b9e9f63b32
  $ hg tip -q -R remote1
  0:08b9e9f63b32
  $ hg tip -q -R remote2
  0:08b9e9f63b32

check names for repositories (clashes with URL schemes, special chars)

  $ for i in bundle file hg http https old-http ssh static-http " " "with space"; do
  >   printf "hg init \"$i\"... "
  >   hg init "$i"
  >   test -d "$i" -a -d "$i/.hg" && echo "ok" || echo "failed"
  > done
  hg init "bundle"... ok
  hg init "file"... ok
  hg init "hg"... ok
  hg init "http"... ok
  hg init "https"... ok
  hg init "old-http"... ok
  hg init "ssh"... ok
  hg init "static-http"... ok
  hg init " "... ok
  hg init "with space"... ok

creating 'local/sub/repo'

  $ hg init local/sub/repo
  $ checknewrepo local/sub/repo
  store created
  00changelog.i created
  revlogv1
  fncache
  store
  dotencode

prepare test of init of url configured from paths

  $ echo '[paths]' >> $HGRCPATH
  $ echo "somewhere = `pwd`/url from paths" >> $HGRCPATH
  $ echo "elsewhere = `pwd`/another paths url" >> $HGRCPATH

init should (for consistency with clone) expand the url

  $ hg init somewhere
  $ checknewrepo "url from paths"
  store created
  00changelog.i created
  revlogv1
  fncache
  store
  dotencode

verify that clone also expand urls

  $ hg clone somewhere elsewhere
  updating to branch default
  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ checknewrepo "another paths url"
  store created
  00changelog.i created
  revlogv1
  fncache
  store
  dotencode

clone bookmarks

  $ hg -R local bookmark test
  $ hg -R local bookmarks
   * test                      0:08b9e9f63b32
  $ hg clone -e "python $TESTDIR/dummyssh" local ssh://user@dummy/remote-bookmarks
  searching for changes
  remote: adding changesets
  remote: adding manifests
  remote: adding file changes
  remote: added 1 changesets with 1 changes to 1 files
  $ hg -R remote-bookmarks bookmarks
     test                      0:08b9e9f63b32