view tests/test-convert-datesort.t @ 39270:37e56607cbb9

lfs: add a progress bar when searching for blobs to upload

The search itself can take an extreme amount of time if there are a lot of
revisions involved.  I've got a local repo that took 6 minutes to push 1850
commits, and 60% of that time was spent here (there are ~70K files):

     \ 58.1%  wrapper.py:     extractpointers      line 297:  pointers = extractpointers(...
       | 57.7%  wrapper.py:     pointersfromctx    line 352:  for p in pointersfromctx(ct...
       | 57.4%  wrapper.py:     pointerfromctx     line 397:  p = pointerfromctx(ctx, f, ...
         \ 38.7%  context.py:     __contains__     line 368:  if f not in ctx:
           | 38.7%  util.py:        __get__        line 82:  return key in self._manifest
           | 38.7%  context.py:     _manifest      line 1416:  result = self.func(obj)
           | 38.7%  manifest.py:    read           line 472:  return self._manifestctx.re...
             \ 25.6%  revlog.py:      revision     line 1562:  text = rl.revision(self._node)
               \ 12.8%  revlog.py:      _chunks    line 2217:  bins = self._chunks(chain, ...
                 | 12.0%  revlog.py:      decompressline 2112:  ladd(decomp(buffer(data, ch...
               \  7.8%  revlog.py:      checkhash  line 2232:  self.checkhash(text, node, ...
                 |  7.8%  revlog.py:      hash     line 2315:  if node != self.hash(text, ...
                 |  7.8%  revlog.py:      hash     line 2242:  return hash(text, p1, p2)
             \ 12.0%  manifest.py:    __init__     line 1565:  self._data = manifestdict(t...
         \ 16.8%  context.py:     filenode         line 378:  if not _islfs(fctx.filelog(...
           | 15.7%  util.py:        __get__        line 706:  return self._filelog
           | 14.8%  context.py:     _filelog       line 1416:  result = self.func(obj)
           | 14.8%  localrepo.py:   file           line 629:  return self._repo.file(self...
           | 14.8%  filelog.py:     __init__       line 1134:  return filelog.filelog(self...
           | 14.5%  revlog.py:      __init__       line 24:  censorable=True)

  $ cat >> $HGRCPATH <<EOF
  > [extensions]
  > convert=
  > EOF
  $ hg init t
  $ cd t
  $ echo a >> a
  $ hg ci -Am a0 -d '1 0'
  adding a
  $ hg branch brancha
  marked working directory as branch brancha
  (branches are permanent and global, did you want a bookmark?)
  $ echo a >> a
  $ hg ci -m a1 -d '2 0'
  $ echo a >> a
  $ hg ci -m a2 -d '3 0'
  $ echo a >> a
  $ hg ci -m a3 -d '4 0'
  $ hg up -C 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg branch branchb
  marked working directory as branch branchb
  $ echo b >> b
  $ hg ci -Am b0 -d '6 0'
  adding b
  $ hg up -C brancha
  1 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo a >> a
  $ hg ci -m a4 -d '5 0'
  $ echo a >> a
  $ hg ci -m a5 -d '7 0'
  $ echo a >> a
  $ hg ci -m a6 -d '8 0'
  $ hg up -C branchb
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b >> b
  $ hg ci -m b1 -d '9 0'
  $ hg up -C 0
  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo c >> c
  $ hg branch branchc
  marked working directory as branch branchc
  $ hg ci -Am c0 -d '10 0'
  adding c
  $ hg up -C brancha
  1 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ hg ci --close-branch -m a7x -d '11 0'
  $ hg up -C branchb
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg ci --close-branch -m b2x -d '12 0'
  $ hg up -C branchc
  1 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ hg merge branchb
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  (branch merge, don't forget to commit)
  $ hg ci -m c1 -d '13 0'
  $ cd ..

convert with datesort

  $ hg convert --datesort t t-datesort
  initializing destination t-datesort repository
  scanning source...
  sorting...
  converting...
  12 a0
  11 a1
  10 a2
  9 a3
  8 a4
  7 b0
  6 a5
  5 a6
  4 b1
  3 c0
  2 a7x
  1 b2x
  0 c1

graph converted repo

  $ hg -R t-datesort log -G --template '{rev} "{desc}"\n'
  o    12 "c1"
  |\
  | _  11 "b2x"
  | |
  | | _  10 "a7x"
  | | |
  o | |  9 "c0"
  | | |
  | o |  8 "b1"
  | | |
  | | o  7 "a6"
  | | |
  | | o  6 "a5"
  | | |
  | o |  5 "b0"
  |/ /
  | o  4 "a4"
  | |
  | o  3 "a3"
  | |
  | o  2 "a2"
  | |
  | o  1 "a1"
  |/
  o  0 "a0"
  

convert with datesort (default mode)

  $ hg convert t t-sourcesort
  initializing destination t-sourcesort repository
  scanning source...
  sorting...
  converting...
  12 a0
  11 a1
  10 a2
  9 a3
  8 b0
  7 a4
  6 a5
  5 a6
  4 b1
  3 c0
  2 a7x
  1 b2x
  0 c1

graph converted repo

  $ hg -R t-sourcesort log -G --template '{rev} "{desc}"\n'
  o    12 "c1"
  |\
  | _  11 "b2x"
  | |
  | | _  10 "a7x"
  | | |
  o | |  9 "c0"
  | | |
  | o |  8 "b1"
  | | |
  | | o  7 "a6"
  | | |
  | | o  6 "a5"
  | | |
  | | o  5 "a4"
  | | |
  | o |  4 "b0"
  |/ /
  | o  3 "a3"
  | |
  | o  2 "a2"
  | |
  | o  1 "a1"
  |/
  o  0 "a0"
  

convert with closesort

  $ hg convert --closesort t t-closesort
  initializing destination t-closesort repository
  scanning source...
  sorting...
  converting...
  12 a0
  11 a1
  10 a2
  9 a3
  8 b0
  7 a4
  6 a5
  5 a6
  4 a7x
  3 b1
  2 b2x
  1 c0
  0 c1

graph converted repo

  $ hg -R t-closesort log -G --template '{rev} "{desc}"\n'
  o    12 "c1"
  |\
  | o  11 "c0"
  | |
  _ |  10 "b2x"
  | |
  o |  9 "b1"
  | |
  | | _  8 "a7x"
  | | |
  | | o  7 "a6"
  | | |
  | | o  6 "a5"
  | | |
  | | o  5 "a4"
  | | |
  o | |  4 "b0"
  |/ /
  | o  3 "a3"
  | |
  | o  2 "a2"
  | |
  | o  1 "a1"
  |/
  o  0 "a0"