view tests/test-pull-branch.t @ 34824:e2ad93bcc084

revlog: introduce an experimental flag to slice chunks reads when too sparse

Delta chains can become quite sparse if there is a lot of unrelated data between
relevant pieces. Right now, revlog always reads all the necessary data for the
delta chain in one single read. This can lead to a lot of unrelated data to be
read (see issue5482 for more details).
One can use the `experimental.maxdeltachainspan` option with a large value
(or -1) to easily produce a very sparse delta chain.

This change introduces the ability to slice the chunks retrieval into  multiple
reads, skipping large sections of unrelated data. Preliminary testing shows
interesting results. For example the peak memory consumption to read a manifest
on a large repository is reduced from 600MB to 250MB (200MB without
maxdeltachainspan). However, the slicing itself and the multiple reads can have
an negative impact on performance. This is why the new feature is hidden behind
an experimental flag.

Future changesets will add various parameters to control the slicing heuristics.
We hope to experiment a wide variety of repositories during 4.4 and hopefully
turn the feature on by default in 4.5.
As a first try, the algorithm itself is prone to deep changes. However, we wish
to define APIs and have a baseline to work on.

  $ hg init t
  $ cd t
  $ echo 1 > foo
  $ hg ci -Am1 # 0
  adding foo
  $ hg branch branchA
  marked working directory as branch branchA
  (branches are permanent and global, did you want a bookmark?)
  $ echo a1 > foo
  $ hg ci -ma1 # 1

  $ cd ..
  $ hg init tt
  $ cd tt
  $ hg pull ../t
  pulling from ../t
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files
  new changesets 495a0ec48aaf:50e089d141b7
  (run 'hg update' to get a working copy)
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cd ../t
  $ echo a2 > foo
  $ hg ci -ma2 # 2

Create branch B:

  $ hg up 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg branch branchB
  marked working directory as branch branchB
  $ echo b1 > foo
  $ hg ci -mb1 # 3

  $ cd ../tt

A new branch is there

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files (+1 heads)
  new changesets 9f878dea0b96:5be59ce5067b
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Develop both branches:

  $ cd ../t
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a3 > foo
  $ hg ci -ma3 # 4
  $ hg up branchB
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b2 > foo
  $ hg ci -mb2 # 5

  $ cd ../tt

Should succeed, no new heads:

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files
  new changesets 7c8fe7e20c32:453e93fa00a5
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Add a head on other branch:

  $ cd ../t
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a4 > foo
  $ hg ci -ma4 # 6
  $ hg up branchB
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b3.1 > foo
  $ hg ci -m b3.1 # 7
  $ hg up 5
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b3.2 > foo
  $ hg ci -m b3.2 # 8
  created new head

  $ cd ../tt

Should succeed because there is only one head on our branch:

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 3 changesets with 3 changes to 1 files (+1 heads)
  new changesets da3a8a0161c6:b61cab8fe4e8
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cd ../t
  $ hg up -C branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a5.1 > foo
  $ hg ci -ma5.1 # 9
  $ hg up 6
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a5.2 > foo
  $ hg ci -ma5.2 # 10
  created new head
  $ hg up 7
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b4.1 > foo
  $ hg ci -m b4.1 # 11
  $ hg up -C 8
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b4.2 > foo
  $ hg ci -m b4.2 # 12

  $ cd ../tt

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 4 changesets with 4 changes to 1 files (+1 heads)
  new changesets 0c4d148ae29e:ecfc3f4a6fd9
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  updated to "d740e1a584e7: a5.2"
  1 other heads for branch "branchA"

Make changes on new branch on tt

  $ hg up 6
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg branch branchC
  marked working directory as branch branchC
  $ echo b1 > bar
  $ hg ci -Am "commit on branchC on tt"
  adding bar

Make changes on default branch on t

  $ cd ../t
  $ hg up -C default
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar
  $ hg ci -Am "commit on default on t"
  adding bar

Pull branchC from tt

  $ hg pull ../tt
  pulling from ../tt
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 1 changes to 1 files (+1 heads)
  new changesets 7d8ffa4c0b22
  (run 'hg heads' to see heads)

Make changes on default and branchC on tt

  $ cd ../tt
  $ hg pull ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 1 changes to 1 files (+1 heads)
  new changesets 2b94b54b6b5f
  (run 'hg heads' to see heads)
  $ hg up -C default
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar1
  $ hg ci -Am "commit on default on tt"
  adding bar1
  $ hg up branchC
  2 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo a1 > bar2
  $ hg ci -Am "commit on branchC on tt"
  adding bar2

Make changes on default and branchC on t

  $ cd ../t
  $ hg up default
  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar3
  $ hg ci -Am "commit on default on t"
  adding bar3
  $ hg up branchC
  2 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo a1 > bar4
  $ hg ci -Am "commit on branchC on tt"
  adding bar4

Pull from tt

  $ hg pull ../tt
  pulling from ../tt
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 2 files (+2 heads)
  new changesets eed40c14b407:e634733b0309
  (run 'hg heads .' to see heads, 'hg merge' to merge)

  $ cd ..