view tests/test-pull-branch.t @ 42469:4a3abb33380a

deltas: set estimated compression upper bound to "3x" instead of "10x"

In pratice, we very rarely observer compression better than "3x" on manifest
deltas. Having a more aggressive estimate significantly helps our pathological
use case on a private repository. Here are a comparison of timings using
different upper bound.

Estimated compression |    ø   |  ×10 |  ×5  |  ×3  |
timing                |  14.11 | 2.61 | 1.96 | 1.53 |


We also tested the impact of this series on an array of public repositories.
This shown no impact in either size nor timing.

Full data set below for those interested.

Size
----

Regarding size, not significant impact have been noticed on neither public nor
private repositories. Here are the number we gathered on public repositories:

zlib/upperbound | no            | 10x           | 5x            | 3x
mercurial       |     5 875 730 |     5 875 730 |     5 875 730 |     5 875 730
pypy            |    27 782 913 |    27 782 913 |    27 782 913 |    27 782 913
netbeans        |   159 161 207 |   159 161 207 |   159 161 207 |   159 959 879 (+0.5%)
mozilla-central |   323 841 642 |   323 841 642 |   323 841 642 |   319 867 519 (-2.5%)
mozilla-try     |   746 649 123 |   746 649 123 |   746 649 123 |   741 155 568 (-0.7%)
private-repo    | 1 485 287 294 | 1 485 287 294 | 1 485 287 294 | 1 409 248 382 (-5.1%)

zstd/upperbound | no            | 10x           | 5x            | 3x
mercurial       |     5 895 206 |     5 895 206 |     5 895 206 |     5 895 206
pypy            |    28 689 230 |    28 689 230 |    28 689 230 |    28 689 230
netbeans        |   157 636 387 |   157 636 387 |   157 636 387 |   159 692 678 (+1.3%)
mozilla-central |   317 650 281 |   317 650 281 |   317 650 281 |   319 613 603 (+0.6%)
mozilla-try     |   737 555 275 |   737 555 275 |   737 555 275 |   738 079 473 (+0.1%)
private-repo    | 1 352 362 982 | 1 352 362 982 | 1 346 961 880 | 1 361 327 384 (+0.7%)


Speed
------

Timing gathered using `hg perfrevlogwrite -m`. Value are in seconds.

mercurial

zlib   | no        | 10x       | 5x        | 3x        |
total  | 65.551783 | 65.388887 | 65.260658 | 65.321199 |
max    |  0.034544 |  0.034571 |  0.034659 |  0.034521 |
99.99% |  0.034544 |  0.034571 |  0.034659 |  0.034521 |

zstd   | no        | 10x       | 5x        | 3x        |
total  | 49.118449 | 49.054062 | 48.753588 | 48.740230 |
max    |  0.009338 |  0.009239 |  0.009202 |  0.009178 |
99.99% |  0.007618 |  0.007639 |  0.007626 |  0.007621 |

pypy

zlib   | no         | 10x        | 5x         | 3x         |
total  | 560.865984 | 558.983817 | 559.083815 | 559.349152 |
max    |   0.219614 |   0.215922 |   0.218112 |   0.218107 |
99.99% |   0.219614 |   0.215922 |   0.218112 |   0.218107 |

zstd   | no         | 10x        | 5x         | 3x         |
total  | 349.393280 | 347.395819 | 347.185407 | 345.643985 |
max    |   0.084143 |   0.083536 |   0.081834 |   0.082178 |
99.99% |   0.039445 |   0.039639 |   0.039612 |   0.039175 |

netbeans
zlib   | no           | 10x          | 5x           | 3x           |
total  | 33103.327727 | 33314.932260 | 33211.745233 | 33345.891778 |
max    |     2.666852 |     2.672059 |     2.662453 |     2.662936 |
99.99% |     2.058772 |     2.070429 |     2.069569 |     2.064653 |

zstd   | no           | 10x         | 5x            | 3x           |
total  | 20112.102708 | 20095.879719 | 20083.390300 | 20123.221859 |
max    |     2.063482 |     2.062851 |     2.065229 |     2.060147 |
99.99% |     1.146647 |     1.143794 |     1.142933 |     1.146529 |

mozilla
zlib   | no           | 10x          | 5x           | 3x           |
total  | 41374.102138 | 41418.816773 | 41381.956370 | 41334.280732 |
max    |     3.383474 |     3.387400 |     3.405711 |     3.387316 |
99.99% |     1.006755 |     1.005954 |     1.007700 |     1.007373 |

zstd   | no           | 10x          | 5x           | 3x           |
total  | 24689.691520 | 24643.939662 | 24664.630027 | 24664.512714 |
max    |     1.460822 |     1.449640 |     1.439747 |     1.465304 |
99.99% |     0.527111 |     0.527377 |     0.527807 |     0.527226 |

  $ hg init t
  $ cd t
  $ echo 1 > foo
  $ hg ci -Am1 # 0
  adding foo
  $ hg branch branchA
  marked working directory as branch branchA
  (branches are permanent and global, did you want a bookmark?)
  $ echo a1 > foo
  $ hg ci -ma1 # 1

  $ cd ..
  $ hg init tt
  $ cd tt
  $ hg pull ../t
  pulling from ../t
  requesting all changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files
  new changesets 495a0ec48aaf:50e089d141b7
  (run 'hg update' to get a working copy)
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cd ../t
  $ echo a2 > foo
  $ hg ci -ma2 # 2

Create branch B:

  $ hg up 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg branch branchB
  marked working directory as branch branchB
  $ echo b1 > foo
  $ hg ci -mb1 # 3

  $ cd ../tt

A new branch is there

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files (+1 heads)
  new changesets 9f878dea0b96:5be59ce5067b
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Develop both branches:

  $ cd ../t
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a3 > foo
  $ hg ci -ma3 # 4
  $ hg up branchB
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b2 > foo
  $ hg ci -mb2 # 5

  $ cd ../tt

Should succeed, no new heads:

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 1 files
  new changesets 7c8fe7e20c32:453e93fa00a5
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

Add a head on other branch:

  $ cd ../t
  $ hg up branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a4 > foo
  $ hg ci -ma4 # 6
  $ hg up branchB
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b3.1 > foo
  $ hg ci -m b3.1 # 7
  $ hg up 5
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b3.2 > foo
  $ hg ci -m b3.2 # 8
  created new head

  $ cd ../tt

Should succeed because there is only one head on our branch:

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 3 changesets with 3 changes to 1 files (+1 heads)
  new changesets da3a8a0161c6:b61cab8fe4e8
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

  $ cd ../t
  $ hg up -C branchA
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a5.1 > foo
  $ hg ci -ma5.1 # 9
  $ hg up 6
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a5.2 > foo
  $ hg ci -ma5.2 # 10
  created new head
  $ hg up 7
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b4.1 > foo
  $ hg ci -m b4.1 # 11
  $ hg up -C 8
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo b4.2 > foo
  $ hg ci -m b4.2 # 12

  $ cd ../tt

  $ hg pull -u ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 4 changesets with 4 changes to 1 files (+1 heads)
  new changesets 0c4d148ae29e:ecfc3f4a6fd9
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  updated to "d740e1a584e7: a5.2"
  1 other heads for branch "branchA"

Make changes on new branch on tt

  $ hg up 6
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ hg branch branchC
  marked working directory as branch branchC
  $ echo b1 > bar
  $ hg ci -Am "commit on branchC on tt"
  adding bar

Make changes on default branch on t

  $ cd ../t
  $ hg up -C default
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar
  $ hg ci -Am "commit on default on t"
  adding bar

Pull branchC from tt

  $ hg pull ../tt
  pulling from ../tt
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 1 changes to 1 files (+1 heads)
  new changesets 7d8ffa4c0b22
  13 local changesets published
  (run 'hg heads' to see heads)

Make changes on default and branchC on tt

  $ cd ../tt
  $ hg pull ../t
  pulling from ../t
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 1 changes to 1 files (+1 heads)
  new changesets 2b94b54b6b5f
  1 local changesets published
  (run 'hg heads' to see heads)
  $ hg up -C default
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar1
  $ hg ci -Am "commit on default on tt"
  adding bar1
  $ hg up branchC
  2 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo a1 > bar2
  $ hg ci -Am "commit on branchC on tt"
  adding bar2

Make changes on default and branchC on t

  $ cd ../t
  $ hg up default
  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo a1 > bar3
  $ hg ci -Am "commit on default on t"
  adding bar3
  $ hg up branchC
  2 files updated, 0 files merged, 1 files removed, 0 files unresolved
  $ echo a1 > bar4
  $ hg ci -Am "commit on branchC on tt"
  adding bar4

Pull from tt

  $ hg pull ../tt
  pulling from ../tt
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 2 changesets with 2 changes to 2 files (+2 heads)
  new changesets eed40c14b407:e634733b0309
  1 local changesets published
  (run 'hg heads .' to see heads, 'hg merge' to merge)

  $ cd ..