Mercurial > hg
view tests/test-bundle-phases.t @ 38732:be4984261611
merge: mark file gets as not thread safe (issue5933)
In default installs, this has the effect of disabling the thread-based
worker on Windows when manifesting files in the working directory. My
measurements have shown that with revlog-based repositories, Mercurial
spends a lot of CPU time in revlog code resolving file data. This ends
up incurring a lot of context switching across threads and slows down
`hg update` operations when going from an empty working directory to
the tip of the repo.
On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs):
before: 487s wall
after: 360s wall (equivalent to worker.enabled=false)
cpus=2: 379s wall
Even with only 2 threads, the thread pool is still slower.
The introduction of the thread-based worker (02b36e860e0b) states that
it resulted in a "~50%" speedup for `hg sparse --enable-profile` and
`hg sparse --disable-profile`. This disagrees with my measurement
above. I theorize a few reasons for this:
1) Removal of files from the working directory is I/O - not CPU - bound
and should benefit from a thread pool (unless I/O is insanely fast
and the GIL release is near instantaneous). So tests like `hg sparse
--enable-profile` may exercise deletion throughput and aren't good
benchmarks for worker tasks that are CPU heavy.
2) The patch was authored by someone at Facebook. The results were
likely measured against a repository using remotefilelog. And I
believe that revision retrieval during working directory updates with
remotefilelog will often use a remote store, thus being I/O and not
CPU bound. This probably resulted in an overstated performance gain.
Since there appears to be a need to enable the thread-based worker with
some stores, I've made the flagging of file gets as thread safe
configurable. I've made it experimental because I don't want to formalize
a boolean flag for this option and because this attribute is best
captured against the store implementation. But we don't have a proper
store API for this yet. I'd rather cross this bridge later.
It is possible there are revlog-based repositories that do benefit from
a thread-based worker. I didn't do very comprehensive testing. If there
are, we may want to devise a more proper algorithm for whether to use
the thread-based worker, including possibly config options to limit the
number of threads to use. But until I see evidence that justifies
complexity, simplicity wins.
Differential Revision: https://phab.mercurial-scm.org/D3963
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 18 Jul 2018 09:49:34 -0700 |
parents | 326b174c6a47 |
children | 197204dba8a2 |
line wrap: on
line source
$ cat >> $HGRCPATH <<EOF > [experimental] > bundle-phases=yes > [extensions] > strip= > drawdag=$TESTDIR/drawdag.py > EOF Set up repo with linear history $ hg init linear $ cd linear $ hg debugdrawdag <<'EOF' > E > | > D > | > C > | > B > | > A > EOF $ hg phase --public A $ hg phase --force --secret D $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C draft | o B draft | o A public Phases are restored when unbundling $ hg bundle --base B -r E bundle 3 changesets found $ hg debugbundle bundle Stream params: {Compression: BZ} changegroup -- {nbchanges: 3, targetphase: 2, version: 02} (mandatory: True) 26805aba1e600a82e93661149f2313866a221a7b f585351a92f85104bff7c284233c338b10eb1df7 9bc730a19041f9ec7cb33c626e811aa233efb18c cache:rev-branch-cache -- {} (mandatory: False) phase-heads -- {} (mandatory: True) 26805aba1e600a82e93661149f2313866a221a7b draft $ hg strip --no-backup C $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C draft | o B draft | o A public Root revision's phase is preserved $ hg bundle -a bundle 5 changesets found $ hg strip --no-backup A $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C draft | o B draft | o A public Completely public history can be restored $ hg phase --public E $ hg bundle -a bundle 5 changesets found $ hg strip --no-backup A $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E public | o D public | o C public | o B public | o A public Direct transition from public to secret can be restored $ hg phase --secret --force D $ hg bundle -a bundle 5 changesets found $ hg strip --no-backup A $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C public | o B public | o A public Revisions within bundle preserve their phase even if parent changes its phase $ hg phase --draft --force B $ hg bundle --base B -r E bundle 3 changesets found $ hg strip --no-backup C $ hg phase --public B $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C draft | o B public | o A public Phase of ancestors of stripped node get advanced to accommodate child $ hg bundle --base B -r E bundle 3 changesets found $ hg strip --no-backup C $ hg phase --force --secret B $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E secret | o D secret | o C draft | o B draft | o A public Unbundling advances phases of changesets even if they were already in the repo. To test that, create a bundle of everything in draft phase and then unbundle to see that secret becomes draft, but public remains public. $ hg phase --draft --force A $ hg phase --draft E $ hg bundle -a bundle 5 changesets found $ hg phase --public A $ hg phase --secret --force E $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o E draft | o D draft | o C draft | o B draft | o A public Unbundling change in the middle of a stack does not affect later changes $ hg strip --no-backup E $ hg phase --secret --force D $ hg log -G -T '{desc} {phase}\n' o D secret | o C draft | o B draft | o A public $ hg bundle --base A -r B bundle 1 changesets found $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{desc} {phase}\n' o D secret | o C draft | o B draft | o A public $ cd .. Set up repo with non-linear history $ hg init non-linear $ cd non-linear $ hg debugdrawdag <<'EOF' > D E > |\| > B C > |/ > A > EOF $ hg phase --public C $ hg phase --force --secret B $ hg log -G -T '{node|short} {desc} {phase}\n' o 03ca77807e91 E draft | | o 4e4f9194f9f1 D secret |/| o | dc0947a82db8 C public | | | o 112478962961 B secret |/ o 426bada5c675 A public Restore bundle of entire repo $ hg bundle -a bundle 5 changesets found $ hg debugbundle bundle Stream params: {Compression: BZ} changegroup -- {nbchanges: 5, targetphase: 2, version: 02} (mandatory: True) 426bada5c67598ca65036d57d9e4b64b0c1ce7a0 112478962961147124edd43549aedd1a335e44bf dc0947a82db884575bb76ea10ac97b08536bfa03 4e4f9194f9f181c57f62e823e8bdfa46ab9e4ff4 03ca77807e919db8807c3749086dc36fb478cac0 cache:rev-branch-cache -- {} (mandatory: False) phase-heads -- {} (mandatory: True) dc0947a82db884575bb76ea10ac97b08536bfa03 public 03ca77807e919db8807c3749086dc36fb478cac0 draft $ hg strip --no-backup A $ hg unbundle -q bundle $ rm bundle $ hg log -G -T '{node|short} {desc} {phase}\n' o 03ca77807e91 E draft | | o 4e4f9194f9f1 D secret |/| o | dc0947a82db8 C public | | | o 112478962961 B secret |/ o 426bada5c675 A public $ hg bundle --base 'A + C' -r D bundle 2 changesets found $ hg debugbundle bundle Stream params: {Compression: BZ} changegroup -- {nbchanges: 2, targetphase: 2, version: 02} (mandatory: True) 112478962961147124edd43549aedd1a335e44bf 4e4f9194f9f181c57f62e823e8bdfa46ab9e4ff4 cache:rev-branch-cache -- {} (mandatory: False) phase-heads -- {} (mandatory: True) $ rm bundle $ hg bundle --base A -r D bundle 3 changesets found $ hg debugbundle bundle Stream params: {Compression: BZ} changegroup -- {nbchanges: 3, targetphase: 2, version: 02} (mandatory: True) 112478962961147124edd43549aedd1a335e44bf dc0947a82db884575bb76ea10ac97b08536bfa03 4e4f9194f9f181c57f62e823e8bdfa46ab9e4ff4 cache:rev-branch-cache -- {} (mandatory: False) phase-heads -- {} (mandatory: True) dc0947a82db884575bb76ea10ac97b08536bfa03 public $ rm bundle $ hg bundle --base 'B + C' -r 'D + E' bundle 2 changesets found $ hg debugbundle bundle Stream params: {Compression: BZ} changegroup -- {nbchanges: 2, targetphase: 2, version: 02} (mandatory: True) 4e4f9194f9f181c57f62e823e8bdfa46ab9e4ff4 03ca77807e919db8807c3749086dc36fb478cac0 cache:rev-branch-cache -- {} (mandatory: False) phase-heads -- {} (mandatory: True) 03ca77807e919db8807c3749086dc36fb478cac0 draft $ rm bundle