view tests/test-rebase-partial.t @ 38732:be4984261611

merge: mark file gets as not thread safe (issue5933)

In default installs, this has the effect of disabling the thread-based
worker on Windows when manifesting files in the working directory. My
measurements have shown that with revlog-based repositories, Mercurial
spends a lot of CPU time in revlog code resolving file data. This ends
up incurring a lot of context switching across threads and slows down
`hg update` operations when going from an empty working directory to
the tip of the repo.

On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs):

before: 487s wall
after:  360s wall (equivalent to worker.enabled=false)
cpus=2: 379s wall

Even with only 2 threads, the thread pool is still slower.

The introduction of the thread-based worker (02b36e860e0b) states that
it resulted in a "~50%" speedup for `hg sparse --enable-profile` and
`hg sparse --disable-profile`. This disagrees with my measurement
above. I theorize a few reasons for this:

1) Removal of files from the working directory is I/O - not CPU - bound
   and should benefit from a thread pool (unless I/O is insanely fast
   and the GIL release is near instantaneous). So tests like `hg sparse
   --enable-profile` may exercise deletion throughput and aren't good
   benchmarks for worker tasks that are CPU heavy.
2) The patch was authored by someone at Facebook. The results were
   likely measured against a repository using remotefilelog. And I
   believe that revision retrieval during working directory updates with
   remotefilelog will often use a remote store, thus being I/O and not
   CPU bound. This probably resulted in an overstated performance gain.

Since there appears to be a need to enable the thread-based worker with
some stores, I've made the flagging of file gets as thread safe
configurable. I've made it experimental because I don't want to formalize
a boolean flag for this option and because this attribute is best
captured against the store implementation. But we don't have a proper
store API for this yet. I'd rather cross this bridge later.

It is possible there are revlog-based repositories that do benefit from
a thread-based worker. I didn't do very comprehensive testing. If there
are, we may want to devise a more proper algorithm for whether to use
the thread-based worker, including possibly config options to limit the
number of threads to use. But until I see evidence that justifies
complexity, simplicity wins.

Differential Revision: https://phab.mercurial-scm.org/D3963

Tests rebasing with part of the rebase set already in the
destination (issue5422)

  $ cat >> $HGRCPATH <<EOF
  > [extensions]
  > rebase=
  > drawdag=$TESTDIR/drawdag.py
  > 
  > [experimental]
  > evolution.createmarkers=True
  > evolution.allowunstable=True
  > 
  > [alias]
  > tglog = log -G --template "{rev}: {node|short} {desc}"
  > EOF

  $ rebasewithdag() {
  >   N=`$PYTHON -c "print($N+1)"`
  >   hg init repo$N && cd repo$N
  >   hg debugdrawdag
  >   hg rebase "$@" > _rebasetmp
  >   r=$?
  >   grep -v 'saved backup bundle' _rebasetmp
  >   [ $r -eq 0 ] && hg tglog
  >   cd ..
  >   return $r
  > }

Rebase two commits, of which one is already in the right place

  $ rebasewithdag -r C+D -d B <<EOF
  > C
  > |
  > B D
  > |/
  > A
  > EOF
  rebasing 2:b18e25de2cf5 "D" (D)
  already rebased 3:26805aba1e60 "C" (C tip)
  o  4: fe3b4c6498fa D
  |
  | o  3: 26805aba1e60 C
  |/
  | x  2: b18e25de2cf5 D
  | |
  o |  1: 112478962961 B
  |/
  o  0: 426bada5c675 A
  
Can collapse commits even if one is already in the right place

  $ rebasewithdag --collapse -r C+D -d B <<EOF
  > C
  > |
  > B D
  > |/
  > A
  > EOF
  rebasing 2:b18e25de2cf5 "D" (D)
  rebasing 3:26805aba1e60 "C" (C tip)
  o  4: a2493f4ace65 Collapsed revision
  |  * D
  |  * C
  | x  3: 26805aba1e60 C
  |/
  | x  2: b18e25de2cf5 D
  | |
  o |  1: 112478962961 B
  |/
  o  0: 426bada5c675 A
  
Abort doesn't lose the commits that were already in the right place

  $ hg init abort
  $ cd abort
  $ hg debugdrawdag <<EOF
  > C
  > |
  > B D  # B/file = B
  > |/   # D/file = D
  > A
  > EOF
  $ hg rebase -r C+D -d B
  rebasing 2:ef8c0fe0897b "D" (D)
  merging file
  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
  unresolved conflicts (see hg resolve, then hg rebase --continue)
  [1]
  $ hg rebase --abort
  rebase aborted
  $ hg tglog
  o  3: 79f6d6ab7b14 C
  |
  | o  2: ef8c0fe0897b D
  | |
  o |  1: 594087dbaf71 B
  |/
  o  0: 426bada5c675 A
  
  $ cd ..

Rebase with "holes". The commits after the hole should end up on the parent of
the hole (B below), not on top of the destination (A).

  $ rebasewithdag -r B+D -d A <<EOF
  > D
  > |
  > C
  > |
  > B
  > |
  > A
  > EOF
  already rebased 1:112478962961 "B" (B)
  rebasing 3:f585351a92f8 "D" (D tip)
  o  4: 1e6da8103bc7 D
  |
  | x  3: f585351a92f8 D
  | |
  | o  2: 26805aba1e60 C
  |/
  o  1: 112478962961 B
  |
  o  0: 426bada5c675 A