Mercurial > hg
view tests/test-diffstat.t @ 38732:be4984261611
merge: mark file gets as not thread safe (issue5933)
In default installs, this has the effect of disabling the thread-based
worker on Windows when manifesting files in the working directory. My
measurements have shown that with revlog-based repositories, Mercurial
spends a lot of CPU time in revlog code resolving file data. This ends
up incurring a lot of context switching across threads and slows down
`hg update` operations when going from an empty working directory to
the tip of the repo.
On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs):
before: 487s wall
after: 360s wall (equivalent to worker.enabled=false)
cpus=2: 379s wall
Even with only 2 threads, the thread pool is still slower.
The introduction of the thread-based worker (02b36e860e0b) states that
it resulted in a "~50%" speedup for `hg sparse --enable-profile` and
`hg sparse --disable-profile`. This disagrees with my measurement
above. I theorize a few reasons for this:
1) Removal of files from the working directory is I/O - not CPU - bound
and should benefit from a thread pool (unless I/O is insanely fast
and the GIL release is near instantaneous). So tests like `hg sparse
--enable-profile` may exercise deletion throughput and aren't good
benchmarks for worker tasks that are CPU heavy.
2) The patch was authored by someone at Facebook. The results were
likely measured against a repository using remotefilelog. And I
believe that revision retrieval during working directory updates with
remotefilelog will often use a remote store, thus being I/O and not
CPU bound. This probably resulted in an overstated performance gain.
Since there appears to be a need to enable the thread-based worker with
some stores, I've made the flagging of file gets as thread safe
configurable. I've made it experimental because I don't want to formalize
a boolean flag for this option and because this attribute is best
captured against the store implementation. But we don't have a proper
store API for this yet. I'd rather cross this bridge later.
It is possible there are revlog-based repositories that do benefit from
a thread-based worker. I didn't do very comprehensive testing. If there
are, we may want to devise a more proper algorithm for whether to use
the thread-based worker, including possibly config options to limit the
number of threads to use. But until I see evidence that justifies
complexity, simplicity wins.
Differential Revision: https://phab.mercurial-scm.org/D3963
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 18 Jul 2018 09:49:34 -0700 |
parents | 87c4253bebdb |
children | f1b0d9988825 |
line wrap: on
line source
$ hg init repo $ cd repo $ i=0; while [ "$i" -lt 213 ]; do echo a >> a; i=`expr $i + 1`; done $ hg add a $ cp a b $ hg add b Wide diffstat: $ hg diff --stat a | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ b | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 426 insertions(+), 0 deletions(-) diffstat width: $ COLUMNS=24 hg diff --config ui.interactive=true --stat a | 213 ++++++++++++++ b | 213 ++++++++++++++ 2 files changed, 426 insertions(+), 0 deletions(-) $ hg ci -m adda $ cat >> a <<EOF > a > a > a > EOF Narrow diffstat: $ hg diff --stat a | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) $ hg ci -m appenda >>> open("c", "wb").write(b"\0") and None $ touch d $ hg add c d Binary diffstat: $ hg diff --stat c | Bin 1 files changed, 0 insertions(+), 0 deletions(-) Binary git diffstat: $ hg diff --stat --git c | Bin d | 0 2 files changed, 0 insertions(+), 0 deletions(-) $ hg ci -m createb >>> open("file with spaces", "wb").write(b"\0") and None $ hg add "file with spaces" Filename with spaces diffstat: $ hg diff --stat file with spaces | Bin 1 files changed, 0 insertions(+), 0 deletions(-) Filename with spaces git diffstat: $ hg diff --stat --git file with spaces | Bin 1 files changed, 0 insertions(+), 0 deletions(-) Filename without "a/" or "b/" (issue5759): $ hg diff --config 'diff.noprefix=1' -c1 --stat --git a | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) $ hg diff --config 'diff.noprefix=1' -c2 --stat --git c | Bin d | 0 2 files changed, 0 insertions(+), 0 deletions(-) $ hg log --config 'diff.noprefix=1' -r '1:' -p --stat --git changeset: 1:3a95b07bb77f user: test date: Thu Jan 01 00:00:00 1970 +0000 summary: appenda a | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a a --- a +++ a @@ -211,3 +211,6 @@ a a a +a +a +a changeset: 2:c60a6c753773 tag: tip user: test date: Thu Jan 01 00:00:00 1970 +0000 summary: createb c | Bin d | 0 2 files changed, 0 insertions(+), 0 deletions(-) diff --git c c new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..f76dd238ade08917e6712764a16a22005a50573d GIT binary patch literal 1 Ic${MZ000310RR91 diff --git d d new file mode 100644 diffstat within directories: $ hg rm -f 'file with spaces' $ mkdir dir1 dir2 $ echo new1 > dir1/new $ echo new2 > dir2/new $ hg add dir1/new dir2/new $ hg diff --stat dir1/new | 1 + dir2/new | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) $ hg diff --stat --root dir1 new | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) $ hg diff --stat --root dir1 dir2 warning: dir2 not inside relative root dir1 $ hg diff --stat --root dir1 -I dir1/old $ cd dir1 $ hg diff --stat . dir1/new | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) $ hg diff --stat --root . new | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) $ hg diff --stat --root ../dir1 ../dir2 warning: ../dir2 not inside relative root . $ hg diff --stat --root . -I old $ cd .. Files with lines beginning with '--' or '++' should be properly counted in diffstat $ hg up -Cr tip 0 files updated, 0 files merged, 0 files removed, 0 files unresolved $ rm dir1/new $ rm dir2/new $ rm "file with spaces" $ cat > file << EOF > line 1 > line 2 > line 3 > EOF $ hg commit -Am file adding file Lines added starting with '--' should count as additions $ cat > file << EOF > line 1 > -- line 2, with dashes > line 3 > EOF $ hg diff --root . diff -r be1569354b24 file --- a/file Thu Jan 01 00:00:00 1970 +0000 +++ b/file * (glob) @@ -1,3 +1,3 @@ line 1 -line 2 +-- line 2, with dashes line 3 $ hg diff --root . --stat file | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) Lines changed starting with '--' should count as deletions $ hg commit -m filev2 $ cat > file << EOF > line 1 > -- line 2, with dashes, changed again > line 3 > EOF $ hg diff --root . diff -r 160f7c034df6 file --- a/file Thu Jan 01 00:00:00 1970 +0000 +++ b/file * (glob) @@ -1,3 +1,3 @@ line 1 --- line 2, with dashes +-- line 2, with dashes, changed again line 3 $ hg diff --root . --stat file | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) Lines changed starting with '--' should count as deletions and starting with '++' should count as additions $ cat > file << EOF > line 1 > ++ line 2, switched dashes to plusses > line 3 > EOF $ hg diff --root . diff -r 160f7c034df6 file --- a/file Thu Jan 01 00:00:00 1970 +0000 +++ b/file * (glob) @@ -1,3 +1,3 @@ line 1 --- line 2, with dashes +++ line 2, switched dashes to plusses line 3 $ hg diff --root . --stat file | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)