Mercurial > hg
view tests/test-diff-ignore-whitespace.t @ 38732:be4984261611
merge: mark file gets as not thread safe (issue5933)
In default installs, this has the effect of disabling the thread-based
worker on Windows when manifesting files in the working directory. My
measurements have shown that with revlog-based repositories, Mercurial
spends a lot of CPU time in revlog code resolving file data. This ends
up incurring a lot of context switching across threads and slows down
`hg update` operations when going from an empty working directory to
the tip of the repo.
On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs):
before: 487s wall
after: 360s wall (equivalent to worker.enabled=false)
cpus=2: 379s wall
Even with only 2 threads, the thread pool is still slower.
The introduction of the thread-based worker (02b36e860e0b) states that
it resulted in a "~50%" speedup for `hg sparse --enable-profile` and
`hg sparse --disable-profile`. This disagrees with my measurement
above. I theorize a few reasons for this:
1) Removal of files from the working directory is I/O - not CPU - bound
and should benefit from a thread pool (unless I/O is insanely fast
and the GIL release is near instantaneous). So tests like `hg sparse
--enable-profile` may exercise deletion throughput and aren't good
benchmarks for worker tasks that are CPU heavy.
2) The patch was authored by someone at Facebook. The results were
likely measured against a repository using remotefilelog. And I
believe that revision retrieval during working directory updates with
remotefilelog will often use a remote store, thus being I/O and not
CPU bound. This probably resulted in an overstated performance gain.
Since there appears to be a need to enable the thread-based worker with
some stores, I've made the flagging of file gets as thread safe
configurable. I've made it experimental because I don't want to formalize
a boolean flag for this option and because this attribute is best
captured against the store implementation. But we don't have a proper
store API for this yet. I'd rather cross this bridge later.
It is possible there are revlog-based repositories that do benefit from
a thread-based worker. I didn't do very comprehensive testing. If there
are, we may want to devise a more proper algorithm for whether to use
the thread-based worker, including possibly config options to limit the
number of threads to use. But until I see evidence that justifies
complexity, simplicity wins.
Differential Revision: https://phab.mercurial-scm.org/D3963
| author | Gregory Szorc <gregory.szorc@gmail.com> |
|---|---|
| date | Wed, 18 Jul 2018 09:49:34 -0700 |
| parents | da07367d683b |
| children | 55c6ebd11cb9 |
line wrap: on
line source
GNU diff is the reference for all of these results. Prepare tests: $ echo '[alias]' >> $HGRCPATH $ echo 'ndiff = diff --nodates' >> $HGRCPATH $ hg init $ printf 'hello world\ngoodbye world\n' >foo $ hg ci -Amfoo -ufoo adding foo Test added blank lines: $ printf '\nhello world\n\ngoodbye world\n\n' >foo >>> two diffs showing three added lines <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ + hello world + goodbye world + $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ + hello world + goodbye world + >>> no diffs <<< $ hg ndiff -B $ hg ndiff -Bb Test added horizontal space first on a line(): $ printf '\t hello world\ngoodbye world\n' >foo >>> four diffs showing added space first on the first line <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world + hello world goodbye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world + hello world goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world + hello world goodbye world $ hg ndiff -Bb diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world + hello world goodbye world Test added horizontal space last on a line: $ printf 'hello world\t \ngoodbye world\n' >foo >>> two diffs showing space appended to the first line <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world +hello world goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world +hello world goodbye world >>> no diffs <<< $ hg ndiff -b $ hg ndiff -Bb Test added horizontal space in the middle of a word: $ printf 'hello world\ngood bye world\n' >foo >>> four diffs showing space inserted into "goodbye" <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +good bye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +good bye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +good bye world $ hg ndiff -Bb diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +good bye world Test increased horizontal whitespace amount: $ printf 'hello world\ngoodbye\t\t \tworld\n' >foo >>> two diffs showing changed whitespace amount in the last line <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ hello world -goodbye world +goodbye world >>> no diffs <<< $ hg ndiff -b $ hg ndiff -Bb Test added blank line with horizontal whitespace: $ printf 'hello world\n \t\ngoodbye world\n' >foo >>> three diffs showing added blank line with horizontal space <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world + goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world + goodbye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world + goodbye world >>> no diffs <<< $ hg ndiff -Bb Test added blank line with other whitespace: $ printf 'hello world\n \t\ngoodbye world \n' >foo >>> three diffs showing added blank line with other space <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ -hello world -goodbye world +hello world + +goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ -hello world -goodbye world +hello world + +goodbye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world + goodbye world >>> no diffs <<< $ hg ndiff -Bb Test whitespace changes: $ printf 'helloworld\ngoodbye\tworld \n' >foo >>> four diffs showing changed whitespace <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world -goodbye world +helloworld +goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world -goodbye world +helloworld +goodbye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world +helloworld goodbye world $ hg ndiff -Bb diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,2 @@ -hello world +helloworld goodbye world >>> no diffs <<< $ hg ndiff -w Test whitespace changes and blank lines: $ printf 'helloworld\n\n\n\ngoodbye\tworld \n' >foo >>> five diffs showing changed whitespace <<< $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ -hello world -goodbye world +helloworld + + + +goodbye world $ hg ndiff -B diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ -hello world -goodbye world +helloworld + + + +goodbye world $ hg ndiff -b diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ -hello world +helloworld + + + goodbye world $ hg ndiff -Bb diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ -hello world +helloworld + + + goodbye world $ hg ndiff -w diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,5 @@ hello world + + + goodbye world >>> no diffs <<< $ hg ndiff -wB Test \r (carriage return) as used in "DOS" line endings: $ printf 'hello world\r\n\r\ngoodbye\rworld\n' >foo $ hg ndiff diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ -hello world -goodbye world +hello world\r (esc) +\r (esc) +goodbye\r (no-eol) (esc) world Test \r (carriage return) as used in "DOS" line endings: $ printf 'hello world \r\n\t\ngoodbye world\n' >foo $ hg ndiff --ignore-space-at-eol diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world +\t (esc) goodbye world No completely blank lines to ignore: $ printf 'hello world\r\n\r\ngoodbye\rworld\n' >foo $ hg ndiff --ignore-blank-lines diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ -hello world -goodbye world +hello world\r (esc) +\r (esc) +goodbye\r (no-eol) (esc) world Only new line noticed: $ hg ndiff --ignore-space-change diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world +\r (esc) goodbye world $ hg ndiff --ignore-all-space diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ hello world +\r (esc) goodbye world New line not noticed when space change ignored: $ hg ndiff --ignore-blank-lines --ignore-all-space Do not ignore all newlines, only blank lines $ printf 'hello \nworld\ngoodbye world\n' > foo $ hg ndiff --ignore-blank-lines diff -r 540c40a65b78 foo --- a/foo +++ b/foo @@ -1,2 +1,3 @@ -hello world +hello +world goodbye world Test hunk offsets adjustments with --ignore-blank-lines $ hg revert -aC reverting foo $ printf '\nb\nx\nd\n' > a $ printf 'b\ny\nd\n' > b $ hg add a b $ hg ci -m add $ hg cat -r . a > b $ hg cat -r . b > a $ hg diff -B --nodates a > ../diffa $ cat ../diffa diff -r 0e66aa54f318 a --- a/a +++ b/a @@ -1,4 +1,4 @@ b -x +y d $ hg diff -B --nodates b > ../diffb $ cat ../diffb diff -r 0e66aa54f318 b --- a/b +++ b/b @@ -1,3 +1,3 @@ b -y +x d $ hg revert -aC reverting a reverting b $ hg import --no-commit ../diffa applying ../diffa $ hg revert -aC reverting a $ hg import --no-commit ../diffb applying ../diffb $ hg revert -aC reverting b