stream-clone: avoid opening a revlog in case we do not need it
Opening an revlog has a cost, especially if it is inline as we have to scan the
file and construct an index.
To prevent the associated slowdown, we just do a minimal scan to check that an
inline file is still inline, and simply stream the file without creating a
revlog when we can.
This provides a big boost compared to the previous changeset, even if the full
generation is still penalized by the initial gathering of information.
All benchmarks are run on linux with Python 3.10.7.
# benchmark.name = hg.exchange.stream.generate
# benchmark.variants.version = v2
### Compared to the previous changesets
We get a large win all across the board!
# mercurial-2018-08-01-zstd-sparse-revlog
before: 0.250694 seconds
after: 0.105986 seconds (-57.72%)
# pypy-2018-08-01-zstd-sparse-revlog
before: 3.885657 seconds
after: 1.709748 seconds (-56.00%)
# netbeans-2018-08-01-zstd-sparse-revlog
before: 16.679371 seconds
after: 7.687469 seconds (-53.91%)
# mozilla-central-2018-08-01-zstd-sparse-revlog
before: 38.575482 seconds
after: 17.520316 seconds (-54.58%)
# mozilla-try-2019-02-18-zstd-sparse-revlog
before: 81.160994 seconds
after: 37.073753 seconds (-54.32%)
### Compared to 6.4.3
We are still significantly slower than 6.4.3, the extra time is usually twice
slower than the extra time we observe on the locked section, which is a quite
interesting information.
Except for mercurial-central that is much faster. That discrepancy is not really
explained yet.
# mercurial-2018-08-01-zstd-sparse-revlog
6.4.3: 0.072560 seconds
after: 0.105986 seconds (+46.07%) (- 0.03 seconds)
# pypy-2018-08-01-zstd-sparse-revlog
6.4.3: 1.211193 seconds
after: 1.709748 seconds (+41.16%) (-0.45 seconds)
# netbeans-2018-08-01-zstd-sparse-revlog
6.4.3: 4.932843 seconds
after: 7.687469 seconds (+55.84%) (-2.75 seconds)
# mozilla-central-2018-08-01-zstd-sparse-revlog
6.4.3: 34.012226 seconds
after: 17.520316 seconds (-48.49%) (-16.49 seconds)
# mozilla-try-2019-02-18-zstd-sparse-revlog
6.4.3: 23.850555 seconds
after: 37.073753 seconds (+55.44%) (+13.22 seconds)
#!/bin/sh
# Script to get stable diff output on any platform.
#
# Output of this script is almost equivalent to GNU diff with "-Nru".
#
# Use this script as "hg pdiff" via extdiff extension with preparation
# below in test scripts:
#
# $ cat >> $HGRCPATH <<EOF
# > [extdiff]
# > pdiff = sh "$RUNTESTDIR/pdiff"
# > EOF
filediff(){
# USAGE: filediff file1 file2 [header]
# compare with /dev/null if file doesn't exist (as "-N" option)
file1="$1"
if test ! -f "$file1"; then
file1=/dev/null
fi
file2="$2"
if test ! -f "$file2"; then
file2=/dev/null
fi
if cmp -s "$file1" "$file2" 2> /dev/null; then
# Return immediately, because comparison isn't needed. This
# also avoids redundant message of diff like "No differences
# encountered" (on Solaris)
return
fi
if test -n "$3"; then
# show header only in recursive case
echo "$3"
fi
# replace "/dev/null" by corresponded filename (as "-N" option)
diff -u "$file1" "$file2" |
sed "s@^--- /dev/null\(.*\)\$@--- $1\1@" |
sed "s@^\+\+\+ /dev/null\(.*\)\$@+++ $2\1@"
# in this case, files differ from each other
return 1
}
if test -d "$1" -o -d "$2"; then
# ensure comparison in dictionary order
(
if test -d "$1"; then (cd "$1" && find . -type f); fi
if test -d "$2"; then (cd "$2" && find . -type f); fi
) |
sed 's@^\./@@g' | sort | uniq |
while read file; do
filediff "$1/$file" "$2/$file" "diff -Nru $1/$file $2/$file"
done
# TODO: there is no portable way for current while-read based
# implementation to return 1 at detecting changes.
#
# On bash and dash, assignment to variable inside while-block
# doesn't affect outside, because inside while-block is executed
# in sub-shell. BTW, it affects outside while-block on ksh (as sh
# on Solaris).
else
filediff "$1" "$2"
fi