Mercurial > hg
view tests/test-add.t @ 49658:523cacdfd324
delta-find: set the default candidate chunk size to 10
I ran performance and storage tests on repositories of various sizes and shapes
for the following values of the config : 5, 10, 20, 50, 100, no-chunking
The performance tests do not show any statistical impact on computation
times for large pushes and pulls.
For searching for an individual delta, this can provide a significant
performance improvement with a minor degradation of space-quality on the
result. (see data at the end of the commit).
For overall store size, the change :
- does not have any impact on many small repositories,
- has an observable, but very negligible impact on most larger repositories.
- One private repository we use for testing sees a small increase in size
(1%) in the narrower version.
We will try to get more numbers on a larger version of that repository to
make sure nothing pathological happens.
We pick "10" as the limit as "5" seems a bit more risky.
There are room to improve the current code, by using more aggressive filtering
and better (i.e any) sorting of the candidates. However this is already a large
improvement for pathological cases, with little impact in the common
situations.
The initial motivation for this change is to fix performance of delta
computation for a file where the previous code ended up testing 20 000 possible
candidate-bases in one go, which is… slow. This affected about ½ of the file
revisions leading to atrocious performance, especially during some push/pull
operations.
Details about individual delta finding timing:
----------------------------------------------
The vast majority of benchmark cases are unchanged but the three below. The first
two do not see any impact on the final delta. The last one sees a change in
delta-size that is negligible compared to the full text size.
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096)
∞: 5.844783
5: 4.473523 (-23.46%)
10: 4.970053 (-14.97%)
20: 5.770386 (-1.27%)
50 5.821358
100: 5.834887
MANIFESTLOG: rev = 756096: (no-limit)
delta-base = 301840
search-rounds = 6
try-count = 60
delta-type = snapshot
snap-depth = 7
delta-size = 179
MANIFESTLOG: rev=756096: (limit = 10)
delta-base=301840
search-rounds=9
try-count=51
delta-type=snapshot
snap-depth=7
delta-size=179
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060)
∞: 5.017663
5: 3.655931 (-27.14%)
10: 4.095436 (-18.38%)
20: 4.828949 (-3.76%)
50 4.987574
100: 4.994889
MANIFESTLOG: rev=754060: (no limit)
delta-base=301840
search-rounds=5
try-count=53
delta-type=snapshot
snap-depth=7
delta-size = 179
MANIFESTLOG: rev=754060: (limite = 10)
delta-base=301840
search-rounds=8
try-count=45
delta-type=snapshot
snap-depth=7
delta-size = 179
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# bin-env-vars.hg.flavor = rust
# benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368)
∞: 4.869282
5: 2.039732 (-58.11%)
10: 2.413537 (-50.43%)
20: 4.449639 (-8.62%)
50 4.865863
100: 4.882649
MANIFESTLOG: rev=693368:
delta-base=693336
search-rounds=6
try-count=53
delta-type=snapshot
snap-depth=6
full-test-size=131065
delta-size=199
MANIFESTLOG: rev=693368:
delta-base=278023
search-rounds=5
try-count=21
delta-type=snapshot
snap-depth=4
full-test-size=131065
delta-size=278
Raw data for store size (in bytes) for various chunk size value below:
----------------------------------------------------------------------
440 134 384 5 pypy/.hg/store/
440 134 384 10 pypy/.hg/store/
440 134 384 20 pypy/.hg/store/
440 134 384 50 pypy/.hg/store/
440 134 384 100 pypy/.hg/store/
440 134 384 ... pypy/.hg/store/
666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/
852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/
1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/
3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/
3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/
3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/
4 531 441 314 5 mozilla-central/.hg/store/
4 531 435 157 10 mozilla-central/.hg/store/
4 531 432 045 20 mozilla-central/.hg/store/
4 531 429 119 50 mozilla-central/.hg/store/
4 531 429 119 100 mozilla-central/.hg/store/
4 531 429 119 ... mozilla-central/.hg/store/
4 875 861 390 5 mozilla-unified/.hg/store/
4 875 855 155 10 mozilla-unified/.hg/store/
4 875 852 027 20 mozilla-unified/.hg/store/
4 875 848 851 50 mozilla-unified/.hg/store/
4 875 848 851 100 mozilla-unified/.hg/store/
4 875 848 851 ... mozilla-unified/.hg/store/
11 498 764 601 5 mozilla-try/.hg/store/
11 497 968 858 10 mozilla-try/.hg/store/
11 497 958 730 20 mozilla-try/.hg/store/
11 497 927 156 50 mozilla-try/.hg/store/
11 497 925 963 100 mozilla-try/.hg/store/
11 497 923 428 ... mozilla-try/.hg/store/
10 047 914 031 5 private-repo
9 969 132 101 10 private-repo
9 944 745 015 20 private-repo
9 939 756 703 50 private-repo
9 939 833 016 100 private-repo
9 939 822 035 ... private-repo
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Wed, 23 Nov 2022 19:08:27 +0100 |
parents | 9dc1351d0b5f |
children |
line wrap: on
line source
$ hg init a $ cd a $ echo a > a $ hg add -n adding a $ hg st ? a $ hg add adding a $ hg st A a $ hg forget a $ hg add adding a $ hg forget a $ hg add --color debug [ui.addremove.added ui.status|adding a] $ hg st A a $ mkdir dir $ cd dir $ hg add ../a ../a already tracked! $ cd .. $ echo b > b $ hg add -n b $ hg st A a ? b $ hg add b $ hg st A a A b should fail $ hg add b b already tracked! $ hg st A a A b #if no-windows $ echo foo > con.xml $ hg --config ui.portablefilenames=jump add con.xml config error: ui.portablefilenames value is invalid ('jump') [30] $ hg --config ui.portablefilenames=abort add con.xml abort: filename contains 'con', which is reserved on Windows: con.xml [10] $ hg st A a A b ? con.xml $ hg add con.xml warning: filename contains 'con', which is reserved on Windows: con.xml $ hg st A a A b A con.xml $ hg forget con.xml $ rm con.xml #endif #if eol-in-paths $ echo bla > 'hello:world' $ hg --config ui.portablefilenames=abort add adding hello:world abort: filename contains ':', which is reserved on Windows: 'hello:world' [10] $ hg st A a A b ? hello:world $ hg --config ui.portablefilenames=ignore add adding hello:world $ hg st A a A b A hello:world #endif $ hg ci -m 0 --traceback $ hg log -r "heads(. or wdir() & file('**'))" changeset: 0:* (glob) tag: tip user: test date: Thu Jan 01 00:00:00 1970 +0000 summary: 0 should fail $ hg add a a already tracked! $ echo aa > a $ hg ci -m 1 $ hg up 0 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ echo aaa > a $ hg ci -m 2 created new head $ hg merge merging a warning: conflicts while merging a! (edit, then use 'hg resolve --mark') 0 files updated, 0 files merged, 0 files removed, 1 files unresolved use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon [1] $ hg st M a ? a.orig wdir doesn't cause a crash, and can be dynamically selected if dirty $ hg log -r "heads(. or wdir() & file('**'))" changeset: 2147483647:ffffffffffff parent: 2:* (glob) parent: 1:* (glob) user: test date: * (glob) should fail $ hg add a a already tracked! $ hg st M a ? a.orig $ hg resolve -m a (no more unresolved files) $ hg ci -m merge Issue683: peculiarity with hg revert of an removed then added file $ hg forget a $ hg add a $ hg st ? a.orig $ hg rm a $ hg st R a ? a.orig $ echo a > a $ hg add a $ hg st M a ? a.orig excluded file shouldn't be added even if it is explicitly specified $ hg add a.orig -X '*.orig' $ hg st M a ? a.orig Forgotten file can be added back (as either clean or modified) $ hg forget b $ hg add b $ hg st -A b C b $ hg forget b $ echo modified > b $ hg add b $ hg st -A b M b $ hg revert -qC b $ hg add c && echo "unexpected addition of missing file" c: * (glob) [1] $ echo c > c $ hg add d c && echo "unexpected addition of missing file" d: * (glob) [1] $ hg st M a A c ? a.orig $ hg up -C 1 files updated, 0 files merged, 0 files removed, 0 files unresolved forget and get should have the right order: added but missing dir should be forgotten before file with same name is added $ echo file d > d $ hg add d $ hg ci -md $ hg rm d $ mkdir d $ echo a > d/a $ hg add d/a $ rm -r d $ hg up -C 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat d file d Test that adding a directory doesn't require case matching (issue4578) #if icasefs $ mkdir -p CapsDir1/CapsDir $ echo abc > CapsDir1/CapsDir/AbC.txt $ mkdir CapsDir1/CapsDir/SubDir $ echo def > CapsDir1/CapsDir/SubDir/Def.txt $ hg add capsdir1/capsdir adding CapsDir1/CapsDir/AbC.txt adding CapsDir1/CapsDir/SubDir/Def.txt $ hg forget capsdir1/capsdir/abc.txt $ hg forget capsdir1/capsdir removing CapsDir1/CapsDir/SubDir/Def.txt $ hg add capsdir1 adding CapsDir1/CapsDir/AbC.txt adding CapsDir1/CapsDir/SubDir/Def.txt $ hg ci -m "AbCDef" capsdir1/capsdir $ hg status -A capsdir1/capsdir C CapsDir1/CapsDir/AbC.txt C CapsDir1/CapsDir/SubDir/Def.txt $ hg files capsdir1/capsdir CapsDir1/CapsDir/AbC.txt CapsDir1/CapsDir/SubDir/Def.txt $ echo xyz > CapsDir1/CapsDir/SubDir/Def.txt $ hg ci -m xyz capsdir1/capsdir/subdir/def.txt $ hg revert -r '.^' capsdir1/capsdir reverting CapsDir1/CapsDir/SubDir/Def.txt The conditional tests above mean the hash on the diff line differs on Windows and OS X $ hg diff capsdir1/capsdir diff -r * CapsDir1/CapsDir/SubDir/Def.txt (glob) --- a/CapsDir1/CapsDir/SubDir/Def.txt Thu Jan 01 00:00:00 1970 +0000 +++ b/CapsDir1/CapsDir/SubDir/Def.txt * (glob) @@ -1,1 +1,1 @@ -xyz +def $ hg mv CapsDir1/CapsDir/abc.txt CapsDir1/CapsDir/ABC.txt $ hg ci -m "case changing rename" CapsDir1/CapsDir/AbC.txt CapsDir1/CapsDir/ABC.txt $ hg status -A capsdir1/capsdir M CapsDir1/CapsDir/SubDir/Def.txt C CapsDir1/CapsDir/ABC.txt $ hg remove -f 'glob:**.txt' -X capsdir1/capsdir $ hg remove -f 'glob:**.txt' -I capsdir1/capsdir removing CapsDir1/CapsDir/ABC.txt removing CapsDir1/CapsDir/SubDir/Def.txt #endif $ cd .. test --dry-run mode in forget $ hg init testdir_forget $ cd testdir_forget $ echo foo > foo $ hg add foo $ hg commit -m "foo" $ hg forget foo --dry-run -v removing foo $ hg diff $ hg forget not_exist -n not_exist: $ENOENT$ [1] $ cd .. test --interactive mode in forget $ hg init interactiveforget $ cd interactiveforget $ echo foo > foo $ hg commit -qAm "foo" $ echo bar > bar $ hg commit -qAm "bar" $ hg forget foo --dry-run -i abort: cannot specify both --dry-run and --interactive [10] $ hg forget foo --config ui.interactive=True -i << EOF > ? > n > EOF forget foo [Ynsa?] ? y - yes, forget this file n - no, skip this file s - skip remaining files a - include all remaining files ? - ? (display help) forget foo [Ynsa?] n $ hg forget foo bar --config ui.interactive=True -i << EOF > y > n > EOF forget bar [Ynsa?] y forget foo [Ynsa?] n removing bar $ hg status R bar $ hg up -qC . $ hg forget foo bar --config ui.interactive=True -i << EOF > s > EOF forget bar [Ynsa?] s $ hg st $ hg up -qC . $ hg forget foo bar --config ui.interactive=True -i << EOF > a > EOF forget bar [Ynsa?] a removing bar removing foo $ hg status R bar R foo $ hg up -qC . $ cd ..