view tests/test-add.t @ 49658:523cacdfd324

delta-find: set the default candidate chunk size to 10 I ran performance and storage tests on repositories of various sizes and shapes for the following values of the config : 5, 10, 20, 50, 100, no-chunking The performance tests do not show any statistical impact on computation times for large pushes and pulls. For searching for an individual delta, this can provide a significant performance improvement with a minor degradation of space-quality on the result. (see data at the end of the commit). For overall store size, the change : - does not have any impact on many small repositories, - has an observable, but very negligible impact on most larger repositories. - One private repository we use for testing sees a small increase in size (1%) in the narrower version. We will try to get more numbers on a larger version of that repository to make sure nothing pathological happens. We pick "10" as the limit as "5" seems a bit more risky. There are room to improve the current code, by using more aggressive filtering and better (i.e any) sorting of the candidates. However this is already a large improvement for pathological cases, with little impact in the common situations. The initial motivation for this change is to fix performance of delta computation for a file where the previous code ended up testing 20 000 possible candidate-bases in one go, which is… slow. This affected about ½ of the file revisions leading to atrocious performance, especially during some push/pull operations. Details about individual delta finding timing: ---------------------------------------------- The vast majority of benchmark cases are unchanged but the three below. The first two do not see any impact on the final delta. The last one sees a change in delta-size that is negligible compared to the full text size. ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096) ∞: 5.844783 5: 4.473523 (-23.46%) 10: 4.970053 (-14.97%) 20: 5.770386 (-1.27%) 50 5.821358 100: 5.834887 MANIFESTLOG: rev = 756096: (no-limit) delta-base = 301840 search-rounds = 6 try-count = 60 delta-type = snapshot snap-depth = 7 delta-size = 179 MANIFESTLOG: rev=756096: (limit = 10) delta-base=301840 search-rounds=9 try-count=51 delta-type=snapshot snap-depth=7 delta-size=179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060) ∞: 5.017663 5: 3.655931 (-27.14%) 10: 4.095436 (-18.38%) 20: 4.828949 (-3.76%) 50 4.987574 100: 4.994889 MANIFESTLOG: rev=754060: (no limit) delta-base=301840 search-rounds=5 try-count=53 delta-type=snapshot snap-depth=7 delta-size = 179 MANIFESTLOG: rev=754060: (limite = 10) delta-base=301840 search-rounds=8 try-count=45 delta-type=snapshot snap-depth=7 delta-size = 179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # bin-env-vars.hg.flavor = rust # benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368) ∞: 4.869282 5: 2.039732 (-58.11%) 10: 2.413537 (-50.43%) 20: 4.449639 (-8.62%) 50 4.865863 100: 4.882649 MANIFESTLOG: rev=693368: delta-base=693336 search-rounds=6 try-count=53 delta-type=snapshot snap-depth=6 full-test-size=131065 delta-size=199 MANIFESTLOG: rev=693368: delta-base=278023 search-rounds=5 try-count=21 delta-type=snapshot snap-depth=4 full-test-size=131065 delta-size=278 Raw data for store size (in bytes) for various chunk size value below: ---------------------------------------------------------------------- 440 134 384 5 pypy/.hg/store/ 440 134 384 10 pypy/.hg/store/ 440 134 384 20 pypy/.hg/store/ 440 134 384 50 pypy/.hg/store/ 440 134 384 100 pypy/.hg/store/ 440 134 384 ... pypy/.hg/store/ 666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/ 852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/ 1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/ 3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/ 3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/ 3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/ 4 531 441 314 5 mozilla-central/.hg/store/ 4 531 435 157 10 mozilla-central/.hg/store/ 4 531 432 045 20 mozilla-central/.hg/store/ 4 531 429 119 50 mozilla-central/.hg/store/ 4 531 429 119 100 mozilla-central/.hg/store/ 4 531 429 119 ... mozilla-central/.hg/store/ 4 875 861 390 5 mozilla-unified/.hg/store/ 4 875 855 155 10 mozilla-unified/.hg/store/ 4 875 852 027 20 mozilla-unified/.hg/store/ 4 875 848 851 50 mozilla-unified/.hg/store/ 4 875 848 851 100 mozilla-unified/.hg/store/ 4 875 848 851 ... mozilla-unified/.hg/store/ 11 498 764 601 5 mozilla-try/.hg/store/ 11 497 968 858 10 mozilla-try/.hg/store/ 11 497 958 730 20 mozilla-try/.hg/store/ 11 497 927 156 50 mozilla-try/.hg/store/ 11 497 925 963 100 mozilla-try/.hg/store/ 11 497 923 428 ... mozilla-try/.hg/store/ 10 047 914 031 5 private-repo 9 969 132 101 10 private-repo 9 944 745 015 20 private-repo 9 939 756 703 50 private-repo 9 939 833 016 100 private-repo 9 939 822 035 ... private-repo
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 23 Nov 2022 19:08:27 +0100
parents 9dc1351d0b5f
children
line wrap: on
line source

  $ hg init a
  $ cd a
  $ echo a > a
  $ hg add -n
  adding a
  $ hg st
  ? a
  $ hg add
  adding a
  $ hg st
  A a
  $ hg forget a
  $ hg add
  adding a
  $ hg forget a
  $ hg add --color debug
  [ui.addremove.added ui.status|adding a]
  $ hg st
  A a
  $ mkdir dir
  $ cd dir
  $ hg add ../a
  ../a already tracked!
  $ cd ..

  $ echo b > b
  $ hg add -n b
  $ hg st
  A a
  ? b
  $ hg add b
  $ hg st
  A a
  A b

should fail

  $ hg add b
  b already tracked!
  $ hg st
  A a
  A b

#if no-windows
  $ echo foo > con.xml
  $ hg --config ui.portablefilenames=jump add con.xml
  config error: ui.portablefilenames value is invalid ('jump')
  [30]
  $ hg --config ui.portablefilenames=abort add con.xml
  abort: filename contains 'con', which is reserved on Windows: con.xml
  [10]
  $ hg st
  A a
  A b
  ? con.xml
  $ hg add con.xml
  warning: filename contains 'con', which is reserved on Windows: con.xml
  $ hg st
  A a
  A b
  A con.xml
  $ hg forget con.xml
  $ rm con.xml
#endif

#if eol-in-paths
  $ echo bla > 'hello:world'
  $ hg --config ui.portablefilenames=abort add
  adding hello:world
  abort: filename contains ':', which is reserved on Windows: 'hello:world'
  [10]
  $ hg st
  A a
  A b
  ? hello:world
  $ hg --config ui.portablefilenames=ignore add
  adding hello:world
  $ hg st
  A a
  A b
  A hello:world
#endif

  $ hg ci -m 0 --traceback

  $ hg log -r "heads(. or wdir() & file('**'))"
  changeset:   0:* (glob)
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     0
  
should fail

  $ hg add a
  a already tracked!

  $ echo aa > a
  $ hg ci -m 1
  $ hg up 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo aaa > a
  $ hg ci -m 2
  created new head

  $ hg merge
  merging a
  warning: conflicts while merging a! (edit, then use 'hg resolve --mark')
  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon
  [1]
  $ hg st
  M a
  ? a.orig

wdir doesn't cause a crash, and can be dynamically selected if dirty

  $ hg log -r "heads(. or wdir() & file('**'))"
  changeset:   2147483647:ffffffffffff
  parent:      2:* (glob)
  parent:      1:* (glob)
  user:        test
  date:        * (glob)
  
should fail

  $ hg add a
  a already tracked!
  $ hg st
  M a
  ? a.orig
  $ hg resolve -m a
  (no more unresolved files)
  $ hg ci -m merge

Issue683: peculiarity with hg revert of an removed then added file

  $ hg forget a
  $ hg add a
  $ hg st
  ? a.orig
  $ hg rm a
  $ hg st
  R a
  ? a.orig
  $ echo a > a
  $ hg add a
  $ hg st
  M a
  ? a.orig

excluded file shouldn't be added even if it is explicitly specified

  $ hg add a.orig -X '*.orig'
  $ hg st
  M a
  ? a.orig

Forgotten file can be added back (as either clean or modified)

  $ hg forget b
  $ hg add b
  $ hg st -A b
  C b
  $ hg forget b
  $ echo modified > b
  $ hg add b
  $ hg st -A b
  M b
  $ hg revert -qC b

  $ hg add c && echo "unexpected addition of missing file"
  c: * (glob)
  [1]
  $ echo c > c
  $ hg add d c && echo "unexpected addition of missing file"
  d: * (glob)
  [1]
  $ hg st
  M a
  A c
  ? a.orig
  $ hg up -C
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

forget and get should have the right order: added but missing dir should be
forgotten before file with same name is added

  $ echo file d > d
  $ hg add d
  $ hg ci -md
  $ hg rm d
  $ mkdir d
  $ echo a > d/a
  $ hg add d/a
  $ rm -r d
  $ hg up -C
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cat d
  file d

Test that adding a directory doesn't require case matching (issue4578)
#if icasefs
  $ mkdir -p CapsDir1/CapsDir
  $ echo abc > CapsDir1/CapsDir/AbC.txt
  $ mkdir CapsDir1/CapsDir/SubDir
  $ echo def > CapsDir1/CapsDir/SubDir/Def.txt

  $ hg add capsdir1/capsdir
  adding CapsDir1/CapsDir/AbC.txt
  adding CapsDir1/CapsDir/SubDir/Def.txt

  $ hg forget capsdir1/capsdir/abc.txt

  $ hg forget capsdir1/capsdir
  removing CapsDir1/CapsDir/SubDir/Def.txt

  $ hg add capsdir1
  adding CapsDir1/CapsDir/AbC.txt
  adding CapsDir1/CapsDir/SubDir/Def.txt

  $ hg ci -m "AbCDef" capsdir1/capsdir

  $ hg status -A capsdir1/capsdir
  C CapsDir1/CapsDir/AbC.txt
  C CapsDir1/CapsDir/SubDir/Def.txt

  $ hg files capsdir1/capsdir
  CapsDir1/CapsDir/AbC.txt
  CapsDir1/CapsDir/SubDir/Def.txt

  $ echo xyz > CapsDir1/CapsDir/SubDir/Def.txt
  $ hg ci -m xyz capsdir1/capsdir/subdir/def.txt

  $ hg revert -r '.^' capsdir1/capsdir
  reverting CapsDir1/CapsDir/SubDir/Def.txt

The conditional tests above mean the hash on the diff line differs on Windows
and OS X
  $ hg diff capsdir1/capsdir
  diff -r * CapsDir1/CapsDir/SubDir/Def.txt (glob)
  --- a/CapsDir1/CapsDir/SubDir/Def.txt	Thu Jan 01 00:00:00 1970 +0000
  +++ b/CapsDir1/CapsDir/SubDir/Def.txt	* (glob)
  @@ -1,1 +1,1 @@
  -xyz
  +def

  $ hg mv CapsDir1/CapsDir/abc.txt CapsDir1/CapsDir/ABC.txt
  $ hg ci -m "case changing rename" CapsDir1/CapsDir/AbC.txt CapsDir1/CapsDir/ABC.txt

  $ hg status -A capsdir1/capsdir
  M CapsDir1/CapsDir/SubDir/Def.txt
  C CapsDir1/CapsDir/ABC.txt

  $ hg remove -f 'glob:**.txt' -X capsdir1/capsdir
  $ hg remove -f 'glob:**.txt' -I capsdir1/capsdir
  removing CapsDir1/CapsDir/ABC.txt
  removing CapsDir1/CapsDir/SubDir/Def.txt
#endif

  $ cd ..

test --dry-run mode in forget

  $ hg init testdir_forget
  $ cd testdir_forget
  $ echo foo > foo
  $ hg add foo
  $ hg commit -m "foo"
  $ hg forget foo --dry-run -v
  removing foo
  $ hg diff
  $ hg forget not_exist -n
  not_exist: $ENOENT$
  [1]

  $ cd ..

test --interactive mode in forget

  $ hg init interactiveforget
  $ cd interactiveforget
  $ echo foo > foo
  $ hg commit -qAm "foo"
  $ echo bar > bar
  $ hg commit -qAm "bar"
  $ hg forget foo --dry-run -i
  abort: cannot specify both --dry-run and --interactive
  [10]

  $ hg forget foo --config ui.interactive=True -i << EOF
  > ?
  > n
  > EOF
  forget foo [Ynsa?] ?
  y - yes, forget this file
  n - no, skip this file
  s - skip remaining files
  a - include all remaining files
  ? - ? (display help)
  forget foo [Ynsa?] n

  $ hg forget foo bar --config ui.interactive=True -i << EOF
  > y
  > n
  > EOF
  forget bar [Ynsa?] y
  forget foo [Ynsa?] n
  removing bar
  $ hg status
  R bar
  $ hg up -qC .

  $ hg forget foo bar --config ui.interactive=True -i << EOF
  > s
  > EOF
  forget bar [Ynsa?] s
  $ hg st
  $ hg up -qC .

  $ hg forget foo bar --config ui.interactive=True -i << EOF
  > a
  > EOF
  forget bar [Ynsa?] a
  removing bar
  removing foo
  $ hg status
  R bar
  R foo
  $ hg up -qC .

  $ cd ..