util: lower water mark when removing nodes after cost limit reached
See the inline comment for the reasoning here. This is a pretty
common strategy for garbage collectors, other cache-like primtives.
The performance impact is substantial:
$ hg perflrucachedict --size 4 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 100
! inserts w/ cost limit
! wall 1.659181 comb 1.650000 user 1.650000 sys 0.000000 (best of 7)
! wall 1.722122 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed w/ cost limit
! wall 1.139955 comb 1.140000 user 1.140000 sys 0.000000 (best of 9)
! wall 1.182513 comb 1.180000 user 1.180000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000
! inserts
! wall 0.679546 comb 0.680000 user 0.680000 sys 0.000000 (best of 15)
! sets
! wall 0.825147 comb 0.830000 user 0.830000 sys 0.000000 (best of 13)
! inserts w/ cost limit
! wall 25.105273 comb 25.080000 user 25.080000 sys 0.000000 (best of 3)
! wall 1.724397 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed
! wall 0.807096 comb 0.810000 user 0.810000 sys 0.000000 (best of 13)
! mixed w/ cost limit
! wall 12.104470 comb 12.070000 user 12.070000 sys 0.000000 (best of 3)
! wall 1.190563 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000 --mixedgetfreq 90
! inserts
! wall 0.711177 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
! sets
! wall 0.846992 comb 0.850000 user 0.850000 sys 0.000000 (best of 12)
! inserts w/ cost limit
! wall 25.963028 comb 25.960000 user 25.960000 sys 0.000000 (best of 3)
! wall 2.184311 comb 2.180000 user 2.180000 sys 0.000000 (best of 5)
! mixed
! wall 0.728256 comb 0.730000 user 0.730000 sys 0.000000 (best of 14)
! mixed w/ cost limit
! wall 3.174256 comb 3.170000 user 3.170000 sys 0.000000 (best of 4)
! wall 0.773186 comb 0.770000 user 0.770000 sys 0.000000 (best of 13)
$ hg perflrucachedict --size 100000 --gets 1000000 --sets 1000000 --mixed 1000000 --mixedgetfreq 90 --costlimit 5000000
! gets
! wall 1.191368 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! wall 1.195304 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! inserts
! wall 0.950995 comb 0.950000 user 0.950000 sys 0.000000 (best of 11)
! inserts w/ cost limit
! wall 1.589732 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! sets
! wall 1.094941 comb 1.100000 user 1.090000 sys 0.010000 (best of 9)
! mixed
! wall 0.936420 comb 0.940000 user 0.930000 sys 0.010000 (best of 10)
! mixed w/ cost limit
! wall 0.882780 comb 0.870000 user 0.870000 sys 0.000000 (best of 11)
This puts us ~2x slower than caches without cost accounting. And for
read-heavy workloads (the prime use cases for caches), performance is
nearly identical.
In the worst case (pure write workloads with cost accounting enabled),
we're looking at ~1.5us per insert on large caches. That seems "fast
enough."
Differential Revision: https://phab.mercurial-scm.org/D4505
https://bz.mercurial-scm.org/660 and:
https://bz.mercurial-scm.org/322
$ hg init
$ echo a > a
$ mkdir b
$ echo b > b/b
$ hg commit -A -m "a is file, b is dir"
adding a
adding b/b
File replaced with directory:
$ rm a
$ mkdir a
$ echo a > a/a
Should fail - would corrupt dirstate:
$ hg add a/a
abort: file 'a' in dirstate clashes with 'a/a'
[255]
Removing shadow:
$ hg rm --after a
Should succeed - shadow removed:
$ hg add a/a
Directory replaced with file:
$ rm -r b
$ echo b > b
Should fail - would corrupt dirstate:
$ hg add b
abort: directory 'b' already in dirstate
[255]
Removing shadow:
$ hg rm --after b/b
Should succeed - shadow removed:
$ hg add b
Look what we got:
$ hg st
A a/a
A b
R a
R b/b
Revert reintroducing shadow - should fail:
$ rm -r a b
$ hg revert b/b
abort: file 'b' in dirstate clashes with 'b/b'
[255]
Revert all - should succeed:
$ hg revert --all
forgetting a/a
forgetting b
undeleting a
undeleting b/b
$ hg st
Issue3423:
$ hg forget a
$ echo zed > a
$ hg revert a
$ hg st
? a.orig
$ rm a.orig
addremove:
$ rm -r a b
$ mkdir a
$ echo a > a/a
$ echo b > b
$ hg addremove -s 0
removing a
adding a/a
adding b
removing b/b
$ hg st
A a/a
A b
R a
R b/b
commit:
$ hg ci -A -m "a is dir, b is file"
$ hg st --all
C a/a
C b
Long directory replaced with file:
$ mkdir d
$ mkdir d/d
$ echo d > d/d/d
$ hg commit -A -m "d is long directory"
adding d/d/d
$ rm -r d
$ echo d > d
Should fail - would corrupt dirstate:
$ hg add d
abort: directory 'd' already in dirstate
[255]
Removing shadow:
$ hg rm --after d/d/d
Should succeed - shadow removed:
$ hg add d
$ hg ci -md
Update should work at least with clean working directory:
$ rm -r a b d
$ hg up -r 0
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg st --all
C a
C b/b
$ rm -r a b
$ hg up -r 1
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg st --all
C a/a
C b