util: lower water mark when removing nodes after cost limit reached
See the inline comment for the reasoning here. This is a pretty
common strategy for garbage collectors, other cache-like primtives.
The performance impact is substantial:
$ hg perflrucachedict --size 4 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 100
! inserts w/ cost limit
! wall 1.659181 comb 1.650000 user 1.650000 sys 0.000000 (best of 7)
! wall 1.722122 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed w/ cost limit
! wall 1.139955 comb 1.140000 user 1.140000 sys 0.000000 (best of 9)
! wall 1.182513 comb 1.180000 user 1.180000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000
! inserts
! wall 0.679546 comb 0.680000 user 0.680000 sys 0.000000 (best of 15)
! sets
! wall 0.825147 comb 0.830000 user 0.830000 sys 0.000000 (best of 13)
! inserts w/ cost limit
! wall 25.105273 comb 25.080000 user 25.080000 sys 0.000000 (best of 3)
! wall 1.724397 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed
! wall 0.807096 comb 0.810000 user 0.810000 sys 0.000000 (best of 13)
! mixed w/ cost limit
! wall 12.104470 comb 12.070000 user 12.070000 sys 0.000000 (best of 3)
! wall 1.190563 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000 --mixedgetfreq 90
! inserts
! wall 0.711177 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
! sets
! wall 0.846992 comb 0.850000 user 0.850000 sys 0.000000 (best of 12)
! inserts w/ cost limit
! wall 25.963028 comb 25.960000 user 25.960000 sys 0.000000 (best of 3)
! wall 2.184311 comb 2.180000 user 2.180000 sys 0.000000 (best of 5)
! mixed
! wall 0.728256 comb 0.730000 user 0.730000 sys 0.000000 (best of 14)
! mixed w/ cost limit
! wall 3.174256 comb 3.170000 user 3.170000 sys 0.000000 (best of 4)
! wall 0.773186 comb 0.770000 user 0.770000 sys 0.000000 (best of 13)
$ hg perflrucachedict --size 100000 --gets 1000000 --sets 1000000 --mixed 1000000 --mixedgetfreq 90 --costlimit 5000000
! gets
! wall 1.191368 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! wall 1.195304 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! inserts
! wall 0.950995 comb 0.950000 user 0.950000 sys 0.000000 (best of 11)
! inserts w/ cost limit
! wall 1.589732 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! sets
! wall 1.094941 comb 1.100000 user 1.090000 sys 0.010000 (best of 9)
! mixed
! wall 0.936420 comb 0.940000 user 0.930000 sys 0.010000 (best of 10)
! mixed w/ cost limit
! wall 0.882780 comb 0.870000 user 0.870000 sys 0.000000 (best of 11)
This puts us ~2x slower than caches without cost accounting. And for
read-heavy workloads (the prime use cases for caches), performance is
nearly identical.
In the worst case (pure write workloads with cost accounting enabled),
we're looking at ~1.5us per insert on large caches. That seems "fast
enough."
Differential Revision: https://phab.mercurial-scm.org/D4505
#require no-reposimplestore
$ . "$TESTDIR/narrow-library.sh"
$ hg init master
$ cd master
$ cat >> .hg/hgrc <<EOF
> [narrow]
> serveellipses=True
> EOF
$ for x in `$TESTDIR/seq.py 10`
> do
> echo $x > "f$x"
> hg add "f$x"
> done
$ hg commit -m "Add root files"
$ mkdir d1 d2
$ for x in `$TESTDIR/seq.py 10`
> do
> echo d1/$x > "d1/f$x"
> hg add "d1/f$x"
> echo d2/$x > "d2/f$x"
> hg add "d2/f$x"
> done
$ hg commit -m "Add d1 and d2"
$ for x in `$TESTDIR/seq.py 10`
> do
> echo f$x rev2 > "f$x"
> echo d1/f$x rev2 > "d1/f$x"
> echo d2/f$x rev2 > "d2/f$x"
> hg commit -m "Commit rev2 of f$x, d1/f$x, d2/f$x"
> done
$ cd ..
narrow and shallow clone the d2 directory
$ hg clone --narrow ssh://user@dummy/master shallow --include "d2" --depth 2
requesting all changes
adding changesets
adding manifests
adding file changes
added 4 changesets with 13 changes to 10 files
new changesets *:* (glob)
updating to branch default
10 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd shallow
$ hg log -T '{rev}{if(ellipsis,"...")}: {desc}\n'
3: Commit rev2 of f10, d1/f10, d2/f10
2: Commit rev2 of f9, d1/f9, d2/f9
1: Commit rev2 of f8, d1/f8, d2/f8
0...: Commit rev2 of f7, d1/f7, d2/f7
$ hg update 0
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat d2/f7 d2/f8
d2/f7 rev2
d2/8
$ cd ..
change every upstream file once
$ cd master
$ for x in `$TESTDIR/seq.py 10`
> do
> echo f$x rev3 > "f$x"
> echo d1/f$x rev3 > "d1/f$x"
> echo d2/f$x rev3 > "d2/f$x"
> hg commit -m "Commit rev3 of f$x, d1/f$x, d2/f$x"
> done
$ cd ..
pull new changes with --depth specified. There were 10 changes to the d2
directory but the shallow pull should only fetch 3.
$ cd shallow
$ hg pull --depth 2
pulling from ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
adding file changes
added 4 changesets with 10 changes to 10 files
new changesets *:* (glob)
(run 'hg update' to get a working copy)
$ hg log -T '{rev}{if(ellipsis,"...")}: {desc}\n'
7: Commit rev3 of f10, d1/f10, d2/f10
6: Commit rev3 of f9, d1/f9, d2/f9
5: Commit rev3 of f8, d1/f8, d2/f8
4...: Commit rev3 of f7, d1/f7, d2/f7
3: Commit rev2 of f10, d1/f10, d2/f10
2: Commit rev2 of f9, d1/f9, d2/f9
1: Commit rev2 of f8, d1/f8, d2/f8
0...: Commit rev2 of f7, d1/f7, d2/f7
$ hg update 4
merging d2/f1
merging d2/f2
merging d2/f3
merging d2/f4
merging d2/f5
merging d2/f6
merging d2/f7
3 files updated, 7 files merged, 0 files removed, 0 files unresolved
$ cat d2/f7 d2/f8
d2/f7 rev3
d2/f8 rev2
$ hg update 7
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat d2/f10
d2/f10 rev3
$ cd ..
cannot clone with zero or negative depth
$ hg clone --narrow ssh://user@dummy/master bad --include "d2" --depth 0
requesting all changes
remote: abort: depth must be positive, got 0
abort: pull failed on remote
[255]
$ hg clone --narrow ssh://user@dummy/master bad --include "d2" --depth -1
requesting all changes
remote: abort: depth must be positive, got -1
abort: pull failed on remote
[255]