util: lower water mark when removing nodes after cost limit reached
See the inline comment for the reasoning here. This is a pretty
common strategy for garbage collectors, other cache-like primtives.
The performance impact is substantial:
$ hg perflrucachedict --size 4 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 100
! inserts w/ cost limit
! wall 1.659181 comb 1.650000 user 1.650000 sys 0.000000 (best of 7)
! wall 1.722122 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed w/ cost limit
! wall 1.139955 comb 1.140000 user 1.140000 sys 0.000000 (best of 9)
! wall 1.182513 comb 1.180000 user 1.180000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000
! inserts
! wall 0.679546 comb 0.680000 user 0.680000 sys 0.000000 (best of 15)
! sets
! wall 0.825147 comb 0.830000 user 0.830000 sys 0.000000 (best of 13)
! inserts w/ cost limit
! wall 25.105273 comb 25.080000 user 25.080000 sys 0.000000 (best of 3)
! wall 1.724397 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed
! wall 0.807096 comb 0.810000 user 0.810000 sys 0.000000 (best of 13)
! mixed w/ cost limit
! wall 12.104470 comb 12.070000 user 12.070000 sys 0.000000 (best of 3)
! wall 1.190563 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000 --mixedgetfreq 90
! inserts
! wall 0.711177 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
! sets
! wall 0.846992 comb 0.850000 user 0.850000 sys 0.000000 (best of 12)
! inserts w/ cost limit
! wall 25.963028 comb 25.960000 user 25.960000 sys 0.000000 (best of 3)
! wall 2.184311 comb 2.180000 user 2.180000 sys 0.000000 (best of 5)
! mixed
! wall 0.728256 comb 0.730000 user 0.730000 sys 0.000000 (best of 14)
! mixed w/ cost limit
! wall 3.174256 comb 3.170000 user 3.170000 sys 0.000000 (best of 4)
! wall 0.773186 comb 0.770000 user 0.770000 sys 0.000000 (best of 13)
$ hg perflrucachedict --size 100000 --gets 1000000 --sets 1000000 --mixed 1000000 --mixedgetfreq 90 --costlimit 5000000
! gets
! wall 1.191368 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! wall 1.195304 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! inserts
! wall 0.950995 comb 0.950000 user 0.950000 sys 0.000000 (best of 11)
! inserts w/ cost limit
! wall 1.589732 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! sets
! wall 1.094941 comb 1.100000 user 1.090000 sys 0.010000 (best of 9)
! mixed
! wall 0.936420 comb 0.940000 user 0.930000 sys 0.010000 (best of 10)
! mixed w/ cost limit
! wall 0.882780 comb 0.870000 user 0.870000 sys 0.000000 (best of 11)
This puts us ~2x slower than caches without cost accounting. And for
read-heavy workloads (the prime use cases for caches), performance is
nearly identical.
In the worst case (pure write workloads with cost accounting enabled),
we're looking at ~1.5us per insert on large caches. That seems "fast
enough."
Differential Revision: https://phab.mercurial-scm.org/D4505
#testcases sshv1 sshv2
#if sshv2
$ cat >> $HGRCPATH << EOF
> [experimental]
> sshpeer.advertise-v2 = true
> sshserver.support-v2 = true
> EOF
#endif
$ cat > bundle2.py << EOF
> """A small extension to test bundle2 pushback parts.
> Current bundle2 implementation doesn't provide a way to generate those
> parts, so they must be created by extensions.
> """
> from __future__ import absolute_import
> from mercurial import bundle2, exchange, pushkey, util
> def _newhandlechangegroup(op, inpart):
> """This function wraps the changegroup part handler for getbundle.
> It issues an additional pushkey part to send a new
> bookmark back to the client"""
> result = bundle2.handlechangegroup(op, inpart)
> if b'pushback' in op.reply.capabilities:
> params = {b'namespace': b'bookmarks',
> b'key': b'new-server-mark',
> b'old': b'',
> b'new': b'tip'}
> encodedparams = [(k, pushkey.encode(v)) for (k,v) in params.items()]
> op.reply.newpart(b'pushkey', mandatoryparams=encodedparams)
> else:
> op.reply.newpart(b'output', data=b'pushback not enabled')
> return result
> _newhandlechangegroup.params = bundle2.handlechangegroup.params
> bundle2.parthandlermapping[b'changegroup'] = _newhandlechangegroup
> EOF
$ cat >> $HGRCPATH <<EOF
> [ui]
> ssh = $PYTHON "$TESTDIR/dummyssh"
> username = nobody <no.reply@example.com>
>
> [alias]
> tglog = log -G -T "{desc} [{phase}:{node|short}]"
> EOF
Set up server repository
$ hg init server
$ cd server
$ echo c0 > f0
$ hg commit -Am 0
adding f0
Set up client repository
$ cd ..
$ hg clone ssh://user@dummy/server client -q
$ cd client
Enable extension
$ cat >> $HGRCPATH <<EOF
> [extensions]
> bundle2=$TESTTMP/bundle2.py
> EOF
Without config
$ cd ../client
$ echo c1 > f1
$ hg commit -Am 1
adding f1
$ hg push
pushing to ssh://user@dummy/server
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: pushback not enabled
$ hg bookmark
no bookmarks set
$ cd ../server
$ hg tglog
o 1 [public:2b9c7234e035]
|
@ 0 [public:6cee5c8f3e5b]
With config
$ cd ../client
$ echo '[experimental]' >> .hg/hgrc
$ echo 'bundle2.pushback = True' >> .hg/hgrc
$ echo c2 > f2
$ hg commit -Am 2
adding f2
$ hg push
pushing to ssh://user@dummy/server
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
$ hg bookmark
new-server-mark 2:0a76dfb2e179
$ cd ../server
$ hg tglog
o 2 [public:0a76dfb2e179]
|
o 1 [public:2b9c7234e035]
|
@ 0 [public:6cee5c8f3e5b]