util: lower water mark when removing nodes after cost limit reached
See the inline comment for the reasoning here. This is a pretty
common strategy for garbage collectors, other cache-like primtives.
The performance impact is substantial:
$ hg perflrucachedict --size 4 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 100
! inserts w/ cost limit
! wall 1.659181 comb 1.650000 user 1.650000 sys 0.000000 (best of 7)
! wall 1.722122 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed w/ cost limit
! wall 1.139955 comb 1.140000 user 1.140000 sys 0.000000 (best of 9)
! wall 1.182513 comb 1.180000 user 1.180000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000
! inserts
! wall 0.679546 comb 0.680000 user 0.680000 sys 0.000000 (best of 15)
! sets
! wall 0.825147 comb 0.830000 user 0.830000 sys 0.000000 (best of 13)
! inserts w/ cost limit
! wall 25.105273 comb 25.080000 user 25.080000 sys 0.000000 (best of 3)
! wall 1.724397 comb 1.720000 user 1.720000 sys 0.000000 (best of 6)
! mixed
! wall 0.807096 comb 0.810000 user 0.810000 sys 0.000000 (best of 13)
! mixed w/ cost limit
! wall 12.104470 comb 12.070000 user 12.070000 sys 0.000000 (best of 3)
! wall 1.190563 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
$ hg perflrucachedict --size 1000 --gets 1000000 --sets 1000000 --mixed 1000000 --costlimit 10000 --mixedgetfreq 90
! inserts
! wall 0.711177 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
! sets
! wall 0.846992 comb 0.850000 user 0.850000 sys 0.000000 (best of 12)
! inserts w/ cost limit
! wall 25.963028 comb 25.960000 user 25.960000 sys 0.000000 (best of 3)
! wall 2.184311 comb 2.180000 user 2.180000 sys 0.000000 (best of 5)
! mixed
! wall 0.728256 comb 0.730000 user 0.730000 sys 0.000000 (best of 14)
! mixed w/ cost limit
! wall 3.174256 comb 3.170000 user 3.170000 sys 0.000000 (best of 4)
! wall 0.773186 comb 0.770000 user 0.770000 sys 0.000000 (best of 13)
$ hg perflrucachedict --size 100000 --gets 1000000 --sets 1000000 --mixed 1000000 --mixedgetfreq 90 --costlimit 5000000
! gets
! wall 1.191368 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! wall 1.195304 comb 1.190000 user 1.190000 sys 0.000000 (best of 9)
! inserts
! wall 0.950995 comb 0.950000 user 0.950000 sys 0.000000 (best of 11)
! inserts w/ cost limit
! wall 1.589732 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! sets
! wall 1.094941 comb 1.100000 user 1.090000 sys 0.010000 (best of 9)
! mixed
! wall 0.936420 comb 0.940000 user 0.930000 sys 0.010000 (best of 10)
! mixed w/ cost limit
! wall 0.882780 comb 0.870000 user 0.870000 sys 0.000000 (best of 11)
This puts us ~2x slower than caches without cost accounting. And for
read-heavy workloads (the prime use cases for caches), performance is
nearly identical.
In the worst case (pure write workloads with cost accounting enabled),
we're looking at ~1.5us per insert on large caches. That seems "fast
enough."
Differential Revision: https://phab.mercurial-scm.org/D4505
Test changesets filtering during exchanges (some tests are still in
test-obsolete.t)
$ cat >> $HGRCPATH << EOF
> [experimental]
> evolution.createmarkers=True
> EOF
Push does not corrupt remote
----------------------------
Create a DAG where a changeset reuses a revision from a file first used in an
extinct changeset.
$ hg init local
$ cd local
$ echo 'base' > base
$ hg commit -Am base
adding base
$ echo 'A' > A
$ hg commit -Am A
adding A
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg revert -ar 1
adding A
$ hg commit -Am "A'"
created new head
$ hg log -G --template='{desc} {node}'
@ A' f89bcc95eba5174b1ccc3e33a82e84c96e8338ee
|
| o A 9d73aac1b2ed7d53835eaeec212ed41ea47da53a
|/
o base d20a80d4def38df63a4b330b7fb688f3d4cae1e3
$ hg debugobsolete 9d73aac1b2ed7d53835eaeec212ed41ea47da53a f89bcc95eba5174b1ccc3e33a82e84c96e8338ee
obsoleted 1 changesets
Push it. The bundle should not refer to the extinct changeset.
$ hg init ../other
$ hg push ../other
pushing to ../other
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
$ hg -R ../other verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
checked 2 changesets with 2 changes to 2 files
Adding a changeset going extinct locally
------------------------------------------
Pull a changeset that will immediatly goes extinct (because you already have a
marker to obsolete him)
(test resolution of issue3788)
$ hg phase --draft --force f89bcc95eba5
$ hg phase -R ../other --draft --force f89bcc95eba5
$ hg commit --amend -m "A''"
$ hg --hidden --config extensions.mq= strip --no-backup f89bcc95eba5
$ hg pull ../other
pulling from ../other
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 0 changes to 1 files (+1 heads)
1 new phase-divergent changesets
new changesets f89bcc95eba5
(run 'hg heads' to see heads, 'hg merge' to merge)
check that bundle is not affected
$ hg bundle --hidden --rev f89bcc95eba5 --base "f89bcc95eba5^" ../f89bcc95eba5.hg
1 changesets found
$ hg --hidden --config extensions.mq= strip --no-backup f89bcc95eba5
$ hg unbundle ../f89bcc95eba5.hg
adding changesets
adding manifests
adding file changes
added 1 changesets with 0 changes to 1 files (+1 heads)
(run 'hg heads' to see heads)
check-that bundle can contain markers:
$ hg bundle --hidden --rev f89bcc95eba5 --base "f89bcc95eba5^" ../f89bcc95eba5-obs.hg --config experimental.evolution.bundle-obsmarker=1
1 changesets found
$ hg debugbundle ../f89bcc95eba5.hg
Stream params: {Compression: BZ}
changegroup -- {nbchanges: 1, version: 02} (mandatory: True)
f89bcc95eba5174b1ccc3e33a82e84c96e8338ee
cache:rev-branch-cache -- {} (mandatory: False)
$ hg debugbundle ../f89bcc95eba5-obs.hg
Stream params: {Compression: BZ}
changegroup -- {nbchanges: 1, version: 02} (mandatory: True)
f89bcc95eba5174b1ccc3e33a82e84c96e8338ee
cache:rev-branch-cache -- {} (mandatory: False)
obsmarkers -- {} (mandatory: True)
version: 1 (70 bytes)
9d73aac1b2ed7d53835eaeec212ed41ea47da53a f89bcc95eba5174b1ccc3e33a82e84c96e8338ee 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
$ cd ..
pull does not fetch excessive changesets when common node is hidden (issue4982)
-------------------------------------------------------------------------------
initial repo with server and client matching
$ hg init pull-hidden-common
$ cd pull-hidden-common
$ touch foo
$ hg -q commit -A -m initial
$ echo 1 > foo
$ hg commit -m 1
$ echo 2a > foo
$ hg commit -m 2a
$ cd ..
$ hg clone --pull pull-hidden-common pull-hidden-common-client
requesting all changes
adding changesets
adding manifests
adding file changes
added 3 changesets with 3 changes to 1 files
new changesets 96ee1d7354c4:6a29ed9c68de
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
server obsoletes the old head
$ cd pull-hidden-common
$ hg -q up -r 1
$ echo 2b > foo
$ hg -q commit -m 2b
$ hg debugobsolete 6a29ed9c68defff1a139e5c6fa9696fb1a75783d bec0734cd68e84477ba7fc1d13e6cff53ab70129
obsoleted 1 changesets
$ cd ..
client only pulls down 1 changeset
$ cd pull-hidden-common-client
$ hg pull --debug
pulling from $TESTTMP/pull-hidden-common
query 1; heads
searching for changes
taking quick initial sample
query 2; still undecided: 2, sample size is: 2
2 total queries in *.????s (glob)
1 changesets found
list of changesets:
bec0734cd68e84477ba7fc1d13e6cff53ab70129
listing keys for "bookmarks"
bundle2-output-bundle: "HG20", 4 parts total
bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
bundle2-output-part: "listkeys" (params: 1 mandatory) empty payload
bundle2-output-part: "phase-heads" 24 bytes payload
bundle2-output-part: "cache:rev-branch-cache" (advisory) streamed payload
bundle2-input-bundle: with-transaction
bundle2-input-part: "changegroup" (params: 1 mandatory 1 advisory) supported
adding changesets
add changeset bec0734cd68e
adding manifests
adding file changes
adding foo revisions
added 1 changesets with 1 changes to 1 files (+1 heads)
bundle2-input-part: total payload size 476
bundle2-input-part: "listkeys" (params: 1 mandatory) supported
bundle2-input-part: "phase-heads" supported
bundle2-input-part: total payload size 24
bundle2-input-part: "cache:rev-branch-cache" (advisory) supported
bundle2-input-part: total payload size 39
bundle2-input-bundle: 3 parts total
checking for updated bookmarks
updating the branch cache
new changesets bec0734cd68e
(run 'hg heads' to see heads, 'hg merge' to merge)