Mercurial > hg

changeset 51492:2e8a88e5809f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
branchcache: stop writing more branchcache file on disk than needed Before this change, we were unconditionally writing a branchmap file for the filter level passed to `update_disk`. This is actually counter productive if no update were needed for this filter level. In many case, the branch cache for a filter level is identical to its parent "subset" and it is better to simply keep the subset update and reuse it every time instead of having to do identical work for similar subset. So we change the `update_disk` method to only write a file when that filter level differ from its parent. This removes many cases where identical files were written, requiring multiple boring update in the test suite. The only notable changes is the change to `test-strip-branch-cache.t`, this case was checking a scenario that no longer reproduce the bug as writing less branchmap file result in less stalled cache on disk. Strictly speaking, we could create a more convoluted scenario that create a similar issue. However the next changeset would also cover that scenario so we directly updated that test case to a "no longer buggy" state.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Sun, 10 Mar 2024 04:53:17 +0100
parents dd5b5f99940d
children 82c1a388e86a
files mercurial/branchmap.py tests/test-acl.t tests/test-blackbox.t tests/test-branches.t tests/test-clone-stream.t tests/test-clone.t tests/test-debugcommands.t tests/test-hardlinks.t tests/test-server-view.t tests/test-share.t tests/test-ssh.t tests/test-strip-branch-cache.t tests/test-tags.t tests/test-treemanifest.t
diffstat 14 files changed, 75 insertions(+), 114 deletions(-) [+]
line wrap: on
line diff
--- a/mercurial/branchmap.py	Fri Mar 08 16:49:06 2024 +0100
+++ b/mercurial/branchmap.py	Sun Mar 10 04:53:17 2024 +0100
@@ -85,7 +85,8 @@
             bcache._filtername,
             repo.filtername,
         )
-        bcache.write(repo)
+        if bcache._dirty:
+            bcache.write(repo)
 
     def updatecache(self, repo):
         """Update the cache for the given filtered view on a repository"""
--- a/tests/test-acl.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-acl.t	Sun Mar 10 04:53:17 2024 +0100
@@ -810,7 +810,6 @@
   acl: bookmark access granted: "ef1ea85a6374b77d6da9dcda9541f498f2d17df7" on bookmark "moving-bookmark"
   bundle2-input-bundle: 7 parts total
   updating the branch cache
-  invalid branch cache (served.hidden): tip differs
   added 1 changesets with 1 changes to 1 files
   bundle2-output-bundle: "HG20", 1 parts total
   bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -1797,7 +1796,6 @@
   bundle2-input-part: total payload size * (glob)
   bundle2-input-bundle: 5 parts total
   updating the branch cache
-  invalid branch cache (served.hidden): tip differs
   added 4 changesets with 4 changes to 4 files (+1 heads)
   bundle2-output-bundle: "HG20", 1 parts total
   bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2104,7 +2102,6 @@
   bundle2-input-part: total payload size * (glob)
   bundle2-input-bundle: 5 parts total
   updating the branch cache
-  invalid branch cache (served.hidden): tip differs
   added 4 changesets with 4 changes to 4 files (+1 heads)
   bundle2-output-bundle: "HG20", 1 parts total
   bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2196,7 +2193,6 @@
   bundle2-input-part: total payload size * (glob)
   bundle2-input-bundle: 5 parts total
   updating the branch cache
-  invalid branch cache (served.hidden): tip differs
   added 4 changesets with 4 changes to 4 files (+1 heads)
   bundle2-output-bundle: "HG20", 1 parts total
   bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
@@ -2360,7 +2356,6 @@
   bundle2-input-part: total payload size * (glob)
   bundle2-input-bundle: 5 parts total
   updating the branch cache
-  invalid branch cache (served.hidden): tip differs
   added 4 changesets with 4 changes to 4 files (+1 heads)
   bundle2-output-bundle: "HG20", 1 parts total
   bundle2-output-part: "reply:changegroup" (advisory) (params: 0 advisory) empty payload
--- a/tests/test-blackbox.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-blackbox.t	Sun Mar 10 04:53:17 2024 +0100
@@ -127,13 +127,11 @@
   added 1 changesets with 1 changes to 1 files
   new changesets d02f48003e62
   (run 'hg update' to get a working copy)
-  $ hg blackbox -l 6
+  $ hg blackbox -l 4
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (served) with 1 labels and 2 nodes
-  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (served.hidden) in * seconds (glob)
-  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (served.hidden) with 1 labels and 2 nodes
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> 1 incoming changes - new heads: d02f48003e62
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> pull exited 0 after * seconds (glob)
-  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6
+  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 4
 
 we must not cause a failure if we cannot write to the log
 
@@ -193,8 +191,8 @@
   $ hg blackbox -l 6
   1970-01-01 00:00:00.000 bob @73f6ee326b27d820b0472f1a825e3a50f3dc489b (5000)> strip tip
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> saved backup bundle to $TESTTMP/blackboxtest2/.hg/strip-backup/73f6ee326b27-7612e004-backup.hg
-  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (immutable) in * seconds (glob)
-  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (immutable) with 1 labels and 2 nodes
+  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> updated branch cache (base) in * seconds (glob)
+  1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> wrote branch cache (base) with 1 labels and 2 nodes
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> strip tip exited 0 after * seconds (glob)
   1970-01-01 00:00:00.000 bob @6563da9dcf87b1949716e38ff3e3dfaa3198eb06 (5000)> blackbox -l 6
 
--- a/tests/test-branches.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-branches.t	Sun Mar 10 04:53:17 2024 +0100
@@ -1316,7 +1316,7 @@
   new changesets 2ab8003a1750:99ba08759bc7
   updating to branch A
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
-  $ cat branchmap-update-01/.hg/cache/branch2-served
+  $ cat branchmap-update-01/.hg/cache/branch2-base
   99ba08759bc7f6fdbe5304e83d0387f35c082479 1
   99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
   $ hg -R branchmap-update-01 unbundle bundle.hg
@@ -1350,7 +1350,7 @@
   updating to branch A
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
-  $ cat branchmap-update-02/.hg/cache/branch2-served
+  $ cat branchmap-update-02/.hg/cache/branch2-base
   99ba08759bc7f6fdbe5304e83d0387f35c082479 1
   99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
   $ hg -R branchmap-update-02 unbundle bundle.hg --config "hooks.pretxnclose=python:$TESTTMP/simplehook.py:hook"
@@ -1361,6 +1361,6 @@
   rollback completed
   abort: pretxnclose hook failed
   [40]
-  $ cat branchmap-update-02/.hg/cache/branch2-served
+  $ cat branchmap-update-02/.hg/cache/branch2-base
   99ba08759bc7f6fdbe5304e83d0387f35c082479 1
   99ba08759bc7f6fdbe5304e83d0387f35c082479 o A
--- a/tests/test-clone-stream.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-clone-stream.t	Sun Mar 10 04:53:17 2024 +0100
@@ -220,11 +220,7 @@
 #if no-stream-legacy
   $ ls -1 clone1/.hg/cache
   branch2-base
-  branch2-immutable
   branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   rbc-names-v1
   rbc-revs-v1
   tags2
--- a/tests/test-clone.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-clone.t	Sun Mar 10 04:53:17 2024 +0100
@@ -47,11 +47,7 @@
 
   $ ls .hg/cache
   branch2-base
-  branch2-immutable
   branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   rbc-names-v1
   rbc-revs-v1
   tags2
@@ -71,42 +67,34 @@
 
 #if hardlink
   $ hg --debug clone -U . ../c --config progress.debug=true
-  linking: 1/16 files (6.25%) (no-rust !)
-  linking: 2/16 files (12.50%) (no-rust !)
-  linking: 3/16 files (18.75%) (no-rust !)
-  linking: 4/16 files (25.00%) (no-rust !)
-  linking: 5/16 files (31.25%) (no-rust !)
-  linking: 6/16 files (37.50%) (no-rust !)
-  linking: 7/16 files (43.75%) (no-rust !)
-  linking: 8/16 files (50.00%) (no-rust !)
-  linking: 9/16 files (56.25%) (no-rust !)
-  linking: 10/16 files (62.50%) (no-rust !)
-  linking: 11/16 files (68.75%) (no-rust !)
-  linking: 12/16 files (75.00%) (no-rust !)
-  linking: 13/16 files (81.25%) (no-rust !)
-  linking: 14/16 files (87.50%) (no-rust !)
-  linking: 15/16 files (93.75%) (no-rust !)
-  linking: 16/16 files (100.00%) (no-rust !)
-  linked 16 files (no-rust !)
-  linking: 1/18 files (5.56%) (rust !)
-  linking: 2/18 files (11.11%) (rust !)
-  linking: 3/18 files (16.67%) (rust !)
-  linking: 4/18 files (22.22%) (rust !)
-  linking: 5/18 files (27.78%) (rust !)
-  linking: 6/18 files (33.33%) (rust !)
-  linking: 7/18 files (38.89%) (rust !)
-  linking: 8/18 files (44.44%) (rust !)
-  linking: 9/18 files (50.00%) (rust !)
-  linking: 10/18 files (55.56%) (rust !)
-  linking: 11/18 files (61.11%) (rust !)
-  linking: 12/18 files (66.67%) (rust !)
-  linking: 13/18 files (72.22%) (rust !)
-  linking: 14/18 files (77.78%) (rust !)
-  linking: 15/18 files (83.33%) (rust !)
-  linking: 16/18 files (88.89%) (rust !)
-  linking: 17/18 files (94.44%) (rust !)
-  linking: 18/18 files (100.00%) (rust !)
-  linked 18 files (rust !)
+  linking: 1/12 files (8.33%) (no-rust !)
+  linking: 2/12 files (16.67%) (no-rust !)
+  linking: 3/12 files (25.00%) (no-rust !)
+  linking: 4/12 files (33.33%) (no-rust !)
+  linking: 5/12 files (41.67%) (no-rust !)
+  linking: 6/12 files (50.00%) (no-rust !)
+  linking: 7/12 files (58.33%) (no-rust !)
+  linking: 8/12 files (66.67%) (no-rust !)
+  linking: 9/12 files (75.00%) (no-rust !)
+  linking: 10/12 files (83.33%) (no-rust !)
+  linking: 11/12 files (91.67%) (no-rust !)
+  linking: 12/12 files (100.00%) (no-rust !)
+  linked 12 files (no-rust !)
+  linking: 1/14 files (7.14%) (rust !)
+  linking: 2/14 files (14.29%) (rust !)
+  linking: 3/14 files (21.43%) (rust !)
+  linking: 4/14 files (28.57%) (rust !)
+  linking: 5/14 files (35.71%) (rust !)
+  linking: 6/14 files (42.86%) (rust !)
+  linking: 7/14 files (50.00%) (rust !)
+  linking: 8/14 files (57.14%) (rust !)
+  linking: 9/14 files (64.29%) (rust !)
+  linking: 10/14 files (71.43%) (rust !)
+  linking: 11/14 files (78.57%) (rust !)
+  linking: 12/14 files (85.71%) (rust !)
+  linking: 13/14 files (92.86%) (rust !)
+  linking: 14/14 files (100.00%) (rust !)
+  linked 14 files (rust !)
   updating the branch cache
 #else
   $ hg --debug clone -U . ../c --config progress.debug=true
@@ -125,11 +113,7 @@
 
   $ ls .hg/cache
   branch2-base
-  branch2-immutable
   branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   rbc-names-v1
   rbc-revs-v1
   tags2
--- a/tests/test-debugcommands.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-debugcommands.t	Sun Mar 10 04:53:17 2024 +0100
@@ -652,12 +652,7 @@
   .hg/cache/rbc-revs-v1
   .hg/cache/rbc-names-v1
   .hg/cache/hgtagsfnodes1
-  .hg/cache/branch2-visible-hidden
-  .hg/cache/branch2-visible
-  .hg/cache/branch2-served.hidden
   .hg/cache/branch2-served
-  .hg/cache/branch2-immutable
-  .hg/cache/branch2-base
 
 Test debug::unbundle
 
--- a/tests/test-hardlinks.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-hardlinks.t	Sun Mar 10 04:53:17 2024 +0100
@@ -263,11 +263,7 @@
   2 r4/.hg/00changelog.i
   [24] r4/.hg/branch (re)
   2 r4/.hg/cache/branch2-base
-  2 r4/.hg/cache/branch2-immutable
   2 r4/.hg/cache/branch2-served
-  2 r4/.hg/cache/branch2-served.hidden
-  2 r4/.hg/cache/branch2-visible
-  2 r4/.hg/cache/branch2-visible-hidden
   2 r4/.hg/cache/rbc-names-v1
   2 r4/.hg/cache/rbc-revs-v1
   2 r4/.hg/cache/tags2
@@ -320,11 +316,7 @@
   2 r4/.hg/00changelog.i
   1 r4/.hg/branch
   2 r4/.hg/cache/branch2-base
-  2 r4/.hg/cache/branch2-immutable
   2 r4/.hg/cache/branch2-served
-  2 r4/.hg/cache/branch2-served.hidden
-  2 r4/.hg/cache/branch2-visible
-  2 r4/.hg/cache/branch2-visible-hidden
   2 r4/.hg/cache/rbc-names-v1
   2 r4/.hg/cache/rbc-revs-v1
   2 r4/.hg/cache/tags2
--- a/tests/test-server-view.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-server-view.t	Sun Mar 10 04:53:17 2024 +0100
@@ -36,12 +36,7 @@
   $ hg -R test --config experimental.extra-filter-revs='not public()' debugupdatecache
   $ ls -1 test/.hg/cache/
   branch2-base%89c45d2fa07e
-  branch2-immutable%89c45d2fa07e
   branch2-served
-  branch2-served%89c45d2fa07e
-  branch2-served.hidden%89c45d2fa07e
-  branch2-visible%89c45d2fa07e
-  branch2-visible-hidden%89c45d2fa07e
   hgtagsfnodes1
   rbc-names-v1
   rbc-revs-v1
--- a/tests/test-share.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-share.t	Sun Mar 10 04:53:17 2024 +0100
@@ -63,11 +63,7 @@
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ ls -1 ../repo2-clone/.hg/cache
   branch2-base
-  branch2-immutable
   branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   rbc-names-v1
   rbc-revs-v1
   tags2
--- a/tests/test-ssh.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-ssh.t	Sun Mar 10 04:53:17 2024 +0100
@@ -72,8 +72,8 @@
   $ hg -R local-stream book mybook
   $ hg clone --stream ssh://user@dummy/local-stream stream2
   streaming all changes
-  16 files to transfer, * of data (glob) (no-rust !)
-  18 files to transfer, * of data (glob) (rust !)
+  12 files to transfer, * of data (glob) (no-rust !)
+  14 files to transfer, * of data (glob) (rust !)
   transferred * in * seconds (*) (glob)
   updating to branch default
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
--- a/tests/test-strip-branch-cache.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-strip-branch-cache.t	Sun Mar 10 04:53:17 2024 +0100
@@ -1,3 +1,5 @@
+This test cover a bug that no longer exist.
+
 Define helpers.
 
   $ hg_log () { hg log -G -T "{rev}:{node|short}"; }
@@ -18,7 +20,10 @@
 
   $ hg pull -q ../repo
 
-  $ cat .hg/cache/branch2-visible
+  $ ls -1 .hg/cache/branch?*
+  .hg/cache/branch2-base
+  .hg/cache/branch2-served
+  $ cat .hg/cache/branch?-served
   222ae9789a75703f9836e44de7db179cbfd420ee 2
   a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
   222ae9789a75703f9836e44de7db179cbfd420ee o default
@@ -33,24 +38,38 @@
 
   $ strip '1:'
 
-The branchmap cache is not adjusted on strip.
-Now mentions a changelog entry that has been stripped.
+After the strip the "served" cache is now identical to the "base" one, and the
+older one have been actively deleted.
 
-  $ cat .hg/cache/branch2-visible
+  $ ls -1 .hg/cache/branch?*
+  .hg/cache/branch2-base
+  .hg/cache/branch2-served
+  $ cat .hg/cache/branch?-served
   222ae9789a75703f9836e44de7db179cbfd420ee 2
   a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
   222ae9789a75703f9836e44de7db179cbfd420ee o default
 
-  $ commit c
-
-Not adjusted on commit, either.
+We do a new commit and we get a new valid branchmap for the served version
 
-  $ cat .hg/cache/branch2-visible
-  222ae9789a75703f9836e44de7db179cbfd420ee 2
-  a3498d6e39376d2456425dd8c692367bdbf00fa2 o default
-  222ae9789a75703f9836e44de7db179cbfd420ee o default
+  $ commit c
+  $ ls -1 .hg/cache/branch?*
+  .hg/cache/branch2-base
+  .hg/cache/branch2-served
+  $ cat .hg/cache/branch?-served
+  a1602b357cfca067600406eb19060c7128804d72 1
+  a1602b357cfca067600406eb19060c7128804d72 o default
+
 
 On pull we end up with the same tip, and so wrongly reuse the invalid cache and crash.
 
-  $ hg pull ../repo 2>&1 | grep 'ValueError:'
-  ValueError: node a3498d6e39376d2456425dd8c692367bdbf00fa2 does not exist (known-bad-output !)
+  $ hg pull ../repo --quiet
+  $ hg heads -T '{rev} {node} {branch}\n'
+  2 222ae9789a75703f9836e44de7db179cbfd420ee default
+  1 a1602b357cfca067600406eb19060c7128804d72 default
+  $ ls -1 .hg/cache/branch?*
+  .hg/cache/branch2-base
+  .hg/cache/branch2-served
+  $ cat .hg/cache/branch?-served
+  222ae9789a75703f9836e44de7db179cbfd420ee 2
+  a1602b357cfca067600406eb19060c7128804d72 o default
+  222ae9789a75703f9836e44de7db179cbfd420ee o default
--- a/tests/test-tags.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-tags.t	Sun Mar 10 04:53:17 2024 +0100
@@ -792,11 +792,6 @@
 
   $ ls tagsclient/.hg/cache
   branch2-base
-  branch2-immutable
-  branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   hgtagsfnodes1
   rbc-names-v1
   rbc-revs-v1
@@ -823,11 +818,6 @@
 
   $ ls tagsclient/.hg/cache
   branch2-base
-  branch2-immutable
-  branch2-served
-  branch2-served.hidden
-  branch2-visible
-  branch2-visible-hidden
   hgtagsfnodes1
   rbc-names-v1
   rbc-revs-v1
--- a/tests/test-treemanifest.t	Fri Mar 08 16:49:06 2024 +0100
+++ b/tests/test-treemanifest.t	Sun Mar 10 04:53:17 2024 +0100
@@ -761,8 +761,8 @@
   $ hg clone --config experimental.changegroup3=True --stream -U \
   >   http://localhost:$HGPORT1 stream-clone-basicstore
   streaming all changes
-  29 files to transfer, * of data (glob) (no-rust !)
-  31 files to transfer, * of data (glob) (rust !)
+  24 files to transfer, * of data (glob) (no-rust !)
+  26 files to transfer, * of data (glob) (rust !)
   transferred * in * seconds (*) (glob)
   $ hg -R stream-clone-basicstore verify -q
   $ cat port-1-errors.log
@@ -771,8 +771,8 @@
   $ hg clone --config experimental.changegroup3=True --stream -U \
   >   http://localhost:$HGPORT2 stream-clone-encodedstore
   streaming all changes
-  29 files to transfer, * of data (glob) (no-rust !)
-  31 files to transfer, * of data (glob) (rust !)
+  24 files to transfer, * of data (glob) (no-rust !)
+  26 files to transfer, * of data (glob) (rust !)
   transferred * in * seconds (*) (glob)
   $ hg -R stream-clone-encodedstore verify -q
   $ cat port-2-errors.log