view tests/test-obsolete-tag-cache.t @ 47120:7109a38830c9

dirstate-tree: Fold "tracked descendants" counter update in main walk

For the purpose of implementing `has_tracked_dir` (which means "has tracked
descendants) without an expensive sub-tree traversal, we maintaing a counter
of tracked descendants on each "directory" node of the tree-shaped dirstate.

Before this changeset, mutating or inserting a node at a given path would
involve:

* Walking the tree from root through ancestors to find the node or the spot
  where to insert it
* Looking at the previous node if any to decide what counter update is needed
* Performing any node mutation
* Walking the tree *again* to update counters in ancestor nodes

When profiling `hg status` on a large repo, this second walk takes times
while loading a the dirstate from disk.

It turns out we have enough information to decide before he first tree walk
what counter update is needed. This changeset merges the two walks, gaining
~10% of the total time for `hg update` (in the same hyperfine benchmark as
the previous changeset).

---

Profiling was done by compiling with this `.cargo/config`:

    [profile.release]
    debug = true

then running with:

    py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \
    ./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1

then visualizing the recorded JSON file in https://www.speedscope.app/

Differential Revision: https://phab.mercurial-scm.org/D10554

  $ cat >> $HGRCPATH << EOF
  > [extensions]
  > blackbox=
  > rebase=
  > mock=$TESTDIR/mockblackbox.py
  > 
  > [blackbox]
  > track = command, commandfinish, tagscache
  > 
  > [experimental]
  > evolution.createmarkers=True
  > EOF

Create a repo with some tags

  $ hg init repo
  $ cd repo
  $ echo initial > foo
  $ hg -q commit -A -m initial
  $ hg tag -m 'test tag' test1
  $ echo first > first
  $ hg -q commit -A -m first
  $ hg tag -m 'test2 tag' test2
  $ hg -q up -r 0
  $ echo newhead > newhead
  $ hg commit -A -m newhead
  adding newhead
  created new head
  $ hg tag -m 'test head 2 tag' head2

  $ hg log -G -T '{rev}:{node|short} {tags} {desc}\n'
  @  5:2942a772f72a tip test head 2 tag
  |
  o  4:042eb6bfcc49 head2 newhead
  |
  | o  3:c3cb30f2d2cd  test2 tag
  | |
  | o  2:d75775ffbc6b test2 first
  | |
  | o  1:5f97d42da03f  test tag
  |/
  o  0:55482a6fb4b1 test1 initial
  

Trigger tags cache population by doing something that accesses tags info

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test2                              2:d75775ffbc6b
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1
  d75775ffbc6bca1794d300f5571272879bd280da test2

Hiding a non-tip changeset should change filtered hash and cause tags recompute

  $ hg debugobsolete -d '0 0' c3cb30f2d2cd0aae008cc91a07876e3c5131fd22 -u dummyuser
  1 new obsolescence markers
  obsoleted 1 changesets

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6 f34fbc9a9769ba9eff5aff3d008a6b49f85c08b1
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1

  $ hg blackbox -l 5
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 2/2 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2-visible with 2 tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags exited 0 after * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5

Hiding another changeset should cause the filtered hash to change

  $ hg debugobsolete -d '0 0' d75775ffbc6bca1794d300f5571272879bd280da -u dummyuser
  1 new obsolescence markers
  obsoleted 1 changesets
  $ hg debugobsolete -d '0 0' 5f97d42da03fd56f3b228b03dfe48af5c0adf75b -u dummyuser
  1 new obsolescence markers
  obsoleted 1 changesets

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6 2fce1eec33263d08a4d04293960fc73a555230e4
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2

  $ hg blackbox -l 5
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 1/1 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2-visible with 1 tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags exited 0 after * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5

Resolving tags on an unfiltered repo writes a separate tags cache

  $ hg --hidden tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test2                              2:d75775ffbc6b
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2
  5 2942a772f72a444bef4bef13874d515f50fa27b6
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1
  d75775ffbc6bca1794d300f5571272879bd280da test2

  $ hg blackbox -l 5
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> --hidden tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 2/2 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2 with 3 tags
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> --hidden tags exited 0 after * seconds (glob)
  1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5