view tests/test-fastannotate-corrupt.t @ 47120:7109a38830c9

dirstate-tree: Fold "tracked descendants" counter update in main walk

For the purpose of implementing `has_tracked_dir` (which means "has tracked
descendants) without an expensive sub-tree traversal, we maintaing a counter
of tracked descendants on each "directory" node of the tree-shaped dirstate.

Before this changeset, mutating or inserting a node at a given path would
involve:

* Walking the tree from root through ancestors to find the node or the spot
  where to insert it
* Looking at the previous node if any to decide what counter update is needed
* Performing any node mutation
* Walking the tree *again* to update counters in ancestor nodes

When profiling `hg status` on a large repo, this second walk takes times
while loading a the dirstate from disk.

It turns out we have enough information to decide before he first tree walk
what counter update is needed. This changeset merges the two walks, gaining
~10% of the total time for `hg update` (in the same hyperfine benchmark as
the previous changeset).

---

Profiling was done by compiling with this `.cargo/config`:

    [profile.release]
    debug = true

then running with:

    py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \
    ./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1

then visualizing the recorded JSON file in https://www.speedscope.app/

Differential Revision: https://phab.mercurial-scm.org/D10554

  $ cat >> $HGRCPATH << EOF
  > [extensions]
  > fastannotate=
  > EOF

  $ hg init repo
  $ cd repo
  $ for i in 0 1 2 3 4; do
  >   echo $i >> a
  >   echo $i >> b
  >   hg commit -A -m $i a b
  > done

use the "debugbuildannotatecache" command to build annotate cache at rev 0

  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=0
  fastannotate: a: 1 new changesets in the main branch
  fastannotate: b: 1 new changesets in the main branch

"debugbuildannotatecache" should work with broken cache (and other files would
be built without being affected). note: linelog being broken is only noticed
when we try to append to it.

  $ echo 'CORRUPT!' >> .hg/fastannotate/default/a.m
  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=1
  fastannotate: a: rebuilding broken cache
  fastannotate: a: 2 new changesets in the main branch
  fastannotate: b: 1 new changesets in the main branch

  $ echo 'CANNOT REUSE!' > .hg/fastannotate/default/a.l
  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=2
  fastannotate: a: rebuilding broken cache
  fastannotate: a: 3 new changesets in the main branch
  fastannotate: b: 1 new changesets in the main branch

  $ rm .hg/fastannotate/default/a.m
  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=3
  fastannotate: a: rebuilding broken cache
  fastannotate: a: 4 new changesets in the main branch
  fastannotate: b: 1 new changesets in the main branch

  $ rm .hg/fastannotate/default/a.l
  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=3
  $ hg debugbuildannotatecache --debug --config fastannotate.mainbranch=4
  fastannotate: a: rebuilding broken cache
  fastannotate: a: 5 new changesets in the main branch
  fastannotate: b: 1 new changesets in the main branch

"fastannotate" should deal with file corruption as well

  $ rm -rf .hg/fastannotate
  $ hg fastannotate --debug -r 0 a
  fastannotate: a: 1 new changesets in the main branch
  0: 0

  $ echo 'CORRUPT!' >> .hg/fastannotate/default/a.m
  $ hg fastannotate --debug -r 0 a
  fastannotate: a: cache broken and deleted
  fastannotate: a: 1 new changesets in the main branch
  0: 0

  $ echo 'CORRUPT!' > .hg/fastannotate/default/a.l
  $ hg fastannotate --debug -r 1 a
  fastannotate: a: cache broken and deleted
  fastannotate: a: 2 new changesets in the main branch
  0: 0
  1: 1

  $ rm .hg/fastannotate/default/a.l
  $ hg fastannotate --debug -r 1 a
  fastannotate: a: using fast path (resolved fctx: True)
  fastannotate: a: cache broken and deleted
  fastannotate: a: 2 new changesets in the main branch
  0: 0
  1: 1

  $ rm .hg/fastannotate/default/a.m
  $ hg fastannotate --debug -r 2 a
  fastannotate: a: cache broken and deleted
  fastannotate: a: 3 new changesets in the main branch
  0: 0
  1: 1
  2: 2