view tests/test-racy-mutations.t @ 47120:7109a38830c9

dirstate-tree: Fold "tracked descendants" counter update in main walk

For the purpose of implementing `has_tracked_dir` (which means "has tracked
descendants) without an expensive sub-tree traversal, we maintaing a counter
of tracked descendants on each "directory" node of the tree-shaped dirstate.

Before this changeset, mutating or inserting a node at a given path would
involve:

* Walking the tree from root through ancestors to find the node or the spot
  where to insert it
* Looking at the previous node if any to decide what counter update is needed
* Performing any node mutation
* Walking the tree *again* to update counters in ancestor nodes

When profiling `hg status` on a large repo, this second walk takes times
while loading a the dirstate from disk.

It turns out we have enough information to decide before he first tree walk
what counter update is needed. This changeset merges the two walks, gaining
~10% of the total time for `hg update` (in the same hyperfine benchmark as
the previous changeset).

---

Profiling was done by compiling with this `.cargo/config`:

    [profile.release]
    debug = true

then running with:

    py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \
    ./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1

then visualizing the recorded JSON file in https://www.speedscope.app/

Differential Revision: https://phab.mercurial-scm.org/D10554

#testcases skip-detection fail-if-detected

Test situations that "should" only be reproducible:
- on networked filesystems, or
- user using `hg debuglocks` to eliminate the lock file, or
- something (that doesn't respect the lock file) writing to the .hg directory
while we're running

  $ hg init a
  $ cd a

  $ cat > "$TESTTMP/waitlock_editor.sh" <<EOF
  >     [ -n "\${WAITLOCK_ANNOUNCE:-}" ] && touch "\${WAITLOCK_ANNOUNCE}"
  >     f="\${WAITLOCK_FILE}"
  >     start=\`date +%s\`
  >     timeout=5
  >     while [ \\( ! -f \$f \\) -a \\( ! -L \$f \\) ]; do
  >         now=\`date +%s\`
  >         if [ "\`expr \$now - \$start\`" -gt \$timeout ]; then
  >             echo "timeout: \$f was not created in \$timeout seconds (it is now \$(date +%s))"
  >             exit 1
  >         fi
  >         sleep 0.1
  >     done
  >     if [ \$# -gt 1 ]; then
  >         cat "\$@"
  >     fi
  > EOF
  $ chmod +x "$TESTTMP/waitlock_editor.sh"

Things behave differently if we don't already have a 00changelog.i file when
this all starts, so let's make one.

  $ echo r0 > r0
  $ hg commit -qAm 'r0'

Start an hg commit that will take a while
  $ EDITOR_STARTED="$(pwd)/.editor_started"
  $ MISCHIEF_MANAGED="$(pwd)/.mischief_managed"
  $ JOBS_FINISHED="$(pwd)/.jobs_finished"

#if fail-if-detected
  $ cat >> .hg/hgrc << EOF
  > [debug]
  > revlog.verifyposition.changelog = fail
  > EOF
#endif

  $ echo foo > foo
  $ (WAITLOCK_ANNOUNCE="${EDITOR_STARTED}" \
  >      WAITLOCK_FILE="${MISCHIEF_MANAGED}" \
  >           HGEDITOR="$TESTTMP/waitlock_editor.sh" \
  >           hg commit -qAm 'r1 (foo)' --edit foo > .foo_commit_out 2>&1 ; touch "${JOBS_FINISHED}") &

Wait for the "editor" to actually start
  $ WAITLOCK_FILE="${EDITOR_STARTED}" "$TESTTMP/waitlock_editor.sh"

Break the locks, and make another commit.
  $ hg debuglocks -LW
  $ echo bar > bar
  $ hg commit -qAm 'r2 (bar)' bar
  $ hg debugrevlogindex -c
     rev linkrev nodeid       p1           p2
       0       0 222799e2f90b 000000000000 000000000000
       1       1 6f124f6007a0 222799e2f90b 000000000000

Awaken the editor from that first commit
  $ touch "${MISCHIEF_MANAGED}"
And wait for it to finish
  $ WAITLOCK_FILE="${JOBS_FINISHED}" "$TESTTMP/waitlock_editor.sh"

#if skip-detection
(Ensure there was no output)
  $ cat .foo_commit_out
And observe a corrupted repository -- rev 2's linkrev is 1, which should never
happen for the changelog (the linkrev should always refer to itself).
  $ hg debugrevlogindex -c
     rev linkrev nodeid       p1           p2
       0       0 222799e2f90b 000000000000 000000000000
       1       1 6f124f6007a0 222799e2f90b 000000000000
       2       1 ac80e6205bb2 222799e2f90b 000000000000
#endif

#if fail-if-detected
  $ cat .foo_commit_out
  transaction abort!
  rollback completed
  note: commit message saved in .hg/last-message.txt
  note: use 'hg commit --logfile .hg/last-message.txt --edit' to reuse it
  abort: 00changelog.i: file cursor at position 249, expected 121
And no corruption in the changelog.
  $ hg debugrevlogindex -c
     rev linkrev nodeid       p1           p2
       0       0 222799e2f90b 000000000000 000000000000
       1       1 6f124f6007a0 222799e2f90b 000000000000
And, because of transactions, there's none in the manifestlog either.
  $ hg debugrevlogindex -m
     rev linkrev nodeid       p1           p2
       0       0 7b7020262a56 000000000000 000000000000
       1       1 ad3fe36d86d9 7b7020262a56 000000000000
#endif