dirstate-tree: Fold "tracked descendants" counter update in main walk
For the purpose of implementing `has_tracked_dir` (which means "has tracked
descendants) without an expensive sub-tree traversal, we maintaing a counter
of tracked descendants on each "directory" node of the tree-shaped dirstate.
Before this changeset, mutating or inserting a node at a given path would
involve:
* Walking the tree from root through ancestors to find the node or the spot
where to insert it
* Looking at the previous node if any to decide what counter update is needed
* Performing any node mutation
* Walking the tree *again* to update counters in ancestor nodes
When profiling `hg status` on a large repo, this second walk takes times
while loading a the dirstate from disk.
It turns out we have enough information to decide before he first tree walk
what counter update is needed. This changeset merges the two walks, gaining
~10% of the total time for `hg update` (in the same hyperfine benchmark as
the previous changeset).
---
Profiling was done by compiling with this `.cargo/config`:
[profile.release]
debug = true
then running with:
py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \
./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1
then visualizing the recorded JSON file in https://www.speedscope.app/
Differential Revision: https://phab.mercurial-scm.org/D10554
#require svn svn-bindings
$ filter_svn_output () {
> egrep -v 'Committing|Updating|(^$)' | sed -e 's/done$//' || true
> }
$ cat <<EOF >> $HGRCPATH
> [extensions]
> convert =
> mq =
> EOF
$ SVNREPOPATH=`pwd`/svn-repo
$ SVNREPOURL="`"$PYTHON" $TESTDIR/svnurlof.py \"$SVNREPOPATH\"`"
$ svnadmin create "$SVNREPOPATH"
$ cat > "$SVNREPOPATH"/hooks/pre-revprop-change <<EOF
> #!/bin/sh
>
> REPOS="$1"
> REV="$2"
> USER="$3"
> PROPNAME="$4"
> ACTION="$5"
>
> if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi
> if [ "$ACTION" = "A" -a "$PROPNAME" = "hg:convert-branch" ]; then exit 0; fi
> if [ "$ACTION" = "A" -a "$PROPNAME" = "hg:convert-rev" ]; then exit 0; fi
>
> echo "Changing prohibited revision property" >&2
> exit 1
> EOF
$ chmod +x "$SVNREPOPATH"/hooks/pre-revprop-change
$ svn co "$SVNREPOURL" "$SVNREPOPATH"-wc
Checked out revision 0.
$ cd "$SVNREPOPATH"-wc
$ echo a > a
$ svn add a
A a
$ svn ci -m'added a' a | filter_svn_output
Adding a
Transmitting file data .
Committed revision 1.
$ cd ..
initial roundtrip
$ hg convert -s svn -d hg "$SVNREPOPATH"-wc "$SVNREPOPATH"-hg | grep -v initializing
scanning source...
sorting...
converting...
0 added a
$ hg convert -s hg -d svn "$SVNREPOPATH"-hg "$SVNREPOPATH"-wc
scanning source...
sorting...
converting...
second roundtrip should do nothing
$ hg convert -s svn -d hg "$SVNREPOPATH"-wc "$SVNREPOPATH"-hg
scanning source...
sorting...
converting...
$ hg convert -s hg -d svn "$SVNREPOPATH"-hg "$SVNREPOPATH"-wc
scanning source...
sorting...
converting...
new hg rev
$ hg clone "$SVNREPOPATH"-hg "$SVNREPOPATH"-work
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd "$SVNREPOPATH"-work
$ echo b > b
$ hg add b
$ hg ci -mb
adding an empty revision
$ hg qnew -m emtpy empty
$ hg qfinish -a
$ cd ..
echo hg to svn
$ hg --cwd "$SVNREPOPATH"-hg pull -q "$SVNREPOPATH"-work
$ hg convert -s hg -d svn "$SVNREPOPATH"-hg "$SVNREPOPATH"-wc
scanning source...
sorting...
converting...
1 b
0 emtpy
svn back to hg should do nothing
$ hg convert -s svn -d hg "$SVNREPOPATH"-wc "$SVNREPOPATH"-hg
scanning source...
sorting...
converting...
hg back to svn should do nothing
$ hg convert -s hg -d svn "$SVNREPOPATH"-hg "$SVNREPOPATH"-wc
scanning source...
sorting...
converting...
verify which shamap format we are storing and must be able to handle
$ cat svn-repo-hg/.hg/shamap
svn:????????-????-????-????-????????????@1 ???????????????????????????????????????? (glob)
svn:????????-????-????-????-????????????@2 ???????????????????????????????????????? (glob)
svn:????????-????-????-????-????????????@2 ???????????????????????????????????????? (glob)
$ cat svn-repo-wc/.svn/hg-shamap
???????????????????????????????????????? 1 (glob)
???????????????????????????????????????? svn:????????-????-????-????-????????????@2 (glob)
???????????????????????????????????????? svn:????????-????-????-????-????????????@2 (glob)