verify: check for orphaned dirlogs
We already report orphaned filelogs, i.e. revlogs for files that are
not mentioned in any manifest. This change adds checking for orphaned
dirlogs, i.e. revlogs that are not mentioned in any parent-directory
dirlog.
Note that, for fncachestore, only files mentioned in the fncache are
considered, there's not check for files in .hg/store/meta that are not
mentioned in the fncache. This is no different from the current
situation for filelogs.
verify: check directory manifests
In repos with treemanifests, there is no specific verification of
directory manifest revlogs. It simply collects all file nodes by
reading each manifest delta. With treemanifests, that's means calling
the manifest._slowreaddelta(). If there are missing revlog entries in
a subdirectory revlog, 'hg verify' will simply report the exception
that occurred while trying to read the root manifest:
manifest@0: reading delta
1700e2e92882: meta/b/00manifest.i@
67688a370455: no node
This patch changes the verify code to load only the root manifest at
first and verify all revisions of it, then verify all revisions of
each direct subdirectory, and so on, recursively. The above message
becomes
b/@0: parent-directory manifest refers to unknown revision
67688a370455
Since the new algorithm reads a single revlog at a time and in order,
'hg verify' on a treemanifest version of the hg core repo goes from
~50s to ~14s. As expected, there is no significant difference on a
repo with flat manifests.
hg: perform update after pulling during clone with share (
issue5103)
When pooled storage is enabled, `hg clone` will initialize a repo
from a local repo using the store sharing mechanism then pull from
the originally requested repo.
Before this patch, the working directory update occurred between
these steps. This meant that we would only update to revisions that
were already present in the local pooled storage.
This patch moves the update to after we pull from the originally
requested repository so we may check out a revision that didn't yet
exist locally. In other words, it makes the behavior like normal
`hg clone`.
hg: extract post share update logic into own function
A future patch will introduce a new caller that needs to perform
an update. Extract the code so we don't duplicate it.
merge: perform background file closing in batchget
As
2fdbf22a1b63 demonstrated with stream clones, closing files on
background threads on Windows can yield a significant speedup
because closing files that have been created/appended to is slow
on Windows/NTFS.
Working directory updates can write thousands of files. Therefore it
is susceptible to excessive slowness on Windows due to slow file
closes.
This patch enables background file closing when performing working
directory file writes. The impact when performing an `hg up tip` on
mozilla-central (136,357 files) from an empty working directory is
significant:
Before: 535s (8:55)
After: 133s (2:13)
Delta: -402s (6:42)
That's a 4x speedup!
By comparison, that same machine can perform the same operation
in ~15s on Linux. So Windows went from ~35x to ~9x slower. Not bad
but there's still work to do.
As a reminder, background file closing is only activated on Windows
because it is only beneficial on that platform. So this patch
shouldn't change non-Windows behavior at all.
It's worth noting that non-Windows systems perform working directory
updates with multiple processes. Unfortunately, worker.py doesn't
yet support Windows. So, there is still plenty of room for making
working directory updates faster on Windows. Even if multiple
processes are used on Windows, I believe background file closing
will still provide a benefit, as individual processes will still
be slowed down by the file close bottleneck (assuming the I/O system
isn't saturated).
localrepo: support background closing for wwrite()
So working copy update can pass it in.
scmutil: support background closing for write()
Upcoming patches will add background file closer support to
working copy update. This patch adds some plumbing to prepare
for that.
chg: hold a lock file before connected to server
This is a part of the one server per config series. In multiple-server setup,
multiple clients may try to start different servers (on demand) at the same
time. The old lock will not guarantee a client to connect to the server it
just started, and is not crash friendly.
This patch addressed above issues by using flock and does not release the lock
until the client actually connects to the server or times out.
serve: allow --daemon-postexec to be 'unlink:path' or 'none'
This patch changes the format of value of --daemon-postexec. Now it can be
either to unlink a file, or a no-op. The following patch will make chg to
use the no-op option.