Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:51:32 -0800] rev 18639
merge: apply non-interactive working dir updates in parallel
This has a big effect on the performance of working dir updates.
Here are the results of update from null to the given rev in several
repos, on a Linux 3.2 system with 32 cores running ext4, with the progress
extension enabled.
repo rev plain parallel speedup
hg 7068089c95a2 0.9 0.3 3
mozilla-central fe1600b22c77 42.8 7.7 5.5
linux-2.6 9ef4b770e069 31.4 4.9 6.4
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:51:32 -0800] rev 18638
worker: allow a function to be run in multiple worker processes
If we estimate that it will be worth the cost, we run the function in
multiple processes. Otherwise, we run it in-process.
Children report progress to the parent through a pipe.
Not yet implemented on Windows.
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:51:32 -0800] rev 18637
worker: partition a list (of tasks) into equal-sized chunks
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:51:26 -0800] rev 18636
worker: estimate whether it's worth running a task in parallel
Not implemented for Windows yet.
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:22:12 -0800] rev 18635
worker: count the number of CPUs
This works on the major platforms, and falls back to a safe guess of
1 elsewhere.
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:22:10 -0800] rev 18634
tests: getremove test output changes (fold into previous patch)
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:22:09 -0800] rev 18633
merge: report non-interactive progress in chunks
Instead of a monotonic count, getupdates yields the number of files it
has updated since it last reported, and its caller sums the numbers when
updating progress.
Once we run these updates in parallel, this will allow worker processes
to report progress less often, reducing overhead.
Bryan O'Sullivan <bryano@fb.com> [Sat, 09 Feb 2013 15:22:08 -0800] rev 18632
merge: handle subrepo merges and .hgsubstate specially
In an upcoming patch, we will update .hgsubstate in a non-interactive
worker process. Merges of subrepo contents will still need to occur in the
master process (since they may be interactive), so we move that code into
a place where it will always run in what will become the master process.