view contrib/memory.py @ 35599:af25237be091

perf: add threading capability to perfbdiff

Since we are releasing the GIL during diffing, it is interesting to see how a
thread pool would perform on diffing. We add a new `--threads` argument to
commands. Synchronizing the thread pool is a bit complex because we want to be
able to reuse it from one run to another.

On my computer (i7 with 4 cores + hyperthreading), I get the following data for
about 12000 revisions:
threads wall        comb    wall gain   comb overhead
none    31.596715   31.59    0.00%       0.00%
1       31.621228   31.62   -0.08%       0.09%
2       16.406202   32.8    48.08%       3.83%
3       11.598334   34.76   63.29%      10.03%
4        9.205421   36.77   70.87%      16.40%
5        8.517604   42.51   73.04%      34.57%
6        7.94645    47.58   74.85%      50.62%
7        7.434972   51.92   76.47%      64.36%
8        7.070638   55.34   77.62%      75.18%

Compared to the feature disabled (threads=0), the overhead is negligible with
the threading code (threads=1), and the gain is already 48% with two threads.

# memory.py - track memory usage
#
# Copyright 2009 Matt Mackall <[e-mail]> and others
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

'''helper extension to measure memory usage

Reads current and peak memory usage from ``/proc/self/status`` and
prints it to ``stderr`` on exit.
'''

from __future__ import absolute_import

def memusage(ui):
    """Report memory usage of the current process."""
    result = {'peak': 0, 'rss': 0}
    with open('/proc/self/status', 'r') as status:
        # This will only work on systems with a /proc file system
        # (like Linux).
        for line in status:
            parts = line.split()
            key = parts[0][2:-1].lower()
            if key in result:
                result[key] = int(parts[1])
    ui.write_err(", ".join(["%s: %.1f MiB" % (k, v / 1024.0)
                            for k, v in result.iteritems()]) + "\n")

def extsetup(ui):
    ui.atexit(memusage, ui)