contrib/memory.py
author Jun Wu <quark@fb.com>
Sat, 22 Apr 2017 16:50:08 -0700
changeset 32112 31763785094b
parent 31958 de5c9d0e02ea
child 43076 2372284d9457
permissions -rw-r--r--
worker: rewrite error handling so os._exit covers all cases Previously the worker error handling is like: pid = os.fork() --+ if pid == 0: | .... | problematic .... --+ try: --+ .... | worker error handling --+ If a signal arrives when Python is executing the "problematic" lines, an external error handling (dispatch.py) will take over the control flow and it's no longer guaranteed "os._exit" is called (see 86cd09bc13ba for why it is necessary). This patch rewrites the error handling so it covers all possible code paths for a worker even during fork. Note: "os.getpid() == parentpid" is used to test if the process is parent or not intentionally, instead of checking "pid", because "pid = os.fork()" may be not atomic - it's possible that that a signal hits the worker before the assignment completes [1]. The newly added test replaces "os.fork" to exercise that extreme case. [1]: CPython compiles "pid = os.fork()" to 2 byte codes: "CALL_FUNCTION" and "STORE_FAST", so it's probably not atomic: def f(): pid = os.fork() dis.dis(f) 2 0 LOAD_GLOBAL 0 (os) 3 LOAD_ATTR 1 (fork) 6 CALL_FUNCTION 0 9 STORE_FAST 0 (pid) 12 LOAD_CONST 0 (None) 15 RETURN_VALUE
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     1
# memory.py - track memory usage
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     2
#
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     3
# Copyright 2009 Matt Mackall <mpm@selenic.com> and others
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     4
#
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
10264
d6512b3e9ac0 Merge with stable
Matt Mackall <mpm@selenic.com>
parents: 10017
diff changeset
     6
# GNU General Public License version 2 or any later version.
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     7
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     8
'''helper extension to measure memory usage
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     9
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    10
Reads current and peak memory usage from ``/proc/self/status`` and
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    11
prints it to ``stderr`` on exit.
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    12
'''
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    13
28510
ade330deb39a contrib: make memory.py use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 27795
diff changeset
    14
from __future__ import absolute_import
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    15
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    16
def memusage(ui):
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    17
    """Report memory usage of the current process."""
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    18
    result = {'peak': 0, 'rss': 0}
27795
3e0d27d298b7 with: use context manager for file I/O in memusage
Bryan O'Sullivan <bryano@fb.com>
parents: 10282
diff changeset
    19
    with open('/proc/self/status', 'r') as status:
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    20
        # This will only work on systems with a /proc file system
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    21
        # (like Linux).
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    22
        for line in status:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    23
            parts = line.split()
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    24
            key = parts[0][2:-1].lower()
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    25
            if key in result:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    26
                result[key] = int(parts[1])
30386
ff896733c66a memory: avoid shadowing variables inside a list comprehension
Augie Fackler <augie@google.com>
parents: 28510
diff changeset
    27
    ui.write_err(", ".join(["%s: %.1f MiB" % (k, v / 1024.0)
ff896733c66a memory: avoid shadowing variables inside a list comprehension
Augie Fackler <augie@google.com>
parents: 28510
diff changeset
    28
                            for k, v in result.iteritems()]) + "\n")
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    29
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    30
def extsetup(ui):
31958
de5c9d0e02ea atexit: switch to home-grown implementation
Bryan O'Sullivan <bryano@fb.com>
parents: 30386
diff changeset
    31
    ui.atexit(memusage, ui)