mercurial/dirstateguard.py

localrepo: cache types for filtered repos (issue5043)

Python introduces a reference cycle on dynamically created types
via __mro__, making them very easy to leak. See
https://bugs.python.org/issue17950.

Previously, repo.filtered() created a type on every invocation.
Long-running processes (like `hg convert`) could call this
function thousands of times, leading to a steady memory leak.

Since we're Unable to stop the leak because this is a bug in
Python, the next best thing is to contain it.

This patch adds a cache of of the dynamically generated repoview/filter
types on the localrepo object. Since we only generate each type
once, we cap the amount of memory that can leak to something
reasonable.

After this change, `hg convert` no longer leaks memory on every
revision. The process will likely grow memory usage over time due
to e.g. larger manifests. But there are no leaks.

# dirstateguard.py - class to allow restoring dirstate after failure
#
# Copyright 2005-2007 Matt Mackall <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from .i18n import _

from . import (
    error,
)

class dirstateguard(object):
    '''Restore dirstate at unexpected failure.

    At the construction, this class does:

    - write current ``repo.dirstate`` out, and
    - save ``.hg/dirstate`` into the backup file

    This restores ``.hg/dirstate`` from backup file, if ``release()``
    is invoked before ``close()``.

    This just removes the backup file at ``close()`` before ``release()``.
    '''

    def __init__(self, repo, name):
        self._repo = repo
        self._active = False
        self._closed = False
        self._suffix = '.backup.%s.%d' % (name, id(self))
        repo.dirstate.savebackup(repo.currenttransaction(), self._suffix)
        self._active = True

    def __del__(self):
        if self._active: # still active
            # this may occur, even if this class is used correctly:
            # for example, releasing other resources like transaction
            # may raise exception before ``dirstateguard.release`` in
            # ``release(tr, ....)``.
            self._abort()

    def close(self):
        if not self._active: # already inactivated
            msg = (_("can't close already inactivated backup: dirstate%s")
                   % self._suffix)
            raise error.Abort(msg)

        self._repo.dirstate.clearbackup(self._repo.currenttransaction(),
                                         self._suffix)
        self._active = False
        self._closed = True

    def _abort(self):
        self._repo.dirstate.restorebackup(self._repo.currenttransaction(),
                                           self._suffix)
        self._active = False

    def release(self):
        if not self._closed:
            if not self._active: # already inactivated
                msg = (_("can't release already inactivated backup:"
                         " dirstate%s")
                       % self._suffix)
                raise error.Abort(msg)
            self._abort()