Gregory Szorc <gregory.szorc@gmail.com> [Sat, 01 Jul 2017 20:51:19 -0700] rev 33389
localrepo: cache types for filtered repos (issue5043)
Python introduces a reference cycle on dynamically created types
via __mro__, making them very easy to leak. See
https://bugs.python.org/issue17950.
Previously, repo.filtered() created a type on every invocation.
Long-running processes (like `hg convert`) could call this
function thousands of times, leading to a steady memory leak.
Since we're Unable to stop the leak because this is a bug in
Python, the next best thing is to contain it.
This patch adds a cache of of the dynamically generated repoview/filter
types on the localrepo object. Since we only generate each type
once, we cap the amount of memory that can leak to something
reasonable.
After this change, `hg convert` no longer leaks memory on every
revision. The process will likely grow memory usage over time due
to e.g. larger manifests. But there are no leaks.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 11 Jul 2017 02:10:04 +0900] rev 33388
convert: transcode CVS log messages by specified encoding (issue5597)
Converting from CVS to Mercurial assumes that CVS log messages in "cvs
rlog" output are encoded in UTF-8 (or basic Latin-1). But cvs itself
is usually unaware of encoding of log messages, in practice.
Therefore, if there are commits, of which log message is encoded in
other than UTF-8, log message of corresponded revisions in the
converted repository will be broken.
To avoid such broken log messages, this patch transcodes CVS log
messages by encoding specified via "convert.cvsps.logencoding"
configuration.
This patch accepts multiple encoding for convenience, because
"multiple encoding mixed in a repository" easily occurs. For example,
UTF-8 (recent POSIX), cp932 (Windows), and EUC-JP (legacy POSIX) are
well known encoding for Japanese.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Mon, 10 Jul 2017 23:09:52 +0900] rev 33387
fsmonitor: execute setup procedures only if dirstate is already instantiated
Before this patch, reposetup() of fsmonitor executes setup procedures
for dirstate, even if it isn't yet instantiated at that time.
On the other hand, dirstate might be already instantiated before
reposetup() intentionally (prefilling by chg, for example, see
bf3af0eced44 for detail). If so, just discarding already instantiated
one in reposetup() causes issue.
To resolve both issues above, this patch executes setup procedures,
only if dirstate is already instantiated.
BTW, this patch removes "del repo.unfiltered().__dict__['dirstate']",
because it is responsibility of the code path, which causes
instantiation of dirstate before reposetup(). After this patch, using
localrepo.isfilecached() should avoid creating the corresponded entry
in repo.unfiltered().__dict__.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Mon, 10 Jul 2017 23:09:52 +0900] rev 33386
fsmonitor: centralize setup procedures for dirstate
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Mon, 10 Jul 2017 23:09:52 +0900] rev 33385
fsmonitor: avoid needless instantiation of dirstate
Using repo.local() instead of util.safehasattr(repo, 'dirstate') also
avoids executing setup procedures for remote repository (including
statichttprepo).
This is reason why this patch also removes a part of subsequent
comment, and try/except for AttributeError at accessing to repo.wvfs.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Mon, 10 Jul 2017 23:09:51 +0900] rev 33384
journal: use wrapfilecache instead of wrapfunction on func of filecache
wrapfilecache() on filecache-ed property works more strictly than
wrapfunction() directly on func() of filecache.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Mon, 10 Jul 2017 23:09:51 +0900] rev 33383
journal: execute setup procedures for already instantiated dirstate
If dirstate is instantiated before reposetup() of journal extension,
it doesn't have "journalstorage" property, even if it is instantiated
via wrapdirstate() wrapping repo.dirstate(), because wrapdirstate()
works as same as original one before marking repo as "journal"-ing in
reposetup().
This issue can be reproduced by running test-journal.t or
test-journal-share.t with fsmonitor-run-tests.py.
On the other hand, just discarding already instantiated dirstate in
reposetup() prevents chg from filling dirstate before reposetup() (see
bf3af0eced44 for detail).
Therefore, this patch executes setup procedures for already
instantiated dirstate explicitly in reposetup().
To centralize setup procedures for dirstate, this patch also factors
them out from wrapdirstate().