Mercurial > hg
changeset 25882:97a9f7602014 stable
convert: when getting file from Perforce concatenate data at the end
As it turned out, even when getting relatively small files, concatenating
string data every time when new chunk is received is very inefficient.
Maintaining a string list of data chunks and concatenating everything in one go
at the end seems much more efficient - in my testing it made getting 40 MB file
7 times faster, whilst converting of a particularly big changelist with some big
files went down from 20 hours to 3 hours.
| author | Eugene Baranov <eug.baranov@gmail.com> |
|---|---|
| date | Thu, 30 Jul 2015 00:58:05 +0100 |
| parents | 9de443515f1d |
| children | b810b59eca62 |
| files | hgext/convert/p4.py |
| diffstat | 1 files changed, 4 insertions(+), 2 deletions(-) [+] |
line wrap: on
line diff
--- a/hgext/convert/p4.py Sat Jul 18 17:10:28 2015 -0700 +++ b/hgext/convert/p4.py Thu Jul 30 00:58:05 2015 +0100 @@ -216,7 +216,7 @@ stdout = util.popen(cmd, mode='rb') mode = None - contents = "" + contents = [] keywords = None for d in loaditer(stdout): @@ -252,7 +252,7 @@ keywords = self.re_keywords elif code == "text" or code == "binary": - contents += data + contents.append(data) lasterror = None @@ -262,6 +262,8 @@ if mode is None: return None, None + contents = ''.join(contents) + if keywords: contents = keywords.sub("$\\1$", contents) if mode == "l" and contents.endswith("\n"):