Mercurial > hg

changeset 44086:ffac09da7a19

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
lfs: avoid quadratic performance in processing server responses This is also adapted from the Facebook repo[1]. Unlike there, we were already reading the download stream in chunks and immediately writing it to disk, so we basically avoided the problem on download. There shouldn't be a lot of data to read on upload, but it's better to get rid of this pattern. [1] https://github.com/facebookexperimental/eden/commit/82df66ffe97e21f3ee73dfec093c87500fc1f6a7 Differential Revision: https://phab.mercurial-scm.org/D7882
author Matt Harbison <matt_harbison@yahoo.com>
date Tue, 14 Jan 2020 20:05:37 -0500
parents 0ee0a3f6a990
children dc9b53482689
files hgext/lfs/blobstore.py
diffstat 1 files changed, 4 insertions(+), 2 deletions(-) [+]
line wrap: on
line diff
--- a/hgext/lfs/blobstore.py	Tue Jan 14 19:42:24 2020 -0500
+++ b/hgext/lfs/blobstore.py	Tue Jan 14 20:05:37 2020 -0500
@@ -503,7 +503,6 @@
         for k, v in headers:
             request.add_header(pycompat.strurl(k), pycompat.strurl(v))
 
-        response = b''
         try:
             with contextlib.closing(self.urlopener.open(request)) as res:
                 contentlength = res.info().get(b"content-length")
@@ -520,11 +519,14 @@
                     # blobstore
                     localstore.download(oid, res, contentlength)
                 else:
+                    blocks = []
                     while True:
                         data = res.read(1048576)
                         if not data:
                             break
-                        response += data
+                        blocks.append(data)
+
+                    response = b"".join(blocks)
                     if response:
                         ui.debug(b'lfs %s response: %s' % (action, response))
         except util.urlerr.httperror as ex: