annotate mercurial/streamclone.py @ 26610:f9f82c444ff7

filemerge: only copy to backup during premerge step The premerge might leave the original file in an unclean state. Therefore it's important to only copy the file in the beginning.
author Siddharth Agarwal <sid0@fb.com>
date Sun, 11 Oct 2015 20:04:40 -0700
parents 56b2bcea2529
children bb0b955d050d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # streamclone.py - producing and consuming streaming repository data
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 #
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 # Copyright 2015 Gregory Szorc <gregory.szorc@gmail.com>
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 #
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 from __future__ import absolute_import
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
10 import time
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
11
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
12 from .i18n import _
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 from . import (
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 branchmap,
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
15 error,
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
16 store,
26442
ef8d27f53204 streamclone: move stream_in() from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26441
diff changeset
17 util,
26441
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 )
56527b886d1d streamclone: move applystreamclone() from localrepo.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
20 def canperformstreamclone(pullop, bailifbundle2supported=False):
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
21 """Whether it is possible to perform a streaming clone as part of pull.
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
22
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
23 ``bailifbundle2supported`` will cause the function to return False if
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
24 bundle2 stream clones are supported. It should only be called by the
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
25 legacy stream clone code path.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
26
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
27 Returns a tuple of (supported, requirements). ``supported`` is True if
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
28 streaming clone is supported and False otherwise. ``requirements`` is
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
29 a set of repo requirements from the remote, or ``None`` if stream clone
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
30 isn't supported.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
31 """
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
32 repo = pullop.repo
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
33 remote = pullop.remote
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
34
26467
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
35 bundle2supported = False
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
36 if pullop.canusebundle2:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
37 if 'v1' in pullop.remotebundle2caps.get('stream', []):
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
38 bundle2supported = True
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
39 # else
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
40 # Server doesn't support bundle2 stream clone or doesn't support
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
41 # the versions we support. Fall back and possibly allow legacy.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
42
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
43 # Ensures legacy code path uses available bundle2.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
44 if bailifbundle2supported and bundle2supported:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
45 return False, None
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
46 # Ensures bundle2 doesn't try to do a stream clone if it isn't supported.
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
47 #elif not bailifbundle2supported and not bundle2supported:
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
48 # return False, None
ff2c89239d49 streamclone: teach canperformstreamclone to be bundle2 aware
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26466
diff changeset
49
26447
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
50 # Streaming clone only works on empty repositories.
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
51 if len(repo):
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
52 return False, None
591088f7028a streamclone: add explicit check for empty local repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26446
diff changeset
53
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
54 # Streaming clone only works if all data is being requested.
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
55 if pullop.heads:
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
56 return False, None
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
57
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
58 streamrequested = pullop.streamclonerequested
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
59
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
60 # If we don't have a preference, let the server decide for us. This
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
61 # likely only comes into play in LANs.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
62 if streamrequested is None:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
63 # The server can advertise whether to prefer streaming clone.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
64 streamrequested = remote.capable('stream-preferred')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
65
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
66 if not streamrequested:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
67 return False, None
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
68
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
69 # In order for stream clone to work, the client has to support all the
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
70 # requirements advertised by the server.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
71 #
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
72 # The server advertises its requirements via the "stream" and "streamreqs"
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
73 # capability. "stream" (a value-less capability) is advertised if and only
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
74 # if the only requirement is "revlogv1." Else, the "streamreqs" capability
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
75 # is advertised and contains a comma-delimited list of requirements.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
76 requirements = set()
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
77 if remote.capable('stream'):
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
78 requirements.add('revlogv1')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
79 else:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
80 streamreqs = remote.capable('streamreqs')
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
81 # This is weird and shouldn't happen with modern servers.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
82 if not streamreqs:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
83 return False, None
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
84
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
85 streamreqs = set(streamreqs.split(','))
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
86 # Server requires something we don't support. Bail.
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
87 if streamreqs - repo.supportedformats:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
88 return False, None
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
89 requirements = streamreqs
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
90
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
91 return True, requirements
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
92
26462
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
93 def maybeperformlegacystreamclone(pullop):
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
94 """Possibly perform a legacy stream clone operation.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
95
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
96 Legacy stream clones are performed as part of pull but before all other
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
97 operations.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
98
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
99 A legacy stream clone will not be performed if a bundle2 stream clone is
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
100 supported.
3b0ec09192ae streamclone: rename and document maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26461
diff changeset
101 """
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
102 supported, requirements = canperformstreamclone(pullop)
26458
362793295640 streamclone: refactor maybeperformstreamclone to take a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26447
diff changeset
103
26446
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
104 if not supported:
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
105 return
3ea10bb761ce streamclone: refactor code for deciding to stream clone
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26445
diff changeset
106
26466
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
107 repo = pullop.repo
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
108 remote = pullop.remote
3515db5aae05 streamclone: refactor canperformstreamclone to accept a pullop
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26462
diff changeset
109
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
110 # Save remote branchmap. We will use it later to speed up branchcache
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
111 # creation.
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
112 rbranchmap = None
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
113 if remote.capable('branchmap'):
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
114 rbranchmap = remote.branchmap()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
115
26470
4b5647d9ee13 streamclone: move "streaming all changes" message location
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26469
diff changeset
116 repo.ui.status(_('streaming all changes\n'))
4b5647d9ee13 streamclone: move "streaming all changes" message location
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26469
diff changeset
117
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
118 fp = remote.stream_out()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
119 l = fp.readline()
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
120 try:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
121 resp = int(l)
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
122 except ValueError:
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
123 raise error.ResponseError(
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
124 _('unexpected response from remote server:'), l)
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
125 if resp == 1:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26470
diff changeset
126 raise error.Abort(_('operation forbidden by server'))
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
127 elif resp == 2:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26470
diff changeset
128 raise error.Abort(_('locking the remote repository failed'))
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
129 elif resp != 0:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26470
diff changeset
130 raise error.Abort(_('the server sent an unknown error code'))
26459
3b28ffde133a streamclone: move streamin() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26458
diff changeset
131
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
132 l = fp.readline()
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
133 try:
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
134 filecount, bytecount = map(int, l.split(' ', 1))
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
135 except (ValueError, TypeError):
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
136 raise error.ResponseError(
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
137 _('unexpected response from remote server:'), l)
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
138
26461
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
139 lock = repo.lock()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
140 try:
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
141 consumev1(repo, fp, filecount, bytecount)
26461
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
142
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
143 # new requirements = old non-format requirements +
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
144 # new format-related remote requirements
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
145 # requirements from the streamed-in repository
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
146 repo.requirements = requirements | (
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
147 repo.requirements - repo.supportedformats)
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
148 repo._applyopenerreqs()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
149 repo._writerequirements()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
150
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
151 if rbranchmap:
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
152 branchmap.replacecache(repo, rbranchmap)
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
153
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
154 repo.invalidate()
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
155 finally:
09cc3c2e9ece streamclone: move applyremotedata() into maybeperformstreamclone()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26460
diff changeset
156 lock.release()
26445
f134fb33c906 streamclone: move streaming clone logic from localrepo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26444
diff changeset
157
26444
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
158 def allowservergeneration(ui):
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
159 """Whether streaming clones are allowed from the server."""
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
160 return ui.configbool('server', 'uncompressed', True, untrusted=True)
623743010133 streamclone: move _allowstream() from wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26443
diff changeset
161
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
162 # This is it's own function so extensions can override it.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
163 def _walkstreamfiles(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
164 return repo.store.walk()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
165
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
166 def generatev1(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
167 """Emit content for version 1 of a streaming clone.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
168
26469
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
169 This returns a 3-tuple of (file count, byte size, data iterator).
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
170
26469
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
171 The data iterator consists of N entries for each file being transferred.
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
172 Each file entry starts as a line with the file name and integer size
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
173 delimited by a null byte.
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
174
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
175 The raw file data follows. Following the raw file data is the next file
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
176 entry, or EOF.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
177
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
178 When used on the wire protocol, an additional line indicating protocol
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
179 success will be prepended to the stream. This function is not responsible
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
180 for adding it.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
181
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
182 This function will obtain a repository lock to ensure a consistent view of
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
183 the store is captured. It therefore may raise LockError.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
184 """
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
185 entries = []
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
186 total_bytes = 0
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
187 # Get consistent snapshot of repo, lock during scan.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
188 lock = repo.lock()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
189 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
190 repo.ui.debug('scanning\n')
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
191 for name, ename, size in _walkstreamfiles(repo):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
192 if size:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
193 entries.append((name, size))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
194 total_bytes += size
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
195 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
196 lock.release()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
197
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
198 repo.ui.debug('%d files, %d bytes to transfer\n' %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
199 (len(entries), total_bytes))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
200
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
201 svfs = repo.svfs
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
202 oldaudit = svfs.mustaudit
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
203 debugflag = repo.ui.debugflag
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
204 svfs.mustaudit = False
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
205
26469
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
206 def emitrevlogdata():
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
207 try:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
208 for name, size in entries:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
209 if debugflag:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
210 repo.ui.debug('sending %s (%d bytes)\n' % (name, size))
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
211 # partially encode name over the wire for backwards compat
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
212 yield '%s\0%d\n' % (store.encodedir(name), size)
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
213 if size <= 65536:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
214 fp = svfs(name)
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
215 try:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
216 data = fp.read(size)
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
217 finally:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
218 fp.close()
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
219 yield data
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
220 else:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
221 for chunk in util.filechunkiter(svfs(name), limit=size):
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
222 yield chunk
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
223 finally:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
224 svfs.mustaudit = oldaudit
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
225
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
226 return len(entries), total_bytes, emitrevlogdata()
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
227
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
228 def generatev1wireproto(repo):
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
229 """Emit content for version 1 of streaming clone suitable for the wire.
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
230
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
231 This is the data output from ``generatev1()`` with a header line
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
232 indicating file count and byte size.
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
233 """
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
234 filecount, bytecount, it = generatev1(repo)
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
235 yield '%d %d\n' % (filecount, bytecount)
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
236 for chunk in it:
fb743268510e streamclone: move payload header generation into own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26468
diff changeset
237 yield chunk
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
238
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
239 def consumev1(repo, fp, filecount, bytecount):
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
240 """Apply the contents from version 1 of a streaming clone file handle.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
241
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
242 This takes the output from "streamout" and applies it to the specified
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
243 repository.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
244
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
245 Like "streamout," the status line added by the wire protocol is not handled
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
246 by this function.
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
247 """
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
248 lock = repo.lock()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
249 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
250 repo.ui.status(_('%d files to transfer, %s of data\n') %
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
251 (filecount, util.bytecount(bytecount)))
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
252 handled_bytes = 0
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
253 repo.ui.progress(_('clone'), 0, total=bytecount)
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
254 start = time.time()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
255
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
256 tr = repo.transaction(_('clone'))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
257 try:
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
258 for i in xrange(filecount):
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
259 # XXX doesn't support '\n' or '\r' in filenames
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
260 l = fp.readline()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
261 try:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
262 name, size = l.split('\0', 1)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
263 size = int(size)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
264 except (ValueError, TypeError):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
265 raise error.ResponseError(
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
266 _('unexpected response from remote server:'), l)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
267 if repo.ui.debugflag:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
268 repo.ui.debug('adding %s (%s)\n' %
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
269 (name, util.bytecount(size)))
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
270 # for backwards compat, name was partially encoded
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
271 ofp = repo.svfs(store.decodedir(name), 'w')
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
272 for chunk in util.filechunkiter(fp, limit=size):
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
273 handled_bytes += len(chunk)
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
274 repo.ui.progress(_('clone'), handled_bytes, total=bytecount)
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
275 ofp.write(chunk)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
276 ofp.close()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
277 tr.close()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
278 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
279 tr.release()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
280
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
281 # Writing straight to files circumvented the inmemory caches
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
282 repo.invalidate()
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
283
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
284 elapsed = time.time() - start
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
285 if elapsed <= 0:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
286 elapsed = 0.001
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
287 repo.ui.progress(_('clone'), None)
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
288 repo.ui.status(_('transferred %s in %.1f seconds (%s/sec)\n') %
26468
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
289 (util.bytecount(bytecount), elapsed,
19bbd53af46d streamclone: move payload header line consumption
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26467
diff changeset
290 util.bytecount(bytecount / elapsed)))
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
291 finally:
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26442
diff changeset
292 lock.release()