Mercurial > hg
annotate mercurial/exchangev2.py @ 40327:55836a34f41b
exchangev2: recognize narrow patterns when pulling
pulloperation instances were recently taught to record file
include and exclude patterns to facilitate narrow file transfer.
Teaching the exchangev2 code to transfer a subset of files is
as simple as constructing a narrow matcher from these patterns and
filtering all seen file paths through it.
Keep in mind that this change only influences file data: we're
still fetching all changeset and manifest data. So, there's still
a ton of "partial clone" to implement in exchangev2.
On a personal note, I derive gratification that this feature requires
very few lines of new code to implement.
To test this, we implemented a minimal extension which allows us to specify
--include/--exclude to clone. While the narrow extension provides these
arguments, I explicitly wanted to test this functionality without the
narrow extension enabled, as that extension monkeypatches various things
and I want to isolate the behavior of core Mercurial.
Differential Revision: https://phab.mercurial-scm.org/D5132
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 26 Sep 2018 14:38:43 -0700 |
parents | b843356d4ae1 |
children | 00a4cd368e3f |
rev | line source |
---|---|
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
1 # exchangev2.py - repository exchange for wire protocol version 2 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
2 # |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
4 # |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
7 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
8 from __future__ import absolute_import |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
9 |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
10 import collections |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
11 import weakref |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
12 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
13 from .i18n import _ |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
14 from .node import ( |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
15 nullid, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
16 short, |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
17 ) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
18 from . import ( |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
19 bookmarks, |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
20 error, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
21 mdiff, |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
22 narrowspec, |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
23 phases, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
24 pycompat, |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
25 setdiscovery, |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
26 ) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
27 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
28 def pull(pullop): |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
29 """Pull using wire protocol version 2.""" |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
30 repo = pullop.repo |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
31 remote = pullop.remote |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
32 tr = pullop.trmanager.transaction() |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
33 |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
34 # We don't use the repo's narrow matcher here because the patterns passed |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
35 # to exchange.pull() could be different. |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
36 narrowmatcher = narrowspec.match(repo.root, |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
37 # Empty maps to nevermatcher. So always |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
38 # set includes if missing. |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
39 pullop.includepats or {'path:.'}, |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
40 pullop.excludepats) |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
41 |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
42 if pullop.includepats or pullop.excludepats: |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
43 pathfilter = {} |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
44 if pullop.includepats: |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
45 pathfilter[b'include'] = sorted(pullop.includepats) |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
46 if pullop.excludepats: |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
47 pathfilter[b'exclude'] = sorted(pullop.excludepats) |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
48 else: |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
49 pathfilter = None |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
50 |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
51 # Figure out what needs to be fetched. |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
52 common, fetch, remoteheads = _pullchangesetdiscovery( |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
53 repo, remote, pullop.heads, abortwhenunrelated=pullop.force) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
54 |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
55 # And fetch the data. |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
56 pullheads = pullop.heads or remoteheads |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
57 csetres = _fetchchangesets(repo, tr, remote, common, fetch, pullheads) |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
58 |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
59 # New revisions are written to the changelog. But all other updates |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
60 # are deferred. Do those now. |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
61 |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
62 # Ensure all new changesets are draft by default. If the repo is |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
63 # publishing, the phase will be adjusted by the loop below. |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
64 if csetres['added']: |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
65 phases.registernew(repo, tr, phases.draft, csetres['added']) |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
66 |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
67 # And adjust the phase of all changesets accordingly. |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
68 for phase in phases.phasenames: |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
69 if phase == b'secret' or not csetres['nodesbyphase'][phase]: |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
70 continue |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
71 |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
72 phases.advanceboundary(repo, tr, phases.phasenames.index(phase), |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
73 csetres['nodesbyphase'][phase]) |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
74 |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
75 # Write bookmark updates. |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
76 bookmarks.updatefromremote(repo.ui, repo, csetres['bookmarks'], |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
77 remote.url(), pullop.gettransaction, |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
78 explicit=pullop.explicitbookmarks) |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
79 |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
80 manres = _fetchmanifests(repo, tr, remote, csetres['manifestnodes']) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
81 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
82 # Find all file nodes referenced by added manifests and fetch those |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
83 # revisions. |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
84 fnodes = _derivefilesfrommanifests(repo, narrowmatcher, manres['added']) |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
85 _fetchfilesfromcsets(repo, tr, remote, pathfilter, fnodes, csetres['added'], |
40179
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
86 manres['linkrevs']) |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
87 |
39629
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
88 def _pullchangesetdiscovery(repo, remote, heads, abortwhenunrelated=True): |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
89 """Determine which changesets need to be pulled.""" |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
90 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
91 if heads: |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
92 knownnode = repo.changelog.hasnode |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
93 if all(knownnode(head) for head in heads): |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
94 return heads, False, heads |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
95 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
96 # TODO wire protocol version 2 is capable of more efficient discovery |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
97 # than setdiscovery. Consider implementing something better. |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
98 common, fetch, remoteheads = setdiscovery.findcommonheads( |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
99 repo.ui, repo, remote, abortwhenunrelated=abortwhenunrelated) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
100 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
101 common = set(common) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
102 remoteheads = set(remoteheads) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
103 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
104 # If a remote head is filtered locally, put it back in the common set. |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
105 # See the comment in exchange._pulldiscoverychangegroup() for more. |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
106 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
107 if fetch and remoteheads: |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
108 nodemap = repo.unfiltered().changelog.nodemap |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
109 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
110 common |= {head for head in remoteheads if head in nodemap} |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
111 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
112 if set(remoteheads).issubset(common): |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
113 fetch = [] |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
114 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
115 common.discard(nullid) |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
116 |
a86d21e70b2b
exchangev2: start to implement pull with wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff
changeset
|
117 return common, fetch, remoteheads |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
118 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
119 def _fetchchangesets(repo, tr, remote, common, fetch, remoteheads): |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
120 # TODO consider adding a step here where we obtain the DAG shape first |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
121 # (or ask the server to slice changesets into chunks for us) so that |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
122 # we can perform multiple fetches in batches. This will facilitate |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
123 # resuming interrupted clones, higher server-side cache hit rates due |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
124 # to smaller segments, etc. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
125 with remote.commandexecutor() as e: |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
126 objs = e.callcommand(b'changesetdata', { |
40176
41263df08109
wireprotov2: change how revisions are specified to changesetdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40173
diff
changeset
|
127 b'revisions': [{ |
41263df08109
wireprotov2: change how revisions are specified to changesetdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40173
diff
changeset
|
128 b'type': b'changesetdagrange', |
41263df08109
wireprotov2: change how revisions are specified to changesetdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40173
diff
changeset
|
129 b'roots': sorted(common), |
41263df08109
wireprotov2: change how revisions are specified to changesetdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40173
diff
changeset
|
130 b'heads': sorted(remoteheads), |
41263df08109
wireprotov2: change how revisions are specified to changesetdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40173
diff
changeset
|
131 }], |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
132 b'fields': {b'bookmarks', b'parents', b'phase', b'revision'}, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
133 }).result() |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
134 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
135 # The context manager waits on all response data when exiting. So |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
136 # we need to remain in the context manager in order to stream data. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
137 return _processchangesetdata(repo, tr, objs) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
138 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
139 def _processchangesetdata(repo, tr, objs): |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
140 repo.hook('prechangegroup', throw=True, |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
141 **pycompat.strkwargs(tr.hookargs)) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
142 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
143 urepo = repo.unfiltered() |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
144 cl = urepo.changelog |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
145 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
146 cl.delayupdate(tr) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
147 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
148 # The first emitted object is a header describing the data that |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
149 # follows. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
150 meta = next(objs) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
151 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
152 progress = repo.ui.makeprogress(_('changesets'), |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
153 unit=_('chunks'), |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
154 total=meta.get(b'totalitems')) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
155 |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
156 manifestnodes = {} |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
157 |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
158 def linkrev(node): |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
159 repo.ui.debug('add changeset %s\n' % short(node)) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
160 # Linkrev for changelog is always self. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
161 return len(cl) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
162 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
163 def onchangeset(cl, node): |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
164 progress.increment() |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
165 |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
166 revision = cl.changelogrevision(node) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
167 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
168 # We need to preserve the mapping of changelog revision to node |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
169 # so we can set the linkrev accordingly when manifests are added. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
170 manifestnodes[cl.rev(node)] = revision.manifest |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
171 |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
172 nodesbyphase = {phase: set() for phase in phases.phasenames} |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
173 remotebookmarks = {} |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
174 |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
175 # addgroup() expects a 7-tuple describing revisions. This normalizes |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
176 # the wire data to that format. |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
177 # |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
178 # This loop also aggregates non-revision metadata, such as phase |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
179 # data. |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
180 def iterrevisions(): |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
181 for cset in objs: |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
182 node = cset[b'node'] |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
183 |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
184 if b'phase' in cset: |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
185 nodesbyphase[cset[b'phase']].add(node) |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
186 |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
187 for mark in cset.get(b'bookmarks', []): |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
188 remotebookmarks[mark] = node |
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
189 |
39636
399ddd3227a4
wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39635
diff
changeset
|
190 # TODO add mechanism for extensions to examine records so they |
399ddd3227a4
wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39635
diff
changeset
|
191 # can siphon off custom data fields. |
399ddd3227a4
wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39635
diff
changeset
|
192 |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
193 extrafields = {} |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
194 |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
195 for field, size in cset.get(b'fieldsfollowing', []): |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
196 extrafields[field] = next(objs) |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
197 |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
198 # Some entries might only be metadata only updates. |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
199 if b'revision' not in extrafields: |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
200 continue |
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
201 |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
202 data = extrafields[b'revision'] |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
203 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
204 yield ( |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
205 node, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
206 cset[b'parents'][0], |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
207 cset[b'parents'][1], |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
208 # Linknode is always itself for changesets. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
209 cset[b'node'], |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
210 # We always send full revisions. So delta base is not set. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
211 nullid, |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
212 mdiff.trivialdiffheader(len(data)) + data, |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
213 # Flags not yet supported. |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
214 0, |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
215 ) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
216 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
217 added = cl.addgroup(iterrevisions(), linkrev, weakref.proxy(tr), |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
218 addrevisioncb=onchangeset) |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
219 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
220 progress.complete() |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
221 |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
222 return { |
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
223 'added': added, |
39633
ff2de4f2eb3c
exchangev2: fetch and apply phases data
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39631
diff
changeset
|
224 'nodesbyphase': nodesbyphase, |
39635
349482d726ee
exchangev2: fetch and apply bookmarks
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39633
diff
changeset
|
225 'bookmarks': remotebookmarks, |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
226 'manifestnodes': manifestnodes, |
39631
b9e453d683a1
exchangev2: fetch changeset revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39629
diff
changeset
|
227 } |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
228 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
229 def _fetchmanifests(repo, tr, remote, manifestnodes): |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
230 rootmanifest = repo.manifestlog.getstorage(b'') |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
231 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
232 # Some manifests can be shared between changesets. Filter out revisions |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
233 # we already know about. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
234 fetchnodes = [] |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
235 linkrevs = {} |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
236 seen = set() |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
237 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
238 for clrev, node in sorted(manifestnodes.iteritems()): |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
239 if node in seen: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
240 continue |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
241 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
242 try: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
243 rootmanifest.rev(node) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
244 except error.LookupError: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
245 fetchnodes.append(node) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
246 linkrevs[node] = clrev |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
247 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
248 seen.add(node) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
249 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
250 # TODO handle tree manifests |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
251 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
252 # addgroup() expects 7-tuple describing revisions. This normalizes |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
253 # the wire data to that format. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
254 def iterrevisions(objs, progress): |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
255 for manifest in objs: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
256 node = manifest[b'node'] |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
257 |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
258 extrafields = {} |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
259 |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
260 for field, size in manifest.get(b'fieldsfollowing', []): |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
261 extrafields[field] = next(objs) |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
262 |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
263 if b'delta' in extrafields: |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
264 basenode = manifest[b'deltabasenode'] |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
265 delta = extrafields[b'delta'] |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
266 elif b'revision' in extrafields: |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
267 basenode = nullid |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
268 revision = extrafields[b'revision'] |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
269 delta = mdiff.trivialdiffheader(len(revision)) + revision |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
270 else: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
271 continue |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
272 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
273 yield ( |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
274 node, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
275 manifest[b'parents'][0], |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
276 manifest[b'parents'][1], |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
277 # The value passed in is passed to the lookup function passed |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
278 # to addgroup(). We already have a map of manifest node to |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
279 # changelog revision number. So we just pass in the |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
280 # manifest node here and use linkrevs.__getitem__ as the |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
281 # resolution function. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
282 node, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
283 basenode, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
284 delta, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
285 # Flags not yet supported. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
286 0 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
287 ) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
288 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
289 progress.increment() |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
290 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
291 progress = repo.ui.makeprogress(_('manifests'), unit=_('chunks'), |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
292 total=len(fetchnodes)) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
293 |
40173
b797150a1ab9
exchangev2: honor server advertised manifestdata recommended batch size
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40035
diff
changeset
|
294 commandmeta = remote.apidescriptor[b'commands'][b'manifestdata'] |
b797150a1ab9
exchangev2: honor server advertised manifestdata recommended batch size
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40035
diff
changeset
|
295 batchsize = commandmeta.get(b'recommendedbatchsize', 10000) |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
296 # TODO make size configurable on client? |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
297 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
298 # We send commands 1 at a time to the remote. This is not the most |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
299 # efficient because we incur a round trip at the end of each batch. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
300 # However, the existing frame-based reactor keeps consuming server |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
301 # data in the background. And this results in response data buffering |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
302 # in memory. This can consume gigabytes of memory. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
303 # TODO send multiple commands in a request once background buffering |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
304 # issues are resolved. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
305 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
306 added = [] |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
307 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
308 for i in pycompat.xrange(0, len(fetchnodes), batchsize): |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
309 batch = [node for node in fetchnodes[i:i + batchsize]] |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
310 if not batch: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
311 continue |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
312 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
313 with remote.commandexecutor() as e: |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
314 objs = e.callcommand(b'manifestdata', { |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
315 b'tree': b'', |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
316 b'nodes': batch, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
317 b'fields': {b'parents', b'revision'}, |
39641
aa7e312375cf
wireprotov2: let clients drive delta behavior
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39640
diff
changeset
|
318 b'haveparents': True, |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
319 }).result() |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
320 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
321 # Chomp off header object. |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
322 next(objs) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
323 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
324 added.extend(rootmanifest.addgroup( |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
325 iterrevisions(objs, progress), |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
326 linkrevs.__getitem__, |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
327 weakref.proxy(tr))) |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
328 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
329 progress.complete() |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
330 |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
331 return { |
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
332 'added': added, |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
333 'linkrevs': linkrevs, |
39638
d292328e0143
exchangev2: fetch manifest revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39636
diff
changeset
|
334 } |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
335 |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
336 def _derivefilesfrommanifests(repo, matcher, manifestnodes): |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
337 """Determine what file nodes are relevant given a set of manifest nodes. |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
338 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
339 Returns a dict mapping file paths to dicts of file node to first manifest |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
340 node. |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
341 """ |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
342 ml = repo.manifestlog |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
343 fnodes = collections.defaultdict(dict) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
344 |
40035
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
345 progress = repo.ui.makeprogress( |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
346 _('scanning manifests'), total=len(manifestnodes)) |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
347 |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
348 with progress: |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
349 for manifestnode in manifestnodes: |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
350 m = ml.get(b'', manifestnode) |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
351 |
40035
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
352 # TODO this will pull in unwanted nodes because it takes the storage |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
353 # delta into consideration. What we really want is something that |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
354 # takes the delta between the manifest's parents. And ideally we |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
355 # would ignore file nodes that are known locally. For now, ignore |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
356 # both these limitations. This will result in incremental fetches |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
357 # requesting data we already have. So this is far from ideal. |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
358 md = m.readfast() |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
359 |
40035
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
360 for path, fnode in md.items(): |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
361 if matcher(path): |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
362 fnodes[path].setdefault(fnode, manifestnode) |
40035
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
363 |
7a347d362a45
exchangev2: add progress bar around manifest scanning
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39814
diff
changeset
|
364 progress.increment() |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
365 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
366 return fnodes |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
367 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
368 def _fetchfiles(repo, tr, remote, fnodes, linkrevs): |
40179
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
369 """Fetch file data from explicit file revisions.""" |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
370 def iterrevisions(objs, progress): |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
371 for filerevision in objs: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
372 node = filerevision[b'node'] |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
373 |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
374 extrafields = {} |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
375 |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
376 for field, size in filerevision.get(b'fieldsfollowing', []): |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
377 extrafields[field] = next(objs) |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
378 |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
379 if b'delta' in extrafields: |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
380 basenode = filerevision[b'deltabasenode'] |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
381 delta = extrafields[b'delta'] |
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
382 elif b'revision' in extrafields: |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
383 basenode = nullid |
39814
d059cb669632
wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39641
diff
changeset
|
384 revision = extrafields[b'revision'] |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
385 delta = mdiff.trivialdiffheader(len(revision)) + revision |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
386 else: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
387 continue |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
388 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
389 yield ( |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
390 node, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
391 filerevision[b'parents'][0], |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
392 filerevision[b'parents'][1], |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
393 node, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
394 basenode, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
395 delta, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
396 # Flags not yet supported. |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
397 0, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
398 ) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
399 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
400 progress.increment() |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
401 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
402 progress = repo.ui.makeprogress( |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
403 _('files'), unit=_('chunks'), |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
404 total=sum(len(v) for v in fnodes.itervalues())) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
405 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
406 # TODO make batch size configurable |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
407 batchsize = 10000 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
408 fnodeslist = [x for x in sorted(fnodes.items())] |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
409 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
410 for i in pycompat.xrange(0, len(fnodeslist), batchsize): |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
411 batch = [x for x in fnodeslist[i:i + batchsize]] |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
412 if not batch: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
413 continue |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
414 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
415 with remote.commandexecutor() as e: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
416 fs = [] |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
417 locallinkrevs = {} |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
418 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
419 for path, nodes in batch: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
420 fs.append((path, e.callcommand(b'filedata', { |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
421 b'path': path, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
422 b'nodes': sorted(nodes), |
39641
aa7e312375cf
wireprotov2: let clients drive delta behavior
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39640
diff
changeset
|
423 b'fields': {b'parents', b'revision'}, |
aa7e312375cf
wireprotov2: let clients drive delta behavior
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39640
diff
changeset
|
424 b'haveparents': True, |
39640
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
425 }))) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
426 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
427 locallinkrevs[path] = { |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
428 node: linkrevs[manifestnode] |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
429 for node, manifestnode in nodes.iteritems()} |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
430 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
431 for path, f in fs: |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
432 objs = f.result() |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
433 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
434 # Chomp off header objects. |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
435 next(objs) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
436 |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
437 store = repo.file(path) |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
438 store.addgroup( |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
439 iterrevisions(objs, progress), |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
440 locallinkrevs[path].__getitem__, |
039bf1eddc2e
exchangev2: fetch file revisions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39638
diff
changeset
|
441 weakref.proxy(tr)) |
40179
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
442 |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
443 def _fetchfilesfromcsets(repo, tr, remote, pathfilter, fnodes, csets, |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
444 manlinkrevs): |
40179
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
445 """Fetch file data from explicit changeset revisions.""" |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
446 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
447 def iterrevisions(objs, remaining, progress): |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
448 while remaining: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
449 filerevision = next(objs) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
450 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
451 node = filerevision[b'node'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
452 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
453 extrafields = {} |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
454 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
455 for field, size in filerevision.get(b'fieldsfollowing', []): |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
456 extrafields[field] = next(objs) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
457 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
458 if b'delta' in extrafields: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
459 basenode = filerevision[b'deltabasenode'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
460 delta = extrafields[b'delta'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
461 elif b'revision' in extrafields: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
462 basenode = nullid |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
463 revision = extrafields[b'revision'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
464 delta = mdiff.trivialdiffheader(len(revision)) + revision |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
465 else: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
466 continue |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
467 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
468 yield ( |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
469 node, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
470 filerevision[b'parents'][0], |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
471 filerevision[b'parents'][1], |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
472 node, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
473 basenode, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
474 delta, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
475 # Flags not yet supported. |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
476 0, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
477 ) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
478 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
479 progress.increment() |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
480 remaining -= 1 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
481 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
482 progress = repo.ui.makeprogress( |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
483 _('files'), unit=_('chunks'), |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
484 total=sum(len(v) for v in fnodes.itervalues())) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
485 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
486 commandmeta = remote.apidescriptor[b'commands'][b'filesdata'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
487 batchsize = commandmeta.get(b'recommendedbatchsize', 50000) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
488 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
489 for i in pycompat.xrange(0, len(csets), batchsize): |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
490 batch = [x for x in csets[i:i + batchsize]] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
491 if not batch: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
492 continue |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
493 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
494 with remote.commandexecutor() as e: |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
495 args = { |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
496 b'revisions': [{ |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
497 b'type': b'changesetexplicit', |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
498 b'nodes': batch, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
499 }], |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
500 b'fields': {b'parents', b'revision'}, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
501 b'haveparents': True, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
502 } |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
503 |
40327
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
504 if pathfilter: |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
505 args[b'pathfilter'] = pathfilter |
55836a34f41b
exchangev2: recognize narrow patterns when pulling
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40179
diff
changeset
|
506 |
40179
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
507 objs = e.callcommand(b'filesdata', args).result() |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
508 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
509 # First object is an overall header. |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
510 overall = next(objs) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
511 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
512 # We have overall['totalpaths'] segments. |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
513 for i in pycompat.xrange(overall[b'totalpaths']): |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
514 header = next(objs) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
515 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
516 path = header[b'path'] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
517 store = repo.file(path) |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
518 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
519 linkrevs = { |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
520 fnode: manlinkrevs[mnode] |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
521 for fnode, mnode in fnodes[path].iteritems()} |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
522 |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
523 store.addgroup(iterrevisions(objs, header[b'totalitems'], |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
524 progress), |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
525 linkrevs.__getitem__, |
b843356d4ae1
exchangev2: use filesdata
Gregory Szorc <gregory.szorc@gmail.com>
parents:
40176
diff
changeset
|
526 weakref.proxy(tr)) |