Mercurial > hg
annotate mercurial/setdiscovery.py @ 51630:513597087b89
branching: merge stable into default
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Mon, 10 Jun 2024 12:12:56 +0200 |
parents | c217d94cdd9d |
children | 493034cc3265 |
rev | line source |
---|---|
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
1 # setdiscovery.py - improved discovery of common nodeset for mercurial |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
2 # |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
3 # Copyright 2010 Benoit Boissinot <bboissin@gmail.com> |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
4 # and Peter Arrenbrecht <peter@arrenbrecht.ch> |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
5 # |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
6 # This software may be used and distributed according to the terms of the |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
7 # GNU General Public License version 2 or any later version. |
20656
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
8 """ |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
9 Algorithm works in the following way. You have two repository: local and |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
10 remote. They both contains a DAG of changelists. |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
11 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
12 The goal of the discovery protocol is to find one set of node *common*, |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
13 the set of nodes shared by local and remote. |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
14 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
15 One of the issue with the original protocol was latency, it could |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
16 potentially require lots of roundtrips to discover that the local repo was a |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
17 subset of remote (which is a very common case, you usually have few changes |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
18 compared to upstream, while upstream probably had lots of development). |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
19 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
20 The new protocol only requires one interface for the remote repo: `known()`, |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
21 which given a set of changelists tells you if they are present in the DAG. |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
22 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
23 The algorithm then works as follow: |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
24 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
25 - We will be using three sets, `common`, `missing`, `unknown`. Originally |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
26 all nodes are in `unknown`. |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
27 - Take a sample from `unknown`, call `remote.known(sample)` |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
28 - For each node that remote knows, move it and all its ancestors to `common` |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
29 - For each node that remote doesn't know, move it and all its descendants |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
30 to `missing` |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
31 - Iterate until `unknown` is empty |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
32 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
33 There are a couple optimizations, first is instead of starting with a random |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
34 sample of missing, start by sending all heads, in the case where the local |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
35 repo is a subset, you computed the answer in one round trip. |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
36 |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
37 Then you can do something similar to the bisecting strategy used when |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
38 finding faulty changesets. Instead of random samples, you can try picking |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
39 nodes that will maximize the number of nodes that will be |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
40 classified with it (since all ancestors or descendants will be marked as well). |
cdecbc5ab504
setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents:
20034
diff
changeset
|
41 """ |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
42 |
25973
fb5664eb8414
setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25914
diff
changeset
|
43 |
25113
0ca8410ea345
util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents:
23817
diff
changeset
|
44 import collections |
20034
1e5b38a919dd
cleanup: move stdlib imports to their own import statement
Augie Fackler <raf@durin42.com>
parents:
17426
diff
changeset
|
45 import random |
25973
fb5664eb8414
setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25914
diff
changeset
|
46 |
fb5664eb8414
setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25914
diff
changeset
|
47 from .i18n import _ |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
48 from .node import nullrev |
25973
fb5664eb8414
setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25914
diff
changeset
|
49 from . import ( |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25973
diff
changeset
|
50 error, |
42745
4d20b1fe8a72
rust-discovery: using from Python code
Georges Racinet <georges.racinet@octobus.net>
parents:
42742
diff
changeset
|
51 policy, |
32712
43bda143e3b2
discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32291
diff
changeset
|
52 util, |
25973
fb5664eb8414
setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25914
diff
changeset
|
53 ) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
54 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
55 |
39174
71d83b315778
setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39173
diff
changeset
|
56 def _updatesample(revs, heads, sample, parentfn, quicksamplesize=0): |
23809
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
57 """update an existing sample to match the expected size |
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
58 |
39168
2d218db7389b
setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39166
diff
changeset
|
59 The sample is updated with revs exponentially distant from each head of the |
2d218db7389b
setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39166
diff
changeset
|
60 <revs> set. (H~1, H~2, H~4, H~8, etc). |
23809
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
61 |
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
62 If a target size is specified, the sampling will stop once this size is |
39168
2d218db7389b
setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39166
diff
changeset
|
63 reached. Otherwise sampling will happen until roots of the <revs> set are |
23809
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
64 reached. |
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
65 |
39168
2d218db7389b
setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39166
diff
changeset
|
66 :revs: set of revs we want to discover (if None, assume the whole dag) |
39170
754f389b87f2
setdiscovery: pass heads into _updatesample()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39169
diff
changeset
|
67 :heads: set of DAG head revs |
23809
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
68 :sample: a sample to update |
39174
71d83b315778
setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39173
diff
changeset
|
69 :parentfn: a callable to resolve parents for a revision |
23809
9ca2eb881b53
setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23808
diff
changeset
|
70 :quicksamplesize: optional target size of the sample""" |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
71 dist = {} |
25113
0ca8410ea345
util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents:
23817
diff
changeset
|
72 visit = collections.deque(heads) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
73 seen = set() |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
74 factor = 1 |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
75 while visit: |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
76 curr = visit.popleft() |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
77 if curr in seen: |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
78 continue |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
79 d = dist.setdefault(curr, 1) |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
80 if d > factor: |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
81 factor *= 2 |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
82 if d == factor: |
23814
6a5877a73141
setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23813
diff
changeset
|
83 sample.add(curr) |
6a5877a73141
setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23813
diff
changeset
|
84 if quicksamplesize and (len(sample) >= quicksamplesize): |
6a5877a73141
setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23813
diff
changeset
|
85 return |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
86 seen.add(curr) |
39174
71d83b315778
setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39173
diff
changeset
|
87 |
71d83b315778
setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39173
diff
changeset
|
88 for p in parentfn(curr): |
71d83b315778
setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39173
diff
changeset
|
89 if p != nullrev and (not revs or p in revs): |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
90 dist.setdefault(p, d + 1) |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
91 visit.append(p) |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
92 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
93 |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
94 def _limitsample(sample, desiredlen, randomize=True): |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
95 """return a random subset of sample of at most desiredlen item. |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
96 |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
97 If randomize is False, though, a deterministic subset is returned. |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
98 This is meant for integration tests. |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
99 """ |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
100 if len(sample) <= desiredlen: |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
101 return sample |
49647
c217d94cdd9d
python-compat: adapt to Python 3.11 BC breakage with `random.sample`
Raphaël Gomès <rgomes@octobus.net>
parents:
49017
diff
changeset
|
102 sample = list(sample) |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
103 if randomize: |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
104 return set(random.sample(sample, desiredlen)) |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
105 sample.sort() |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
106 return set(sample[:desiredlen]) |
23083
ee45f5c2ffcc
setdiscovery: extract sample limitation in a `_limitsample` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
20656
diff
changeset
|
107 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
108 |
48946
642e31cb55f0
py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents:
48875
diff
changeset
|
109 class partialdiscovery: |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
110 """an object representing ongoing discovery |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
111 |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
112 Feed with data from the remote repository, this object keep track of the |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
113 current set of changeset in various states: |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
114 |
41172
3dcc96582627
discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents:
41171
diff
changeset
|
115 - common: revs also known remotely |
3dcc96582627
discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents:
41171
diff
changeset
|
116 - undecided: revs we don't have information on yet |
3dcc96582627
discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents:
41171
diff
changeset
|
117 - missing: revs missing remotely |
3dcc96582627
discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents:
41171
diff
changeset
|
118 (all tracked revisions are known locally) |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
119 """ |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
120 |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
121 def __init__(self, repo, targetheads, respectsize, randomize=True): |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
122 self._repo = repo |
41167
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
123 self._targetheads = targetheads |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
124 self._common = repo.changelog.incrementalmissingrevs() |
41167
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
125 self._undecided = None |
41170
96201120cdf5
discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41169
diff
changeset
|
126 self.missing = set() |
41885
5baf06d2bb41
discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41884
diff
changeset
|
127 self._childrenmap = None |
42402
5b34972a0094
setdiscovery: make progress on most connected groups each roundtrip
Martin von Zweigbergk <martinvonz@google.com>
parents:
42354
diff
changeset
|
128 self._respectsize = respectsize |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
129 self.randomize = randomize |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
130 |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
131 def addcommons(self, commons): |
42170
6631f3e89b6f
setdiscovery: fix a few typos
Joerg Sonnenberger <joerg@bec.de>
parents:
42159
diff
changeset
|
132 """register nodes known as common""" |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
133 self._common.addbases(commons) |
41304
76873548b051
partialdiscovery: avoid `undecided` related computation sooner than necessary
Boris Feld <boris.feld@octobus.net>
parents:
41280
diff
changeset
|
134 if self._undecided is not None: |
76873548b051
partialdiscovery: avoid `undecided` related computation sooner than necessary
Boris Feld <boris.feld@octobus.net>
parents:
41280
diff
changeset
|
135 self._common.removeancestorsfrom(self._undecided) |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
136 |
41170
96201120cdf5
discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41169
diff
changeset
|
137 def addmissings(self, missings): |
42170
6631f3e89b6f
setdiscovery: fix a few typos
Joerg Sonnenberger <joerg@bec.de>
parents:
42159
diff
changeset
|
138 """register some nodes as missing""" |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
139 newmissing = self._repo.revs(b'%ld::%ld', missings, self.undecided) |
41280
f4277a35c42c
discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents:
41245
diff
changeset
|
140 if newmissing: |
f4277a35c42c
discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents:
41245
diff
changeset
|
141 self.missing.update(newmissing) |
f4277a35c42c
discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents:
41245
diff
changeset
|
142 self.undecided.difference_update(newmissing) |
41170
96201120cdf5
discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41169
diff
changeset
|
143 |
41171
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
144 def addinfo(self, sample): |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
145 """consume an iterable of (rev, known) tuples""" |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
146 common = set() |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
147 missing = set() |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
148 for rev, known in sample: |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
149 if known: |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
150 common.add(rev) |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
151 else: |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
152 missing.add(rev) |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
153 if common: |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
154 self.addcommons(common) |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
155 if missing: |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
156 self.addmissings(missing) |
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
157 |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
158 def hasinfo(self): |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
159 """return True is we have any clue about the remote state""" |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
160 return self._common.hasbases() |
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
161 |
41169
3ce5b96482c6
discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents:
41168
diff
changeset
|
162 def iscomplete(self): |
3ce5b96482c6
discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents:
41168
diff
changeset
|
163 """True if all the necessary data have been gathered""" |
3ce5b96482c6
discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents:
41168
diff
changeset
|
164 return self._undecided is not None and not self._undecided |
3ce5b96482c6
discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents:
41168
diff
changeset
|
165 |
41167
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
166 @property |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
167 def undecided(self): |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
168 if self._undecided is not None: |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
169 return self._undecided |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
170 self._undecided = set(self._common.missingancestors(self._targetheads)) |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
171 return self._undecided |
870a89c6909d
discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents:
41162
diff
changeset
|
172 |
42103
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
173 def stats(self): |
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
174 return { |
43162
3c6976b1f693
py3-discovery: using plain str in stats dict
Georges Racinet <georges.racinet@octobus.net>
parents:
43077
diff
changeset
|
175 'undecided': len(self.undecided), |
42103
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
176 } |
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
177 |
41113
9815d3337f9b
discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41112
diff
changeset
|
178 def commonheads(self): |
9815d3337f9b
discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41112
diff
changeset
|
179 """the heads of the known common set""" |
9815d3337f9b
discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41112
diff
changeset
|
180 # heads(common) == heads(common.bases) since common represents |
9815d3337f9b
discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41112
diff
changeset
|
181 # common.bases and all its ancestors |
41245
2a8782cc2e16
discovery: using the new basesheads()
Georges Racinet <georges.racinet@octobus.net>
parents:
41172
diff
changeset
|
182 return self._common.basesheads() |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
183 |
41881
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
184 def _parentsgetter(self): |
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
185 getrev = self._repo.changelog.index.__getitem__ |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
186 |
41881
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
187 def getparents(r): |
41979
0d467e4de4ae
discovery: fix embarrassing typo in slice definition
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41889
diff
changeset
|
188 return getrev(r)[5:7] |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
189 |
41881
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
190 return getparents |
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
191 |
41886
a05f0bbefdd9
discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41885
diff
changeset
|
192 def _childrengetter(self): |
41884
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
193 |
41885
5baf06d2bb41
discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41884
diff
changeset
|
194 if self._childrenmap is not None: |
41889
3ba9ca537f57
discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41886
diff
changeset
|
195 # During discovery, the `undecided` set keep shrinking. |
3ba9ca537f57
discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41886
diff
changeset
|
196 # Therefore, the map computed for an iteration N will be |
3ba9ca537f57
discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41886
diff
changeset
|
197 # valid for iteration N+1. Instead of computing the same |
3ba9ca537f57
discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41886
diff
changeset
|
198 # data over and over we cached it the first time. |
41885
5baf06d2bb41
discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41884
diff
changeset
|
199 return self._childrenmap.__getitem__ |
5baf06d2bb41
discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41884
diff
changeset
|
200 |
41884
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
201 # _updatesample() essentially does interaction over revisions to look |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
202 # up their children. This lookup is expensive and doing it in a loop is |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
203 # quadratic. We precompute the children for all relevant revisions and |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
204 # make the lookup in _updatesample() a simple dict lookup. |
41885
5baf06d2bb41
discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41884
diff
changeset
|
205 self._childrenmap = children = {} |
41884
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
206 |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
207 parentrevs = self._parentsgetter() |
41886
a05f0bbefdd9
discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41885
diff
changeset
|
208 revs = self.undecided |
41884
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
209 |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
210 for rev in sorted(revs): |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
211 # Always ensure revision has an entry so we don't need to worry |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
212 # about missing keys. |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
213 children[rev] = [] |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
214 for prev in parentrevs(rev): |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
215 if prev == nullrev: |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
216 continue |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
217 c = children.get(prev) |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
218 if c is not None: |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
219 c.append(rev) |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
220 return children.__getitem__ |
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
221 |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
222 def takequicksample(self, headrevs, size): |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
223 """takes a quick sample of size <size> |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
224 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
225 It is meant for initial sampling and focuses on querying heads and close |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
226 ancestors of heads. |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
227 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
228 :headrevs: set of head revisions in local DAG to consider |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
229 :size: the maximum size of the sample""" |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
230 revs = self.undecided |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
231 if len(revs) <= size: |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
232 return list(revs) |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
233 sample = set(self._repo.revs(b'heads(%ld)', revs)) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
234 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
235 if len(sample) >= size: |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
236 return _limitsample(sample, size, randomize=self.randomize) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
237 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
238 _updatesample( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
239 None, headrevs, sample, self._parentsgetter(), quicksamplesize=size |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
240 ) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
241 return sample |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
242 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
243 def takefullsample(self, headrevs, size): |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
244 revs = self.undecided |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
245 if len(revs) <= size: |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
246 return list(revs) |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
247 repo = self._repo |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
248 sample = set(repo.revs(b'heads(%ld)', revs)) |
41881
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
249 parentrevs = self._parentsgetter() |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
250 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
251 # update from heads |
41880
55919b96c02a
discovery: avoid computing identical sets of heads twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41879
diff
changeset
|
252 revsheads = sample.copy() |
41881
e514799e4e07
discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41880
diff
changeset
|
253 _updatesample(revs, revsheads, sample, parentrevs) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
254 |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
255 # update from roots |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
256 revsroots = set(repo.revs(b'roots(%ld)', revs)) |
41886
a05f0bbefdd9
discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41885
diff
changeset
|
257 childrenrevs = self._childrengetter() |
41884
d5e6ae6e8012
discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
41883
diff
changeset
|
258 _updatesample(revs, revsroots, sample, childrenrevs) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
259 assert sample |
42425
b9ff059fd194
discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42402
diff
changeset
|
260 |
b9ff059fd194
discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42402
diff
changeset
|
261 if not self._respectsize: |
b9ff059fd194
discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42402
diff
changeset
|
262 size = max(size, min(len(revsroots), len(revsheads))) |
b9ff059fd194
discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42402
diff
changeset
|
263 |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
264 sample = _limitsample(sample, size, randomize=self.randomize) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
265 if len(sample) < size: |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
266 more = size - len(sample) |
42741
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
267 takefrom = list(revs - sample) |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
268 if self.randomize: |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
269 sample.update(random.sample(takefrom, more)) |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
270 else: |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
271 takefrom.sort() |
4e7bd6180b53
rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents:
42425
diff
changeset
|
272 sample.update(takefrom[:more]) |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
273 return sample |
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
274 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
275 |
47269
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
276 pure_partialdiscovery = partialdiscovery |
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
277 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
278 partialdiscovery = policy.importrust( |
43506
9f70512ae2cf
cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents:
43162
diff
changeset
|
279 'discovery', member='PartialDiscovery', default=partialdiscovery |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
280 ) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
281 |
42745
4d20b1fe8a72
rust-discovery: using from Python code
Georges Racinet <georges.racinet@octobus.net>
parents:
42742
diff
changeset
|
282 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
283 def findcommonheads( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
284 ui, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
285 local, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
286 remote, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
287 abortwhenunrelated=True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
288 ancestorsof=None, |
46110
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
289 audit=None, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
290 ): |
45942
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
291 """Return a tuple (common, anyincoming, remoteheads) used to identify |
14206
2bf60f158ecb
setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents:
14164
diff
changeset
|
292 missing nodes from or in remote. |
46110
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
293 |
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
294 The audit argument is an optional dictionnary that a caller can pass. it |
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
295 will be updated with extra data about the discovery, this is useful for |
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
296 debug. |
45942
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
297 """ |
46300
f17b6e40a775
discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46299
diff
changeset
|
298 |
f17b6e40a775
discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46299
diff
changeset
|
299 samplegrowth = float(ui.config(b'devel', b'discovery.grow-sample.rate')) |
f17b6e40a775
discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46299
diff
changeset
|
300 |
49017
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
301 if audit is not None: |
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
302 audit[b'total-queries'] = 0 |
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
303 |
32712
43bda143e3b2
discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32291
diff
changeset
|
304 start = util.timer() |
43bda143e3b2
discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32291
diff
changeset
|
305 |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
306 roundtrips = 0 |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
307 cl = local.changelog |
39159
5b32b3c618b2
setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38373
diff
changeset
|
308 clnode = cl.node |
39161
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
309 clrev = cl.rev |
39159
5b32b3c618b2
setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38373
diff
changeset
|
310 |
35304
f77121b6bf1b
setdiscover: allow to ignore part of the local graph
Boris Feld <boris.feld@octobus.net>
parents:
32768
diff
changeset
|
311 if ancestorsof is not None: |
39165
860e83cd97de
setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39161
diff
changeset
|
312 ownheads = [clrev(n) for n in ancestorsof] |
860e83cd97de
setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39161
diff
changeset
|
313 else: |
860e83cd97de
setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39161
diff
changeset
|
314 ownheads = [rev for rev in cl.headrevs() if rev != nullrev] |
860e83cd97de
setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39161
diff
changeset
|
315 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
316 initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads') |
46359
0600e8467101
discovery: add config options to control sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46316
diff
changeset
|
317 initialsamplesize = ui.configint(b'devel', b'discovery.sample-size.initial') |
0600e8467101
discovery: add config options to control sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46316
diff
changeset
|
318 fullsamplesize = ui.configint(b'devel', b'discovery.sample-size') |
42159
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
319 # We also ask remote about all the local heads. That set can be arbitrarily |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
320 # large, so we used to limit it size to `initialsamplesize`. We no longer |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
321 # do as it proved counter productive. The skipped heads could lead to a |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
322 # large "undecided" set, slower to be clarified than if we asked the |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
323 # question for all heads right away. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
324 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
325 # We are already fetching all server heads using the `heads` commands, |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
326 # sending a equivalent number of heads the other way should not have a |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
327 # significant impact. In addition, it is very likely that we are going to |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
328 # have to issue "known" request for an equivalent amount of revisions in |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
329 # order to decide if theses heads are common or missing. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
330 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
331 # find a detailled analysis below. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
332 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
333 # Case A: local and server both has few heads |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
334 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
335 # Ownheads is below initialsamplesize, limit would not have any effect. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
336 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
337 # Case B: local has few heads and server has many |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
338 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
339 # Ownheads is below initialsamplesize, limit would not have any effect. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
340 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
341 # Case C: local and server both has many heads |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
342 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
343 # We now transfert some more data, but not significantly more than is |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
344 # already transfered to carry the server heads. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
345 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
346 # Case D: local has many heads, server has few |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
347 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
348 # D.1 local heads are mostly known remotely |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
349 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
350 # All the known head will have be part of a `known` request at some |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
351 # point for the discovery to finish. Sending them all earlier is |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
352 # actually helping. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
353 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
354 # (This case is fairly unlikely, it requires the numerous heads to all |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
355 # be merged server side in only a few heads) |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
356 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
357 # D.2 local heads are mostly missing remotely |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
358 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
359 # To determine that the heads are missing, we'll have to issue `known` |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
360 # request for them or one of their ancestors. This amount of `known` |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
361 # request will likely be in the same order of magnitude than the amount |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
362 # of local heads. |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
363 # |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
364 # The only case where we can be more efficient using `known` request on |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
365 # ancestors are case were all the "missing" local heads are based on a |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
366 # few changeset, also "missing". This means we would have a "complex" |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
367 # graph (with many heads) attached to, but very independant to a the |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
368 # "simple" graph on the server. This is a fairly usual case and have |
4f9a89837f07
setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42103
diff
changeset
|
369 # not been met in the wild so far. |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
370 if initial_head_exchange: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
371 if remote.limitedarguments: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
372 sample = _limitsample(ownheads, initialsamplesize) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
373 # indices between sample and externalized version must match |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
374 sample = list(sample) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
375 else: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
376 sample = ownheads |
37631
2f626233859b
wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37630
diff
changeset
|
377 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
378 ui.debug(b"query 1; heads\n") |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
379 roundtrips += 1 |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
380 with remote.commandexecutor() as e: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
381 fheads = e.callcommand(b'heads', {}) |
49017
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
382 if audit is not None: |
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
383 audit[b'total-queries'] += len(sample) |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
384 fknown = e.callcommand( |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
385 b'known', |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
386 { |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
387 b'nodes': [clnode(r) for r in sample], |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
388 }, |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
389 ) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
390 |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
391 srvheadhashes, yesno = fheads.result(), fknown.result() |
37631
2f626233859b
wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37630
diff
changeset
|
392 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
393 if audit is not None: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
394 audit[b'total-roundtrips'] = 1 |
46110
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
395 |
46841
e7b4607d52e3
setdiscovery: simplify by using tiprev directly
Joerg Sonnenberger <joerg@bec.de>
parents:
46805
diff
changeset
|
396 if cl.tiprev() == nullrev: |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
397 if srvheadhashes != [cl.nullid]: |
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
398 return [cl.nullid], True, srvheadhashes |
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
399 return [cl.nullid], False, [] |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
400 else: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
401 # we still need the remote head for the function return |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
402 with remote.commandexecutor() as e: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
403 fheads = e.callcommand(b'heads', {}) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
404 srvheadhashes = fheads.result() |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
405 |
14206
2bf60f158ecb
setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents:
14164
diff
changeset
|
406 # start actual discovery (we note this before the next "if" for |
2bf60f158ecb
setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents:
14164
diff
changeset
|
407 # compatibility reasons) |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
408 ui.status(_(b"searching for changes\n")) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
409 |
41878
82884bbf8d2b
discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents:
41304
diff
changeset
|
410 knownsrvheads = [] # revnos of remote heads that are known locally |
39161
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
411 for node in srvheadhashes: |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
412 if node == cl.nullid: |
39161
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
413 continue |
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
414 |
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
415 try: |
41878
82884bbf8d2b
discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents:
41304
diff
changeset
|
416 knownsrvheads.append(clrev(node)) |
39161
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
417 # Catches unknown and filtered nodes. |
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
418 except error.LookupError: |
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
419 continue |
858a12846f4f
setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents:
39159
diff
changeset
|
420 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
421 if initial_head_exchange: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
422 # early exit if we know all the specified remote heads already |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
423 if len(knownsrvheads) == len(srvheadhashes): |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
424 ui.debug(b"all remote heads known locally\n") |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
425 return srvheadhashes, False, srvheadhashes |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
426 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
427 if len(sample) == len(ownheads) and all(yesno): |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
428 ui.note(_(b"all local changesets known remotely\n")) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
429 ownheadhashes = [clnode(r) for r in ownheads] |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
430 return ownheadhashes, True, srvheadhashes |
14624
f03c82d1f50a
setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
14206
diff
changeset
|
431 |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
432 # full blown discovery |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
433 |
46299
397e39ad0174
discovery: add a `devel', b'discovery.grow-sample`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46110
diff
changeset
|
434 # if the server has a limit to its arguments size, we can't grow the sample. |
46805
2b1b8f3e6510
setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46804
diff
changeset
|
435 configbool = local.ui.configbool |
2b1b8f3e6510
setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46804
diff
changeset
|
436 grow_sample = configbool(b'devel', b'discovery.grow-sample') |
46804
f165105400d0
setdiscovery: rearrange code deciding if we will grow the sample
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46359
diff
changeset
|
437 grow_sample = grow_sample and not remote.limitedarguments |
46299
397e39ad0174
discovery: add a `devel', b'discovery.grow-sample`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46110
diff
changeset
|
438 |
46805
2b1b8f3e6510
setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46804
diff
changeset
|
439 dynamic_sample = configbool(b'devel', b'discovery.grow-sample.dynamic') |
2b1b8f3e6510
setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46804
diff
changeset
|
440 hard_limit_sample = not (dynamic_sample or remote.limitedarguments) |
2b1b8f3e6510
setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46804
diff
changeset
|
441 |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
442 randomize = ui.configbool(b'devel', b'discovery.randomize') |
47269
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
443 if cl.index.rust_ext_compat: |
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
444 pd = partialdiscovery |
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
445 else: |
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
446 pd = pure_partialdiscovery |
6be2a7ca4b1d
revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
47012
diff
changeset
|
447 disco = pd(local, ownheads, hard_limit_sample, randomize=randomize) |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
448 if initial_head_exchange: |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
449 # treat remote heads (and maybe own heads) as a first implicit sample |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
450 # response |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
451 disco.addcommons(knownsrvheads) |
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
452 disco.addinfo(zip(sample, yesno)) |
16683 | 453 |
46316
6ee9bd69ff7f
discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46301
diff
changeset
|
454 full = not initial_head_exchange |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
455 progress = ui.makeprogress(_(b'searching'), unit=_(b'queries')) |
41169
3ce5b96482c6
discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents:
41168
diff
changeset
|
456 while not disco.iscomplete(): |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
457 |
41112
3023bc4b3da0
discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41111
diff
changeset
|
458 if full or disco.hasinfo(): |
23747
f82173a90c2c
setdiscovery: factorize similar sampling code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23746
diff
changeset
|
459 if full: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
460 ui.note(_(b"sampling from both directions\n")) |
23747
f82173a90c2c
setdiscovery: factorize similar sampling code
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23746
diff
changeset
|
461 else: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
462 ui.debug(b"taking initial sample\n") |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
463 samplefunc = disco.takefullsample |
23130
ced632394371
setdiscovery: limit the size of all sample (issue4411)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23084
diff
changeset
|
464 targetsize = fullsamplesize |
46804
f165105400d0
setdiscovery: rearrange code deciding if we will grow the sample
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
46359
diff
changeset
|
465 if grow_sample: |
42354
dbd0fcca6dfc
discovery: slowly increase sampling size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
42170
diff
changeset
|
466 fullsamplesize = int(fullsamplesize * samplegrowth) |
14624
f03c82d1f50a
setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
14206
diff
changeset
|
467 else: |
f03c82d1f50a
setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
14206
diff
changeset
|
468 # use even cheaper initial sample |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
469 ui.debug(b"taking quick initial sample\n") |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
470 samplefunc = disco.takequicksample |
23130
ced632394371
setdiscovery: limit the size of all sample (issue4411)
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
23084
diff
changeset
|
471 targetsize = initialsamplesize |
41879
e5ece0f46b40
discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents:
41878
diff
changeset
|
472 sample = samplefunc(ownheads, targetsize) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
473 |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
474 roundtrips += 1 |
38350
9e70690a21ac
setdiscovery: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
37631
diff
changeset
|
475 progress.update(roundtrips) |
42103
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
476 stats = disco.stats() |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
477 ui.debug( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
478 b"query %i; still undecided: %i, sample size is: %i\n" |
43162
3c6976b1f693
py3-discovery: using plain str in stats dict
Georges Racinet <georges.racinet@octobus.net>
parents:
43077
diff
changeset
|
479 % (roundtrips, stats['undecided'], len(sample)) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
480 ) |
42103
362726923ba3
discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents:
41979
diff
changeset
|
481 |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
482 # indices between sample and externalized version must match |
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
483 sample = list(sample) |
37630
e1b32dc4646c
wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36718
diff
changeset
|
484 |
e1b32dc4646c
wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36718
diff
changeset
|
485 with remote.commandexecutor() as e: |
49017
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
486 if audit is not None: |
f054a557aab8
discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
48946
diff
changeset
|
487 audit[b'total-queries'] += len(sample) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
488 yesno = e.callcommand( |
45942
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
489 b'known', |
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
490 { |
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
491 b'nodes': [clnode(r) for r in sample], |
89a2afe31e82
formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents:
43506
diff
changeset
|
492 }, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
493 ).result() |
37630
e1b32dc4646c
wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36718
diff
changeset
|
494 |
14624
f03c82d1f50a
setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
14206
diff
changeset
|
495 full = True |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
496 |
41171
f46ffd23dae8
discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents:
41170
diff
changeset
|
497 disco.addinfo(zip(sample, yesno)) |
23343
f8a2647fe020
setdiscovery: avoid a full changelog graph traversal
Siddharth Agarwal <sid0@fb.com>
parents:
23192
diff
changeset
|
498 |
41113
9815d3337f9b
discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents:
41112
diff
changeset
|
499 result = disco.commonheads() |
32712
43bda143e3b2
discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
32291
diff
changeset
|
500 elapsed = util.timer() - start |
38373
ef692614e601
progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents:
38350
diff
changeset
|
501 progress.complete() |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
502 ui.debug(b"%d total queries in %.4fs\n" % (roundtrips, elapsed)) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
503 msg = ( |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
504 b'found %d common and %d unknown server heads,' |
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
505 b' %d roundtrips in %.4fs\n' |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
506 ) |
41878
82884bbf8d2b
discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents:
41304
diff
changeset
|
507 missing = set(result) - set(knownsrvheads) |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
508 ui.log(b'discovery', msg, len(result), len(missing), roundtrips, elapsed) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
509 |
46110
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
510 if audit is not None: |
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
511 audit[b'total-roundtrips'] = roundtrips |
d90f439ff19f
debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
45942
diff
changeset
|
512 |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
513 if not result and srvheadhashes != [cl.nullid]: |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
514 if abortwhenunrelated: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
515 raise error.Abort(_(b"repository is unrelated")) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
516 else: |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
517 ui.warn(_(b"warning: repository is unrelated\n")) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
518 return ( |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
519 {cl.nullid}, |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
520 True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
521 srvheadhashes, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
42884
diff
changeset
|
522 ) |
14164
cb98fed52495
discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff
changeset
|
523 |
47012
d55b71393907
node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents:
46841
diff
changeset
|
524 anyincoming = srvheadhashes != [cl.nullid] |
39159
5b32b3c618b2
setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38373
diff
changeset
|
525 result = {clnode(r) for r in result} |
5b32b3c618b2
setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38373
diff
changeset
|
526 return result, anyincoming, srvheadhashes |