annotate mercurial/setdiscovery.py @ 50896:b2b8c25f9462

hgwebmod: use sysstr to check for attribute presence We do not need bytes here.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 30 Aug 2023 13:28:09 +0200
parents c217d94cdd9d
children 493034cc3265
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
1 # setdiscovery.py - improved discovery of common nodeset for mercurial
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
2 #
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
3 # Copyright 2010 Benoit Boissinot <bboissin@gmail.com>
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
4 # and Peter Arrenbrecht <peter@arrenbrecht.ch>
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
5 #
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
6 # This software may be used and distributed according to the terms of the
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
7 # GNU General Public License version 2 or any later version.
20656
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
8 """
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
9 Algorithm works in the following way. You have two repository: local and
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
10 remote. They both contains a DAG of changelists.
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
11
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
12 The goal of the discovery protocol is to find one set of node *common*,
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
13 the set of nodes shared by local and remote.
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
14
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
15 One of the issue with the original protocol was latency, it could
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
16 potentially require lots of roundtrips to discover that the local repo was a
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
17 subset of remote (which is a very common case, you usually have few changes
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
18 compared to upstream, while upstream probably had lots of development).
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
19
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
20 The new protocol only requires one interface for the remote repo: `known()`,
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
21 which given a set of changelists tells you if they are present in the DAG.
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
22
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
23 The algorithm then works as follow:
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
24
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
25 - We will be using three sets, `common`, `missing`, `unknown`. Originally
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
26 all nodes are in `unknown`.
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
27 - Take a sample from `unknown`, call `remote.known(sample)`
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
28 - For each node that remote knows, move it and all its ancestors to `common`
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
29 - For each node that remote doesn't know, move it and all its descendants
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
30 to `missing`
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
31 - Iterate until `unknown` is empty
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
32
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
33 There are a couple optimizations, first is instead of starting with a random
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
34 sample of missing, start by sending all heads, in the case where the local
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
35 repo is a subset, you computed the answer in one round trip.
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
36
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
37 Then you can do something similar to the bisecting strategy used when
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
38 finding faulty changesets. Instead of random samples, you can try picking
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
39 nodes that will maximize the number of nodes that will be
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
40 classified with it (since all ancestors or descendants will be marked as well).
cdecbc5ab504 setdiscovery: document algorithms used
Olle Lundberg <geek@nerd.sh>
parents: 20034
diff changeset
41 """
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
42
25973
fb5664eb8414 setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25914
diff changeset
43
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 23817
diff changeset
44 import collections
20034
1e5b38a919dd cleanup: move stdlib imports to their own import statement
Augie Fackler <raf@durin42.com>
parents: 17426
diff changeset
45 import random
25973
fb5664eb8414 setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25914
diff changeset
46
fb5664eb8414 setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25914
diff changeset
47 from .i18n import _
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
48 from .node import nullrev
25973
fb5664eb8414 setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25914
diff changeset
49 from . import (
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25973
diff changeset
50 error,
42745
4d20b1fe8a72 rust-discovery: using from Python code
Georges Racinet <georges.racinet@octobus.net>
parents: 42742
diff changeset
51 policy,
32712
43bda143e3b2 discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32291
diff changeset
52 util,
25973
fb5664eb8414 setdiscovery: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25914
diff changeset
53 )
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
54
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
55
39174
71d83b315778 setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39173
diff changeset
56 def _updatesample(revs, heads, sample, parentfn, quicksamplesize=0):
23809
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
57 """update an existing sample to match the expected size
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
58
39168
2d218db7389b setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39166
diff changeset
59 The sample is updated with revs exponentially distant from each head of the
2d218db7389b setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39166
diff changeset
60 <revs> set. (H~1, H~2, H~4, H~8, etc).
23809
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
61
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
62 If a target size is specified, the sampling will stop once this size is
39168
2d218db7389b setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39166
diff changeset
63 reached. Otherwise sampling will happen until roots of the <revs> set are
23809
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
64 reached.
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
65
39168
2d218db7389b setdiscovery: reflect use of revs instead of nodes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39166
diff changeset
66 :revs: set of revs we want to discover (if None, assume the whole dag)
39170
754f389b87f2 setdiscovery: pass heads into _updatesample()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39169
diff changeset
67 :heads: set of DAG head revs
23809
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
68 :sample: a sample to update
39174
71d83b315778 setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39173
diff changeset
69 :parentfn: a callable to resolve parents for a revision
23809
9ca2eb881b53 setdiscovery: document the '_updatesample' function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23808
diff changeset
70 :quicksamplesize: optional target size of the sample"""
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
71 dist = {}
25113
0ca8410ea345 util: drop alias for collections.deque
Martin von Zweigbergk <martinvonz@google.com>
parents: 23817
diff changeset
72 visit = collections.deque(heads)
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
73 seen = set()
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
74 factor = 1
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
75 while visit:
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
76 curr = visit.popleft()
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
77 if curr in seen:
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
78 continue
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
79 d = dist.setdefault(curr, 1)
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
80 if d > factor:
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
81 factor *= 2
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
82 if d == factor:
23814
6a5877a73141 setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23813
diff changeset
83 sample.add(curr)
6a5877a73141 setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23813
diff changeset
84 if quicksamplesize and (len(sample) >= quicksamplesize):
6a5877a73141 setdiscovery: drop the 'always' argument to '_updatesample'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23813
diff changeset
85 return
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
86 seen.add(curr)
39174
71d83b315778 setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39173
diff changeset
87
71d83b315778 setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39173
diff changeset
88 for p in parentfn(curr):
71d83b315778 setdiscovery: don't use dagutil for parent resolution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39173
diff changeset
89 if p != nullrev and (not revs or p in revs):
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
90 dist.setdefault(p, d + 1)
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
91 visit.append(p)
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
92
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
93
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
94 def _limitsample(sample, desiredlen, randomize=True):
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
95 """return a random subset of sample of at most desiredlen item.
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
96
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
97 If randomize is False, though, a deterministic subset is returned.
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
98 This is meant for integration tests.
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
99 """
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
100 if len(sample) <= desiredlen:
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
101 return sample
49647
c217d94cdd9d python-compat: adapt to Python 3.11 BC breakage with `random.sample`
Raphaël Gomès <rgomes@octobus.net>
parents: 49017
diff changeset
102 sample = list(sample)
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
103 if randomize:
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
104 return set(random.sample(sample, desiredlen))
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
105 sample.sort()
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
106 return set(sample[:desiredlen])
23083
ee45f5c2ffcc setdiscovery: extract sample limitation in a `_limitsample` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20656
diff changeset
107
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
108
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
109 class partialdiscovery:
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
110 """an object representing ongoing discovery
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
111
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
112 Feed with data from the remote repository, this object keep track of the
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
113 current set of changeset in various states:
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
114
41172
3dcc96582627 discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents: 41171
diff changeset
115 - common: revs also known remotely
3dcc96582627 discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents: 41171
diff changeset
116 - undecided: revs we don't have information on yet
3dcc96582627 discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents: 41171
diff changeset
117 - missing: revs missing remotely
3dcc96582627 discovery: improve partial discovery documentation
Boris Feld <boris.feld@octobus.net>
parents: 41171
diff changeset
118 (all tracked revisions are known locally)
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
119 """
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
120
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
121 def __init__(self, repo, targetheads, respectsize, randomize=True):
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
122 self._repo = repo
41167
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
123 self._targetheads = targetheads
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
124 self._common = repo.changelog.incrementalmissingrevs()
41167
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
125 self._undecided = None
41170
96201120cdf5 discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41169
diff changeset
126 self.missing = set()
41885
5baf06d2bb41 discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41884
diff changeset
127 self._childrenmap = None
42402
5b34972a0094 setdiscovery: make progress on most connected groups each roundtrip
Martin von Zweigbergk <martinvonz@google.com>
parents: 42354
diff changeset
128 self._respectsize = respectsize
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
129 self.randomize = randomize
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
130
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
131 def addcommons(self, commons):
42170
6631f3e89b6f setdiscovery: fix a few typos
Joerg Sonnenberger <joerg@bec.de>
parents: 42159
diff changeset
132 """register nodes known as common"""
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
133 self._common.addbases(commons)
41304
76873548b051 partialdiscovery: avoid `undecided` related computation sooner than necessary
Boris Feld <boris.feld@octobus.net>
parents: 41280
diff changeset
134 if self._undecided is not None:
76873548b051 partialdiscovery: avoid `undecided` related computation sooner than necessary
Boris Feld <boris.feld@octobus.net>
parents: 41280
diff changeset
135 self._common.removeancestorsfrom(self._undecided)
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
136
41170
96201120cdf5 discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41169
diff changeset
137 def addmissings(self, missings):
42170
6631f3e89b6f setdiscovery: fix a few typos
Joerg Sonnenberger <joerg@bec.de>
parents: 42159
diff changeset
138 """register some nodes as missing"""
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
139 newmissing = self._repo.revs(b'%ld::%ld', missings, self.undecided)
41280
f4277a35c42c discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents: 41245
diff changeset
140 if newmissing:
f4277a35c42c discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents: 41245
diff changeset
141 self.missing.update(newmissing)
f4277a35c42c discovery: compute newly discovered missing in a more efficient way
Boris Feld <boris.feld@octobus.net>
parents: 41245
diff changeset
142 self.undecided.difference_update(newmissing)
41170
96201120cdf5 discovery: move missing tracking inside the partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41169
diff changeset
143
41171
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
144 def addinfo(self, sample):
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
145 """consume an iterable of (rev, known) tuples"""
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
146 common = set()
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
147 missing = set()
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
148 for rev, known in sample:
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
149 if known:
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
150 common.add(rev)
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
151 else:
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
152 missing.add(rev)
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
153 if common:
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
154 self.addcommons(common)
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
155 if missing:
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
156 self.addmissings(missing)
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
157
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
158 def hasinfo(self):
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
159 """return True is we have any clue about the remote state"""
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
160 return self._common.hasbases()
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
161
41169
3ce5b96482c6 discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents: 41168
diff changeset
162 def iscomplete(self):
3ce5b96482c6 discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents: 41168
diff changeset
163 """True if all the necessary data have been gathered"""
3ce5b96482c6 discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents: 41168
diff changeset
164 return self._undecided is not None and not self._undecided
3ce5b96482c6 discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents: 41168
diff changeset
165
41167
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
166 @property
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
167 def undecided(self):
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
168 if self._undecided is not None:
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
169 return self._undecided
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
170 self._undecided = set(self._common.missingancestors(self._targetheads))
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
171 return self._undecided
870a89c6909d discovery: move undecided set on the partialdiscovery
Boris Feld <boris.feld@octobus.net>
parents: 41162
diff changeset
172
42103
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
173 def stats(self):
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
174 return {
43162
3c6976b1f693 py3-discovery: using plain str in stats dict
Georges Racinet <georges.racinet@octobus.net>
parents: 43077
diff changeset
175 'undecided': len(self.undecided),
42103
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
176 }
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
177
41113
9815d3337f9b discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41112
diff changeset
178 def commonheads(self):
9815d3337f9b discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41112
diff changeset
179 """the heads of the known common set"""
9815d3337f9b discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41112
diff changeset
180 # heads(common) == heads(common.bases) since common represents
9815d3337f9b discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41112
diff changeset
181 # common.bases and all its ancestors
41245
2a8782cc2e16 discovery: using the new basesheads()
Georges Racinet <georges.racinet@octobus.net>
parents: 41172
diff changeset
182 return self._common.basesheads()
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
183
41881
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
184 def _parentsgetter(self):
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
185 getrev = self._repo.changelog.index.__getitem__
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
186
41881
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
187 def getparents(r):
41979
0d467e4de4ae discovery: fix embarrassing typo in slice definition
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41889
diff changeset
188 return getrev(r)[5:7]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
189
41881
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
190 return getparents
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
191
41886
a05f0bbefdd9 discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41885
diff changeset
192 def _childrengetter(self):
41884
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
193
41885
5baf06d2bb41 discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41884
diff changeset
194 if self._childrenmap is not None:
41889
3ba9ca537f57 discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41886
diff changeset
195 # During discovery, the `undecided` set keep shrinking.
3ba9ca537f57 discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41886
diff changeset
196 # Therefore, the map computed for an iteration N will be
3ba9ca537f57 discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41886
diff changeset
197 # valid for iteration N+1. Instead of computing the same
3ba9ca537f57 discovery: clarify why the caching of children is valid
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41886
diff changeset
198 # data over and over we cached it the first time.
41885
5baf06d2bb41 discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41884
diff changeset
199 return self._childrenmap.__getitem__
5baf06d2bb41 discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41884
diff changeset
200
41884
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
201 # _updatesample() essentially does interaction over revisions to look
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
202 # up their children. This lookup is expensive and doing it in a loop is
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
203 # quadratic. We precompute the children for all relevant revisions and
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
204 # make the lookup in _updatesample() a simple dict lookup.
41885
5baf06d2bb41 discovery: cache the children mapping used during each discovery
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41884
diff changeset
205 self._childrenmap = children = {}
41884
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
206
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
207 parentrevs = self._parentsgetter()
41886
a05f0bbefdd9 discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41885
diff changeset
208 revs = self.undecided
41884
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
209
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
210 for rev in sorted(revs):
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
211 # Always ensure revision has an entry so we don't need to worry
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
212 # about missing keys.
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
213 children[rev] = []
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
214 for prev in parentrevs(rev):
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
215 if prev == nullrev:
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
216 continue
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
217 c = children.get(prev)
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
218 if c is not None:
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
219 c.append(rev)
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
220 return children.__getitem__
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
221
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
222 def takequicksample(self, headrevs, size):
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
223 """takes a quick sample of size <size>
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
224
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
225 It is meant for initial sampling and focuses on querying heads and close
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
226 ancestors of heads.
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
227
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
228 :headrevs: set of head revisions in local DAG to consider
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
229 :size: the maximum size of the sample"""
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
230 revs = self.undecided
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
231 if len(revs) <= size:
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
232 return list(revs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
233 sample = set(self._repo.revs(b'heads(%ld)', revs))
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
234
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
235 if len(sample) >= size:
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
236 return _limitsample(sample, size, randomize=self.randomize)
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
237
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
238 _updatesample(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
239 None, headrevs, sample, self._parentsgetter(), quicksamplesize=size
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
240 )
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
241 return sample
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
242
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
243 def takefullsample(self, headrevs, size):
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
244 revs = self.undecided
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
245 if len(revs) <= size:
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
246 return list(revs)
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
247 repo = self._repo
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
248 sample = set(repo.revs(b'heads(%ld)', revs))
41881
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
249 parentrevs = self._parentsgetter()
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
250
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
251 # update from heads
41880
55919b96c02a discovery: avoid computing identical sets of heads twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41879
diff changeset
252 revsheads = sample.copy()
41881
e514799e4e07 discovery: use a lower level but faster way to retrieve parents
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41880
diff changeset
253 _updatesample(revs, revsheads, sample, parentrevs)
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
254
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
255 # update from roots
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
256 revsroots = set(repo.revs(b'roots(%ld)', revs))
41886
a05f0bbefdd9 discovery: explicitly use `undecided` for the children mapping
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41885
diff changeset
257 childrenrevs = self._childrengetter()
41884
d5e6ae6e8012 discovery: move children computation in its own method
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41883
diff changeset
258 _updatesample(revs, revsroots, sample, childrenrevs)
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
259 assert sample
42425
b9ff059fd194 discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42402
diff changeset
260
b9ff059fd194 discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42402
diff changeset
261 if not self._respectsize:
b9ff059fd194 discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42402
diff changeset
262 size = max(size, min(len(revsroots), len(revsheads)))
b9ff059fd194 discovery: be more conservative when adjusting the sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42402
diff changeset
263
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
264 sample = _limitsample(sample, size, randomize=self.randomize)
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
265 if len(sample) < size:
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
266 more = size - len(sample)
42741
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
267 takefrom = list(revs - sample)
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
268 if self.randomize:
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
269 sample.update(random.sample(takefrom, more))
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
270 else:
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
271 takefrom.sort()
4e7bd6180b53 rust-discovery: optionally don't randomize at all, for tests
Georges Racinet <georges.racinet@octobus.net>
parents: 42425
diff changeset
272 sample.update(takefrom[:more])
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
273 return sample
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
274
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
275
47269
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
276 pure_partialdiscovery = partialdiscovery
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
277
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
278 partialdiscovery = policy.importrust(
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43162
diff changeset
279 'discovery', member='PartialDiscovery', default=partialdiscovery
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
280 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
281
42745
4d20b1fe8a72 rust-discovery: using from Python code
Georges Racinet <georges.racinet@octobus.net>
parents: 42742
diff changeset
282
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
283 def findcommonheads(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
284 ui,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
285 local,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
286 remote,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
287 abortwhenunrelated=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
288 ancestorsof=None,
46110
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
289 audit=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
290 ):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
291 """Return a tuple (common, anyincoming, remoteheads) used to identify
14206
2bf60f158ecb setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents: 14164
diff changeset
292 missing nodes from or in remote.
46110
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
293
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
294 The audit argument is an optional dictionnary that a caller can pass. it
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
295 will be updated with extra data about the discovery, this is useful for
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
296 debug.
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
297 """
46300
f17b6e40a775 discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46299
diff changeset
298
f17b6e40a775 discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46299
diff changeset
299 samplegrowth = float(ui.config(b'devel', b'discovery.grow-sample.rate'))
f17b6e40a775 discovery: add a discovery.grow-sample.rate
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46299
diff changeset
300
49017
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
301 if audit is not None:
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
302 audit[b'total-queries'] = 0
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
303
32712
43bda143e3b2 discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32291
diff changeset
304 start = util.timer()
43bda143e3b2 discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32291
diff changeset
305
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
306 roundtrips = 0
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
307 cl = local.changelog
39159
5b32b3c618b2 setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38373
diff changeset
308 clnode = cl.node
39161
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
309 clrev = cl.rev
39159
5b32b3c618b2 setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38373
diff changeset
310
35304
f77121b6bf1b setdiscover: allow to ignore part of the local graph
Boris Feld <boris.feld@octobus.net>
parents: 32768
diff changeset
311 if ancestorsof is not None:
39165
860e83cd97de setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39161
diff changeset
312 ownheads = [clrev(n) for n in ancestorsof]
860e83cd97de setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39161
diff changeset
313 else:
860e83cd97de setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39161
diff changeset
314 ownheads = [rev for rev in cl.headrevs() if rev != nullrev]
860e83cd97de setdiscovery: don't use dagutil to compute heads
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39161
diff changeset
315
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
316 initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads')
46359
0600e8467101 discovery: add config options to control sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46316
diff changeset
317 initialsamplesize = ui.configint(b'devel', b'discovery.sample-size.initial')
0600e8467101 discovery: add config options to control sample size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46316
diff changeset
318 fullsamplesize = ui.configint(b'devel', b'discovery.sample-size')
42159
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
319 # We also ask remote about all the local heads. That set can be arbitrarily
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
320 # large, so we used to limit it size to `initialsamplesize`. We no longer
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
321 # do as it proved counter productive. The skipped heads could lead to a
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
322 # large "undecided" set, slower to be clarified than if we asked the
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
323 # question for all heads right away.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
324 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
325 # We are already fetching all server heads using the `heads` commands,
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
326 # sending a equivalent number of heads the other way should not have a
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
327 # significant impact. In addition, it is very likely that we are going to
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
328 # have to issue "known" request for an equivalent amount of revisions in
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
329 # order to decide if theses heads are common or missing.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
330 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
331 # find a detailled analysis below.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
332 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
333 # Case A: local and server both has few heads
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
334 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
335 # Ownheads is below initialsamplesize, limit would not have any effect.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
336 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
337 # Case B: local has few heads and server has many
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
338 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
339 # Ownheads is below initialsamplesize, limit would not have any effect.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
340 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
341 # Case C: local and server both has many heads
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
342 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
343 # We now transfert some more data, but not significantly more than is
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
344 # already transfered to carry the server heads.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
345 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
346 # Case D: local has many heads, server has few
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
347 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
348 # D.1 local heads are mostly known remotely
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
349 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
350 # All the known head will have be part of a `known` request at some
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
351 # point for the discovery to finish. Sending them all earlier is
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
352 # actually helping.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
353 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
354 # (This case is fairly unlikely, it requires the numerous heads to all
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
355 # be merged server side in only a few heads)
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
356 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
357 # D.2 local heads are mostly missing remotely
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
358 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
359 # To determine that the heads are missing, we'll have to issue `known`
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
360 # request for them or one of their ancestors. This amount of `known`
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
361 # request will likely be in the same order of magnitude than the amount
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
362 # of local heads.
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
363 #
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
364 # The only case where we can be more efficient using `known` request on
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
365 # ancestors are case were all the "missing" local heads are based on a
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
366 # few changeset, also "missing". This means we would have a "complex"
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
367 # graph (with many heads) attached to, but very independant to a the
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
368 # "simple" graph on the server. This is a fairly usual case and have
4f9a89837f07 setdiscovery: stop limiting the number of local head we initially send
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42103
diff changeset
369 # not been met in the wild so far.
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
370 if initial_head_exchange:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
371 if remote.limitedarguments:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
372 sample = _limitsample(ownheads, initialsamplesize)
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
373 # indices between sample and externalized version must match
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
374 sample = list(sample)
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
375 else:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
376 sample = ownheads
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
377
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
378 ui.debug(b"query 1; heads\n")
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
379 roundtrips += 1
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
380 with remote.commandexecutor() as e:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
381 fheads = e.callcommand(b'heads', {})
49017
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
382 if audit is not None:
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
383 audit[b'total-queries'] += len(sample)
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
384 fknown = e.callcommand(
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
385 b'known',
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
386 {
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
387 b'nodes': [clnode(r) for r in sample],
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
388 },
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
389 )
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
390
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
391 srvheadhashes, yesno = fheads.result(), fknown.result()
37631
2f626233859b wireproto: implement batching on peer executor interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37630
diff changeset
392
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
393 if audit is not None:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
394 audit[b'total-roundtrips'] = 1
46110
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
395
46841
e7b4607d52e3 setdiscovery: simplify by using tiprev directly
Joerg Sonnenberger <joerg@bec.de>
parents: 46805
diff changeset
396 if cl.tiprev() == nullrev:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
397 if srvheadhashes != [cl.nullid]:
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
398 return [cl.nullid], True, srvheadhashes
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
399 return [cl.nullid], False, []
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
400 else:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
401 # we still need the remote head for the function return
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
402 with remote.commandexecutor() as e:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
403 fheads = e.callcommand(b'heads', {})
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
404 srvheadhashes = fheads.result()
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
405
14206
2bf60f158ecb setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents: 14164
diff changeset
406 # start actual discovery (we note this before the next "if" for
2bf60f158ecb setdiscovery: limit lines to 80 characters
Steven Brown <StevenGBrown@gmail.com>
parents: 14164
diff changeset
407 # compatibility reasons)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
408 ui.status(_(b"searching for changes\n"))
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
409
41878
82884bbf8d2b discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents: 41304
diff changeset
410 knownsrvheads = [] # revnos of remote heads that are known locally
39161
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
411 for node in srvheadhashes:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
412 if node == cl.nullid:
39161
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
413 continue
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
414
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
415 try:
41878
82884bbf8d2b discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents: 41304
diff changeset
416 knownsrvheads.append(clrev(node))
39161
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
417 # Catches unknown and filtered nodes.
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
418 except error.LookupError:
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
419 continue
858a12846f4f setdiscovery: don't use dagutil for node -> rev conversion
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39159
diff changeset
420
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
421 if initial_head_exchange:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
422 # early exit if we know all the specified remote heads already
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
423 if len(knownsrvheads) == len(srvheadhashes):
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
424 ui.debug(b"all remote heads known locally\n")
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
425 return srvheadhashes, False, srvheadhashes
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
426
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
427 if len(sample) == len(ownheads) and all(yesno):
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
428 ui.note(_(b"all local changesets known remotely\n"))
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
429 ownheadhashes = [clnode(r) for r in ownheads]
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
430 return ownheadhashes, True, srvheadhashes
14624
f03c82d1f50a setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14206
diff changeset
431
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
432 # full blown discovery
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
433
46299
397e39ad0174 discovery: add a `devel', b'discovery.grow-sample`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46110
diff changeset
434 # if the server has a limit to its arguments size, we can't grow the sample.
46805
2b1b8f3e6510 setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46804
diff changeset
435 configbool = local.ui.configbool
2b1b8f3e6510 setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46804
diff changeset
436 grow_sample = configbool(b'devel', b'discovery.grow-sample')
46804
f165105400d0 setdiscovery: rearrange code deciding if we will grow the sample
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46359
diff changeset
437 grow_sample = grow_sample and not remote.limitedarguments
46299
397e39ad0174 discovery: add a `devel', b'discovery.grow-sample`
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46110
diff changeset
438
46805
2b1b8f3e6510 setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46804
diff changeset
439 dynamic_sample = configbool(b'devel', b'discovery.grow-sample.dynamic')
2b1b8f3e6510 setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46804
diff changeset
440 hard_limit_sample = not (dynamic_sample or remote.limitedarguments)
2b1b8f3e6510 setdiscovery: add a discovery.grow-sample.dynamic option
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46804
diff changeset
441
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
442 randomize = ui.configbool(b'devel', b'discovery.randomize')
47269
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
443 if cl.index.rust_ext_compat:
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
444 pd = partialdiscovery
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
445 else:
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
446 pd = pure_partialdiscovery
6be2a7ca4b1d revlog: do not call Rust code if the index is not compatible with it
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47012
diff changeset
447 disco = pd(local, ownheads, hard_limit_sample, randomize=randomize)
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
448 if initial_head_exchange:
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
449 # treat remote heads (and maybe own heads) as a first implicit sample
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
450 # response
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
451 disco.addcommons(knownsrvheads)
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
452 disco.addinfo(zip(sample, yesno))
16683
525fdb738975 cleanup: eradicate long lines
Brodie Rao <brodie@sf.io>
parents: 15713
diff changeset
453
46316
6ee9bd69ff7f discovery: add a devel.discovery.exchange-heads
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46301
diff changeset
454 full = not initial_head_exchange
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
455 progress = ui.makeprogress(_(b'searching'), unit=_(b'queries'))
41169
3ce5b96482c6 discovery: add a `iscomplete` method to the `partialdiscovery` object
Boris Feld <boris.feld@octobus.net>
parents: 41168
diff changeset
456 while not disco.iscomplete():
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
457
41112
3023bc4b3da0 discovery: introduce a partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41111
diff changeset
458 if full or disco.hasinfo():
23747
f82173a90c2c setdiscovery: factorize similar sampling code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23746
diff changeset
459 if full:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
460 ui.note(_(b"sampling from both directions\n"))
23747
f82173a90c2c setdiscovery: factorize similar sampling code
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23746
diff changeset
461 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
462 ui.debug(b"taking initial sample\n")
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
463 samplefunc = disco.takefullsample
23130
ced632394371 setdiscovery: limit the size of all sample (issue4411)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23084
diff changeset
464 targetsize = fullsamplesize
46804
f165105400d0 setdiscovery: rearrange code deciding if we will grow the sample
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46359
diff changeset
465 if grow_sample:
42354
dbd0fcca6dfc discovery: slowly increase sampling size
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42170
diff changeset
466 fullsamplesize = int(fullsamplesize * samplegrowth)
14624
f03c82d1f50a setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14206
diff changeset
467 else:
f03c82d1f50a setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14206
diff changeset
468 # use even cheaper initial sample
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
469 ui.debug(b"taking quick initial sample\n")
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
470 samplefunc = disco.takequicksample
23130
ced632394371 setdiscovery: limit the size of all sample (issue4411)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 23084
diff changeset
471 targetsize = initialsamplesize
41879
e5ece0f46b40 discovery: moved sampling functions inside discovery object
Georges Racinet <georges.racinet@octobus.net>
parents: 41878
diff changeset
472 sample = samplefunc(ownheads, targetsize)
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
473
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
474 roundtrips += 1
38350
9e70690a21ac setdiscovery: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 37631
diff changeset
475 progress.update(roundtrips)
42103
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
476 stats = disco.stats()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
477 ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
478 b"query %i; still undecided: %i, sample size is: %i\n"
43162
3c6976b1f693 py3-discovery: using plain str in stats dict
Georges Racinet <georges.racinet@octobus.net>
parents: 43077
diff changeset
479 % (roundtrips, stats['undecided'], len(sample))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
480 )
42103
362726923ba3 discovery: stop direct use of attribute of partialdiscovery
Georges Racinet <georges.racinet@octobus.net>
parents: 41979
diff changeset
481
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
482 # indices between sample and externalized version must match
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
483 sample = list(sample)
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36718
diff changeset
484
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36718
diff changeset
485 with remote.commandexecutor() as e:
49017
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
486 if audit is not None:
f054a557aab8 discovery: also audit the number of queries done
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48946
diff changeset
487 audit[b'total-queries'] += len(sample)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
488 yesno = e.callcommand(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
489 b'known',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
490 {
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
491 b'nodes': [clnode(r) for r in sample],
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43506
diff changeset
492 },
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
493 ).result()
37630
e1b32dc4646c wireproto: implement command executor interface for version 1 peers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36718
diff changeset
494
14624
f03c82d1f50a setdiscovery: batch heads and known(ownheads)
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14206
diff changeset
495 full = True
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
496
41171
f46ffd23dae8 discovery: add a simple `addinfo` method
Boris Feld <boris.feld@octobus.net>
parents: 41170
diff changeset
497 disco.addinfo(zip(sample, yesno))
23343
f8a2647fe020 setdiscovery: avoid a full changelog graph traversal
Siddharth Agarwal <sid0@fb.com>
parents: 23192
diff changeset
498
41113
9815d3337f9b discovery: move common heads computation inside partialdiscovery object
Boris Feld <boris.feld@octobus.net>
parents: 41112
diff changeset
499 result = disco.commonheads()
32712
43bda143e3b2 discovery: include timing in the debug output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32291
diff changeset
500 elapsed = util.timer() - start
38373
ef692614e601 progress: hide update(None) in a new complete() method
Martin von Zweigbergk <martinvonz@google.com>
parents: 38350
diff changeset
501 progress.complete()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
502 ui.debug(b"%d total queries in %.4fs\n" % (roundtrips, elapsed))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
503 msg = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
504 b'found %d common and %d unknown server heads,'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
505 b' %d roundtrips in %.4fs\n'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
506 )
41878
82884bbf8d2b discovery: rename `srvheads` to `knownsrvheads`
Georges Racinet <georges.racinet@octobus.net>
parents: 41304
diff changeset
507 missing = set(result) - set(knownsrvheads)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
508 ui.log(b'discovery', msg, len(result), len(missing), roundtrips, elapsed)
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
509
46110
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
510 if audit is not None:
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
511 audit[b'total-roundtrips'] = roundtrips
d90f439ff19f debugdiscovery: display the number of roundtrip used
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45942
diff changeset
512
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
513 if not result and srvheadhashes != [cl.nullid]:
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
514 if abortwhenunrelated:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
515 raise error.Abort(_(b"repository is unrelated"))
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
516 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
517 ui.warn(_(b"warning: repository is unrelated\n"))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
518 return (
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
519 {cl.nullid},
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
520 True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
521 srvheadhashes,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42884
diff changeset
522 )
14164
cb98fed52495 discovery: add new set-based discovery
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
diff changeset
523
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46841
diff changeset
524 anyincoming = srvheadhashes != [cl.nullid]
39159
5b32b3c618b2 setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38373
diff changeset
525 result = {clnode(r) for r in result}
5b32b3c618b2 setdiscovery: don't use dagutil for rev -> node conversions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38373
diff changeset
526 return result, anyincoming, srvheadhashes