Mercurial > hg
annotate contrib/perf-utils/search-discovery-case @ 48598:a6f16ec07ed7
stream-clone: add a explicit test for format change during stream clone
They are different kind of requirements, the one which impact the data storage
and are relevant to the files being streamed and the one which does not. For
example some requirements are only relevant to the working copy, like sparse, or
dirstate-v2.
Since they are irrelevant to the content being streamed, they do not prevent the
receiving side to use streaming clone and mercurial skip adverting them over
the wire and, ideally, within the bundle.
In addition, this let the client decide to use whichever format it desire for
the part that does not affect the store itself. So the configuration related to
these format are used as normal when doing a streaming clone.
In practice, the feature was not really tested and is badly broken with bundle-2,
since the requirements are not filtered out from the stream bundle.
So we start with adding simple tests as a good base before the fix and adjust
the feature.
Differential Revision: https://phab.mercurial-scm.org/D12029
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Mon, 17 Jan 2022 18:51:47 +0100 |
parents | 6b26e6432554 |
children | 6000f5b25c9b |
rev | line source |
---|---|
46771
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
1 #!/usr/bin/env python3 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
2 # Search for interesting discovery instance |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
3 # |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
4 # search-discovery-case REPO [REPO]… |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
5 # |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
6 # This use a subsetmaker extension (next to this script) to generate a steam of |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
7 # random discovery instance. When interesting case are discovered, information |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
8 # about them are print on the stdout. |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
9 from __future__ import print_function |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
10 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
11 import json |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
12 import os |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
13 import queue |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
14 import random |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
15 import signal |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
16 import subprocess |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
17 import sys |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
18 import threading |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
19 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
20 this_script = os.path.abspath(sys.argv[0]) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
21 this_dir = os.path.dirname(this_script) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
22 hg_dir = os.path.join(this_dir, '..', '..') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
23 HG_REPO = os.path.normpath(hg_dir) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
24 HG_BIN = os.path.join(HG_REPO, 'hg') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
25 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
26 JOB = int(os.environ.get('NUMBER_OF_PROCESSORS', 8)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
27 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
28 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
29 SLICING = ('scratch', 'randomantichain', 'rev') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
30 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
31 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
32 def nb_revs(repo_path): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
33 cmd = [ |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
34 HG_BIN, |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
35 '--repository', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
36 repo_path, |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
37 'log', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
38 '--template', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
39 '{rev}', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
40 '--rev', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
41 'tip', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
42 ] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
43 s = subprocess.Popen(cmd, stdout=subprocess.PIPE) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
44 out, err = s.communicate() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
45 return int(out) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
46 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
47 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
48 repos = [] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
49 for repo in sys.argv[1:]: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
50 size = nb_revs(repo) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
51 repos.append((repo, size)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
52 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
53 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
54 def pick_one(repo): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
55 pick = random.choice(SLICING) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
56 seed = random.randint(0, 100000) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
57 if pick == 'scratch': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
58 start = int(repo[1] * 0.3) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
59 end = int(repo[1] * 0.7) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
60 nb = random.randint(start, end) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
61 return ('scratch', nb, seed) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
62 elif pick == 'randomantichain': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
63 return ('randomantichain', seed) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
64 elif pick == 'rev': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
65 start = int(repo[1] * 0.3) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
66 end = int(repo[1]) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
67 rev = random.randint(start, end) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
68 return ('rev', rev) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
69 else: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
70 assert False |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
71 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
72 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
73 done = threading.Event() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
74 cases = queue.Queue(maxsize=10 * JOB) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
75 results = queue.Queue() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
76 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
77 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
78 def worker(): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
79 while not done.is_set(): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
80 c = cases.get() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
81 if c is None: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
82 return |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
83 try: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
84 res = process(c) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
85 results.put((c, res)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
86 except Exception as exc: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
87 print('processing-failed: %s %s' % (c, exc), file=sys.stderr) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
88 c = (c[0], c[2], c[1]) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
89 try: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
90 res = process(c) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
91 results.put((c, res)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
92 except Exception as exc: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
93 print('processing-failed: %s %s' % (c, exc), file=sys.stderr) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
94 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
95 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
96 SUBSET_PATH = os.path.join(HG_REPO, 'contrib', 'perf-utils', 'subsetmaker.py') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
97 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
98 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
99 CMD_BASE = ( |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
100 HG_BIN, |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
101 'debugdiscovery', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
102 '--template', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
103 'json', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
104 '--config', |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
105 'extensions.subset=%s' % SUBSET_PATH, |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
106 ) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
107 # '--local-as-revs "$left" --local-as-revs "$right"' |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
108 # > /data/discovery-references/results/disco-mozilla-unified-$1-$2.txt |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
109 # ) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
110 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
111 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
112 def to_revsets(case): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
113 t = case[0] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
114 if t == 'scratch': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
115 return 'not scratch(all(), %d, "%d")' % (case[1], case[2]) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
116 elif t == 'randomantichain': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
117 return '::randomantichain(all(), "%d")' % case[1] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
118 elif t == 'rev': |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
119 return '::%d' % case[1] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
120 else: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
121 assert False |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
122 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
123 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
124 def process(case): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
125 (repo, left, right) = case |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
126 cmd = list(CMD_BASE) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
127 cmd.append('-R') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
128 cmd.append(repo[0]) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
129 cmd.append('--local-as-revs') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
130 cmd.append(to_revsets(left)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
131 cmd.append('--remote-as-revs') |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
132 cmd.append(to_revsets(right)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
133 s = subprocess.Popen(cmd, stdout=subprocess.PIPE) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
134 out, err = s.communicate() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
135 return json.loads(out)[0] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
136 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
137 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
138 def interesting_boundary(res): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
139 """check if a case is interesting or not |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
140 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
141 For now we are mostly interrested in case were we do multiple roundstrip |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
142 and where the boundary is somewhere in the middle of the undecided set. |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
143 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
144 Ideally, we would make this configurable, but this is not a focus for now |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
145 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
146 return None or (round-trip, undecided-common, undecided-missing) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
147 """ |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
148 roundtrips = res["total-roundtrips"] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
149 if roundtrips <= 1: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
150 return None |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
151 undecided_common = res["nb-ini_und-common"] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
152 undecided_missing = res["nb-ini_und-missing"] |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
153 if undecided_common == 0: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
154 return None |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
155 if undecided_missing == 0: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
156 return None |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
157 return (roundtrips, undecided_common, undecided_missing) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
158 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
159 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
160 def end(*args, **kwargs): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
161 done.set() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
162 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
163 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
164 def format_case(case): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
165 return '-'.join(str(s) for s in case) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
166 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
167 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
168 signal.signal(signal.SIGINT, end) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
169 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
170 for i in range(JOB): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
171 threading.Thread(target=worker).start() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
172 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
173 nb_cases = 0 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
174 while not done.is_set(): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
175 repo = random.choice(repos) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
176 left = pick_one(repo) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
177 right = pick_one(repo) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
178 cases.put((repo, left, right)) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
179 while not results.empty(): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
180 # results has a single reader so this is fine |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
181 c, res = results.get_nowait() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
182 boundary = interesting_boundary(res) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
183 if boundary is not None: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
184 print(c[0][0], format_case(c[1]), format_case(c[2]), *boundary) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
185 sys.stdout.flush() |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
186 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
187 nb_cases += 1 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
188 if not nb_cases % 100: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
189 print('[%d cases generated]' % nb_cases, file=sys.stderr) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
190 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
191 for i in range(JOB): |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
192 try: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
193 cases.put_nowait(None) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
194 except queue.Full: |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
195 pass |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
196 |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
197 print('[%d cases generated]' % nb_cases, file=sys.stderr) |
6b26e6432554
perf-helpers: add a search-discovery-case script
Pierre-Yves David <pierre-yves.david@octobus.net>
parents:
diff
changeset
|
198 print('[ouput generation is over]' % nb_cases, file=sys.stderr) |