annotate contrib/perf-utils/discovery-helper.sh @ 41925:4f283b7dac44

discovery-helper: add an extra argument to generate only one repo This is useful to generate left and right in parallel when dealing with very large repositories.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Fri, 08 Mar 2019 21:38:57 +0100
parents c1017bceb6ec
children cb6c9d41a23e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
1 #!/bin/bash
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
2 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
3 # produces two repositories with different common and missing subsets
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
4 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
5 # $ discovery-helper.sh REPO NBHEADS DEPT
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
6 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
7 # The Goal is to produce two repositories with some common part and some
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
8 # exclusive part on each side. Provide a source repository REPO, it will
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
9 # produce two repositories REPO-left and REPO-right.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
10 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
11 # Each repository will be missing some revisions exclusive to NBHEADS of the
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
12 # repo topological heads. These heads and revisions exclusive to them (up to
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
13 # DEPTH depth) are stripped.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
14 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
15 # The "left" repository will use the NBHEADS first heads (sorted by
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
16 # description). The "right" use the last NBHEADS one.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
17 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
18 # To find out how many topological heads a repo has, use:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
19 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
20 # $ hg heads -t -T '{rev}\n' | wc -l
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
21 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
22 # Example:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
23 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
24 # The `pypy-2018-09-01` repository has 192 heads. To produce two repositories
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
25 # with 92 common heads and ~50 exclusive heads on each side.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
26 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
27 # $ ./discovery-helper.sh pypy-2018-08-01 50 10
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
28
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
29 set -euo pipefail
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
30
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
31 printusage () {
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
32 echo "usage: `basename $0` REPO NBHEADS DEPTH [left|right]" >&2
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
33 }
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
34
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
35 if [ $# -lt 3 ]; then
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
36 printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
37 exit 64
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
38 fi
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
39
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
40 repo="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
41 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
42
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
43 nbheads="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
44 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
45
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
46 depth="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
47 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
48
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
49 doleft=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
50 doright=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
51 if [ $# -gt 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
52 printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
53 exit 64
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
54 elif [ $# -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
55 if [ "$1" == "left" ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
56 doleft=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
57 doright=0
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
58 elif [ "$1" == "right" ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
59 doleft=0
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
60 doright=1
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
61 else
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
62 printusage
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
63 exit 64
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
64 fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
65 fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
66
41895
c1017bceb6ec discovery-helper: reflect argument value in the name of the results
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41894
diff changeset
67 leftrepo="${repo}-${nbheads}h-${depth}d-left"
c1017bceb6ec discovery-helper: reflect argument value in the name of the results
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41894
diff changeset
68 rightrepo="${repo}-${nbheads}h-${depth}d-right"
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
69
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
70 left="first(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
71 right="last(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
72
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
73 leftsubset="ancestors($left, $depth) and only($left, heads(all() - $left))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
74 rightsubset="ancestors($right, $depth) and only($right, heads(all() - $right))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
75
41894
aea6bc63884d discovery-helper: echo the stripped revsets early
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41893
diff changeset
76 echo '### creating left/right repositories with missing changesets:'
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
77 if [ $doleft -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
78 echo '# left revset:' '"'${leftsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
79 fi
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
80 if [ $doright -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
81 echo '# right revset:' '"'${rightsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
82 fi
41894
aea6bc63884d discovery-helper: echo the stripped revsets early
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41893
diff changeset
83
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
84 if [ $doleft -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
85 echo '### building left repository:' $left-repo
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
86 echo '# cloning'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
87 hg clone --noupdate "${repo}" "${leftrepo}"
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
88 echo '# stripping' '"'${leftsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
89 hg -R "${leftrepo}" --config extensions.strip= strip --rev "$leftsubset" --no-backup
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
90 fi
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
91
41925
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
92 if [ $doright -eq 1 ]; then
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
93 echo '### building right repository:' $right-repo
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
94 echo '# cloning'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
95 hg clone --noupdate "${repo}" "${rightrepo}"
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
96 echo '# stripping:' '"'${rightsubset}'"'
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
97 hg -R "${rightrepo}" --config extensions.strip= strip --rev "$rightsubset" --no-backup
4f283b7dac44 discovery-helper: add an extra argument to generate only one repo
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41895
diff changeset
98 fi