annotate contrib/discovery-helper.sh @ 41426:02d0a7774882

py3: byteify the LFS blobstore module This is almost entirely b'' prefixing, with a couple of exceptions forced to bytes. Much of this is also borrowed from Augie's code. There's an HTTPError.read() that I flagged that I assume needs to be converted to bytes, but I can't find confirmation. Handling the deserialized JSON object over several functions made r'' vs b'' accesses confusing, so this assumes that the JSON object will be converted to bytes immediately. That will be done in the following commits, so it's not buried in these trivial changes.
author Matt Harbison <matt_harbison@yahoo.com>
date Sun, 27 Jan 2019 15:19:28 -0500
parents 01c335afc997
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
40805
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
1 #!/bin/bash
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
2 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
3 # produces two repositories with different common and missing subsets
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
4 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
5 # $ discovery-helper.sh REPO NBHEADS DEPT
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
6 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
7 # The Goal is to produce two repositories with some common part and some
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
8 # exclusive part on each side. Provide a source repository REPO, it will
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
9 # produce two repositories REPO-left and REPO-right.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
10 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
11 # Each repository will be missing some revisions exclusive to NBHEADS of the
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
12 # repo topological heads. These heads and revisions exclusive to them (up to
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
13 # DEPTH depth) are stripped.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
14 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
15 # The "left" repository will use the NBHEADS first heads (sorted by
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
16 # description). The "right" use the last NBHEADS one.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
17 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
18 # To find out how many topological heads a repo has, use:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
19 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
20 # $ hg heads -t -T '{rev}\n' | wc -l
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
21 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
22 # Example:
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
23 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
24 # The `pypy-2018-09-01` repository has 192 heads. To produce two repositories
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
25 # with 92 common heads and ~50 exclusive heads on each side.
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
26 #
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
27 # $ ./discovery-helper.sh pypy-2018-08-01 50 10
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
28
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
29 set -euo pipefail
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
30
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
31 if [ $# -lt 3 ]; then
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
32 echo "usage: `basename $0` REPO NBHEADS DEPTH"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
33 exit 64
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
34 fi
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
35
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
36 repo="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
37 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
38
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
39 nbheads="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
40 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
41
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
42 depth="$1"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
43 shift
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
44
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
45 leftrepo="${repo}-left"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
46 rightrepo="${repo}-right"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
47
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
48 left="first(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
49 right="last(sort(heads(all()), 'desc'), $nbheads)"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
50
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
51 leftsubset="ancestors($left, $depth) and only($left, heads(all() - $left))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
52 rightsubset="ancestors($right, $depth) and only($right, heads(all() - $right))"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
53
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
54 echo '### building left repository:' $left-repo
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
55 echo '# cloning'
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
56 hg clone --noupdate "${repo}" "${leftrepo}"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
57 echo '# stripping' '"'${leftsubset}'"'
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
58 hg -R "${leftrepo}" --config extensions.strip= strip --rev "$leftsubset" --no-backup
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
59
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
60 echo '### building right repository:' $right-repo
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
61 echo '# cloning'
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
62 hg clone --noupdate "${repo}" "${rightrepo}"
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
63 echo '# stripping:' '"'${rightsubset}'"'
01c335afc997 contrib: add a helper script that help to build interesting repositories
Boris Feld <boris.feld@octobus.net>
parents:
diff changeset
64 hg -R "${rightrepo}" --config extensions.strip= strip --rev "$rightsubset" --no-backup