annotate contrib/packaging/packagelib.sh @ 50400:95acba2c29f6

encoding: avoid quadratic time complexity when json-encoding non-UTF8 strings Apparently the code uses "+=" with a bytes object, which is linear-time, so the whole encoding is quadratic-time. This patch makes us use a bytearray object, instead, which has a(n amortized-)constant-time append operation. The encoding is still not particularly fast, but at least a 10MB file takes tens of seconds, not many hours to encode.
author Arseniy Alekseyev <aalekseyev@janestreet.com>
date Mon, 06 Mar 2023 11:27:57 +0000
parents ccccd5064c6c
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26833
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
1 # Extract version number into 4 parts, some of which may be empty:
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
2 #
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
3 # version: the numeric part of the most recent tag. Will always look like 1.3.
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
4 #
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
5 # type: if an rc build, "rc", otherwise empty
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
6 #
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
7 # distance: the distance from the nearest tag, or empty if built from a tag
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
8 #
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
9 # node: the node|short hg was built from, or empty if built from a tag
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
10 gethgversion() {
45983
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
11 if [ -z "${1+x}" ]; then
49418
ccccd5064c6c packagelib: use python3 by default
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 45983
diff changeset
12 python="python3"
45983
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
13 else
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
14 python="$1"
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
15 fi
34903
16d9f0b3e134 build: build deb/rpm independently on config/extensions in the host system
muxator <a.mux@inwind.it>
parents: 34686
diff changeset
16 export HGRCPATH=
16d9f0b3e134 build: build deb/rpm independently on config/extensions in the host system
muxator <a.mux@inwind.it>
parents: 34686
diff changeset
17 export HGPLAIN=
16d9f0b3e134 build: build deb/rpm independently on config/extensions in the host system
muxator <a.mux@inwind.it>
parents: 34686
diff changeset
18
45983
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
19 make cleanbutpackages PYTHON=$python
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
20 make local PURE=--pure PYTHON=$python
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
21 HG="$PWD/hg"
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
22
45983
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
23 $python "$HG" version > /dev/null || { echo 'abort: hg version failed!'; exit 1 ; }
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
24
45983
97205cf0ee4d packaging: don't use plain 'python' if another python has been specified
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 41613
diff changeset
25 hgversion=`LANGUAGE=C $python "$HG" version | sed -ne 's/.*(version \(.*\))$/\1/p'`
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
26
26833
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
27 if echo $hgversion | grep + > /dev/null 2>&1 ; then
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
28 tmp=`echo $hgversion | cut -d+ -f 2`
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
29 hgversion=`echo $hgversion | cut -d+ -f 1`
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
30 distance=`echo $tmp | cut -d- -f 1`
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
31 node=`echo $tmp | cut -d- -f 2`
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
32 else
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
33 distance=''
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
34 node=''
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
35 fi
41613
8f0e8b179842 packaging: modify rc detection to work with X.Yrc instead of X.Y-rc
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 38005
diff changeset
36 if echo $hgversion | grep -E -- '[0-9]\.[0-9](\.[0-9])?rc' > /dev/null 2>&1; then
8f0e8b179842 packaging: modify rc detection to work with X.Yrc instead of X.Y-rc
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 38005
diff changeset
37 version=`echo $hgversion | cut -d'r' -f1`
8f0e8b179842 packaging: modify rc detection to work with X.Yrc instead of X.Y-rc
Mathias De Mare <mathias.de_mare@nokia.com>
parents: 38005
diff changeset
38 type="rc`echo $hgversion | cut -d'c' -f2-`"
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
39 else
26833
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
40 version=$hgversion
6474b64045fb packaging: rework version detection and declaration (issue4912)
Augie Fackler <augie@google.com>
parents: 24972
diff changeset
41 type=''
24972
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
42 fi
56c64c91b429 packaging: extract packagelib for common code from builddeb and buildrpm
Augie Fackler <augie@google.com>
parents:
diff changeset
43 }