annotate contrib/setup-pytype.sh @ 50400:95acba2c29f6

encoding: avoid quadratic time complexity when json-encoding non-UTF8 strings Apparently the code uses "+=" with a bytes object, which is linear-time, so the whole encoding is quadratic-time. This patch makes us use a bytearray object, instead, which has a(n amortized-)constant-time append operation. The encoding is still not particularly fast, but at least a 10MB file takes tens of seconds, not many hours to encode.
author Arseniy Alekseyev <aalekseyev@janestreet.com>
date Mon, 06 Mar 2023 11:27:57 +0000
parents 03792c1ed341
children 6aa74bcd4255
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
49652
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
1 #!/bin/bash
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
2
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
3 set -e
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
4 set -u
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
5
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
6 # Find the python3 setup that would run pytype
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
7 PYTYPE=`which pytype`
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
8 PYTHON3=`head -n1 ${PYTYPE} | sed -s 's/#!//'`
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
9
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
10 # Existing stubs that pytype processes live here
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
11 TYPESHED=$(${PYTHON3} -c "import pytype; print(pytype.__path__[0])")/typeshed/stubs
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
12 HG_STUBS=${TYPESHED}/mercurial
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
13
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
14 echo "Patching typeshed at $HG_STUBS"
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
15
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
16 rm -rf ${HG_STUBS}
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
17 mkdir -p ${HG_STUBS}
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
18
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
19 cat > ${HG_STUBS}/METADATA.toml <<EOF
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
20 version = "0.1"
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
21 EOF
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
22
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
23
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
24 mkdir -p ${HG_STUBS}/mercurial/cext ${HG_STUBS}/mercurial/thirdparty/attr
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
25
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
26 touch ${HG_STUBS}/mercurial/__init__.pyi
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
27 touch ${HG_STUBS}/mercurial/cext/__init__.pyi
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
28 touch ${HG_STUBS}/mercurial/thirdparty/__init__.pyi
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
29
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
30 ln -sf $(hg root)/mercurial/cext/*.{pyi,typed} \
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
31 ${HG_STUBS}/mercurial/cext
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
32 ln -sf $(hg root)/mercurial/thirdparty/attr/*.{pyi,typed} \
03792c1ed341 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
33 ${HG_STUBS}/mercurial/thirdparty/attr