pyoxidizer: produce working Python 3 Windows installers (
issue6366)
While we've had code to produce Python 3 Windows installers with
PyOxidizer, we haven't been advertising them on the web site due to
a bug in making TLS connections and issues around resource handling.
This commit upgrades our PyOxidizer install and configuration to
use a recent Git commit of PyOxidizer. This new version of PyOxidizer
contains a *ton* of changes, improvements, and bug fixes. Notably,
Windows shared distributions now mostly "just work" and the TLS bug
and random problems with Python extension modules in the standard
library go away. And Python has been upgraded from 3.7 to 3.8.6.
The price we pay for this upgrade is a ton of backwards incompatible
changes to Starlark.
I applied this commit (the overall series actually) on stable to
produce Windows installers for Mercurial 5.5.2, which I published
shortly before submitting this commit for review.
In order to get the stable branch working, I decided to take a
less aggressive approach to Python resource management. Previously,
we were attempting to load all Python modules from memory and were
performing some hacks to copy Mercurial's non-module resources
into additional directories in Starlark. This commit implements
a resource callback function in Starlark (a new feature since
PyOxidizer 0.7) to dynamically assign standard library resources
to in-memory loading and all other resources to filesystem loading.
This means that Mercurial's files and all the other packages we ship
in the Windows installers (e.g. certifi and pygments) are loaded
from the filesystem instead of from memory. This avoids issues
due to lack of __file__ and enables us to ship a working Python
3 installer on Windows.
The end state of the install layout after this patch is not
ideal for @: we still copy resource files like templates and
help text to directories next to the hg.exe executable. There
is code in @ to use importlib.resources to load these files and
we could likely remove these copies once this lands on @. But for
now, the install layout mimics what we've shipped for seemingly
forever and is backwards compatible. It allows us to achieve the
milestone of working Python 3 Windows installers and gets us a
giant step closer to deleting Python 2.
Differential Revision: https://phab.mercurial-scm.org/D9148
$ . "$TESTDIR/histedit-helpers.sh"
$ cat >> $HGRCPATH <<EOF
> [extensions]
> histedit=
> EOF
$ initrepo ()
> {
> hg init $1
> cd $1
> for x in a b c d e f ; do
> echo $x$x$x$x$x > $x
> hg add $x
> done
> hg ci -m 'Initial commit'
> for x in a b c d e f ; do
> echo $x > $x
> hg ci -m $x
> done
> echo 'I can haz no commute' > e
> hg ci -m 'does not commute with e'
> cd ..
> }
$ initrepo r1
$ cd r1
Initial generation of the command files
$ EDITED="$TESTTMP/editedhistory"
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 3 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 4 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 7 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 5 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 6 >> $EDITED
$ cat $EDITED
pick 65a9a84f33fd 3 c
pick 00f1c5383965 4 d
pick 39522b764e3d 7 does not commute with e
pick 7b4e2f4b7bcd 5 e
pick 500cac37a696 6 f
log before edit
$ hg log --graph
@ changeset: 7:39522b764e3d
| tag: tip
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: does not commute with e
|
o changeset: 6:500cac37a696
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: f
|
o changeset: 5:7b4e2f4b7bcd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: e
|
o changeset: 4:00f1c5383965
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: d
|
o changeset: 3:65a9a84f33fd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: c
|
o changeset: 2:da6535b52e45
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: b
|
o changeset: 1:c1f09da44841
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: a
|
o changeset: 0:1715188a53c7
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: Initial commit
edit the history
$ hg histedit 3 --commands $EDITED 2>&1 | fixbundle
merging e
warning: conflicts while merging e! (edit, then use 'hg resolve --mark')
Fix up the change (pick 39522b764e3d)
(hg histedit --continue to resume)
abort the edit
$ hg histedit --abort 2>&1 | fixbundle
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
second edit set
$ hg log --graph
@ changeset: 7:39522b764e3d
| tag: tip
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: does not commute with e
|
o changeset: 6:500cac37a696
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: f
|
o changeset: 5:7b4e2f4b7bcd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: e
|
o changeset: 4:00f1c5383965
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: d
|
o changeset: 3:65a9a84f33fd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: c
|
o changeset: 2:da6535b52e45
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: b
|
o changeset: 1:c1f09da44841
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: a
|
o changeset: 0:1715188a53c7
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: Initial commit
edit the history
$ hg histedit 3 --commands $EDITED 2>&1 | fixbundle
merging e
warning: conflicts while merging e! (edit, then use 'hg resolve --mark')
Fix up the change (pick 39522b764e3d)
(hg histedit --continue to resume)
fix up
$ echo 'I can haz no commute' > e
$ hg resolve --mark e
(no more unresolved files)
continue: hg histedit --continue
$ hg histedit --continue 2>&1 | fixbundle
merging e
warning: conflicts while merging e! (edit, then use 'hg resolve --mark')
Fix up the change (pick 7b4e2f4b7bcd)
(hg histedit --continue to resume)
We forcibly enable curses here so we can verify that continuing works
with curses enabled.
$ hg histedit --continue --config ui.interactive=true \
> --config ui.interface=curses 2>&1 | fixbundle
abort: unresolved merge conflicts (see 'hg help resolve')
This failure is caused by 7b4e2f4b7bcd "e" not rebasing the non commutative
former children.
just continue this time
$ hg revert -r 'p1()' e
make sure the to-be-empty commit doesn't screw up the state (issue5545)
$ hg histedit --continue 2>&1 | fixbundle
abort: unresolved merge conflicts (see 'hg help resolve')
$ hg resolve --mark e
(no more unresolved files)
continue: hg histedit --continue
$ hg histedit --continue 2>&1 | fixbundle
7b4e2f4b7bcd: skipping changeset (no changes)
log after edit
$ hg log --graph
@ changeset: 6:7efe1373e4bc
| tag: tip
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: f
|
o changeset: 5:e334d87a1e55
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: does not commute with e
|
o changeset: 4:00f1c5383965
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: d
|
o changeset: 3:65a9a84f33fd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: c
|
o changeset: 2:da6535b52e45
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: b
|
o changeset: 1:c1f09da44841
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: a
|
o changeset: 0:1715188a53c7
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: Initial commit
start over
$ cd ..
$ initrepo r2
$ cd r2
$ rm $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 3 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 4 >> $EDITED
$ hg log --template 'mess {node|short} {rev} {desc}\n' -r 7 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 5 >> $EDITED
$ hg log --template 'pick {node|short} {rev} {desc}\n' -r 6 >> $EDITED
$ cat $EDITED
pick 65a9a84f33fd 3 c
pick 00f1c5383965 4 d
mess 39522b764e3d 7 does not commute with e
pick 7b4e2f4b7bcd 5 e
pick 500cac37a696 6 f
edit the history, this time with a fold action
$ hg histedit 3 --commands $EDITED 2>&1 | fixbundle
merging e
warning: conflicts while merging e! (edit, then use 'hg resolve --mark')
Fix up the change (mess 39522b764e3d)
(hg histedit --continue to resume)
$ echo 'I can haz no commute' > e
$ hg resolve --mark e
(no more unresolved files)
continue: hg histedit --continue
$ hg histedit --continue 2>&1 | fixbundle
merging e
warning: conflicts while merging e! (edit, then use 'hg resolve --mark')
Fix up the change (pick 7b4e2f4b7bcd)
(hg histedit --continue to resume)
second edit also fails, but just continue
$ hg revert -r 'p1()' e
$ hg resolve --mark e
(no more unresolved files)
continue: hg histedit --continue
$ hg histedit --continue 2>&1 | fixbundle
7b4e2f4b7bcd: skipping changeset (no changes)
post message fix
$ hg log --graph
@ changeset: 6:7efe1373e4bc
| tag: tip
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: f
|
o changeset: 5:e334d87a1e55
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: does not commute with e
|
o changeset: 4:00f1c5383965
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: d
|
o changeset: 3:65a9a84f33fd
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: c
|
o changeset: 2:da6535b52e45
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: b
|
o changeset: 1:c1f09da44841
| user: test
| date: Thu Jan 01 00:00:00 1970 +0000
| summary: a
|
o changeset: 0:1715188a53c7
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: Initial commit
An invalid editor shouldn't leave the user in a broken state:
$ EDITOR=totally-not-a-thing-unsensible-editor-value hg histedit 4
$ cd ..