Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 18 Nov 2019 09:38:40 +0100] rev 44169
heptapod-ci: also run the dedicated rust test for the rust code
The Rust code has various standard rust test that are fast to run. So let's run them.
Differential Revision: https://phab.mercurial-scm.org/D8014
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 Nov 2019 12:25:53 +0100] rev 44168
heptapod-ci: run test with python3 too
Python3 is the future^W present, it is important to run tests with it too.
Differential Revision: https://phab.mercurial-scm.org/D8013
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Jan 2020 23:22:29 +0100] rev 44167
heptapod-ci: colorize output
The run result are nicer to read with color.
Differential Revision: https://phab.mercurial-scm.org/D8012
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Jan 2020 17:57:40 +0100] rev 44166
heptapod-ci: add a basic file to be able to run tests with heptapod
Having this yaml file somewhere in the main mercurial repository makes it
trivial for contributors using heptapod to run CI on their in-progress work.
There are alot of different combination (python2/python3 pure/cext/rust/pypy)
to be tested and making sure all of them are covered manually is cumbersome.
Automatic CI runnig on draft really helps in that matters. We start small bu
later changesets will add more step testing more of the variants.
The series is targetted on stable to make it available to the widest amount of contribution possible.
The definition of the docker files used for this are available here:
https://dev.heptapod.net/octobus/ci-dockerfiles
Differential Revision: https://phab.mercurial-scm.org/D8011
Jan Alexander Steffens (heftig) <jan.steffens@gmail.com> [Tue, 04 Feb 2020 22:07:36 +0100] rev 44165
worker: manually buffer reads from pickle stream
My previous fix (D8051, cb52e619c99e, which added Python's built-in buffering
to the pickle stream) has the problem that the selector will ignore the buffer.
When multiple pickled objects are read from the pipe into the buffer at once,
only one object will be loaded.
This can repeat until the buffer is full and delays the processing of completed
items until the worker exits, at which point the pipe is always considered
readable and all remaining items are processed.
This changeset reverts D8051, removing the buffer again. Instead, on Python 3
only, we use a wrapper to modify the "read" provided to the Unpickler to behave
more like a buffered read. We never read more bytes from the pipe than the
Unpickler requests, so the selector behaves as expected.
Also add a test case for "pickle data was truncated" issue.
https://phab.mercurial-scm.org/D8051#119193
Differential Revision: https://phab.mercurial-scm.org/D8076
Kyle Lippincott <spectral@google.com> [Thu, 02 Jan 2020 11:04:18 -0800] rev 44164
py3: __repr__ needs to return str, not bytes
Differential Revision: https://phab.mercurial-scm.org/D8089
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:07:37 +0100] rev 44163
config: also respect HGRCSKIPREPO in the zeroconf extension
Differential Revision: https://phab.mercurial-scm.org/D8075
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:07:42 +0100] rev 44162
config: also respect HGRCSKIPREPO in hgwebdir_mod
Differential Revision: https://phab.mercurial-scm.org/D8074
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 Feb 2020 20:41:11 +0100] rev 44161
config: also respect HGRCSKIPREPO in `dispatch._getlocal`
For some reason, we are also reading the local config in that function.
Differential Revision: https://phab.mercurial-scm.org/D8073
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:31:19 +0100] rev 44160
config: add a function in `rcutil` to abstract HGRCSKIPREPO
We wil need to respect this environment variable in more place.
Differential Revision: https://phab.mercurial-scm.org/D8072
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Feb 2020 20:12:47 -0500] rev 44159
packaging: make the path to Win32 requirements absolute when building WiX
Otherwise this broke automation when not launched from `contrib/packaging`.
Differential Revision: https://phab.mercurial-scm.org/D8068
Augie Fackler <raf@durin42.com> [Mon, 03 Feb 2020 11:07:34 -0500] rev 44158
Added signature for changeset 7f5410dfc8a6
Augie Fackler <raf@durin42.com> [Mon, 03 Feb 2020 11:07:33 -0500] rev 44157
Added tag 5.3 for changeset 7f5410dfc8a6
Raphaël Gomès <rgomes@octobus.net> [Wed, 29 Jan 2020 11:11:18 +0100] rev 44156
rust-dirstatemap: add missing @propertycache
While investigating a regression on `hg update` performance introduced by the
Rust `dirstatemap`, two missing `@propertycache` were identified when comparing
against the Python implementation. This adds back the first one, that has
no observable impact on behavior. The second one (`nonnormalset`) is going to
be more involved, as the caching has to be done from the Rust side of things.
Differential Revision: https://phab.mercurial-scm.org/D8047
Jan Alexander Steffens (heftig) <jan.steffens@gmail.com> [Thu, 30 Jan 2020 19:16:12 +0100] rev 44155
worker: Use buffered input from the pickle stream
On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError:
pickle data was truncated") when it gets a short read, i.e. it receives fewer
bytes than it requested.
On our build machine, Mercurial seems to frequently hit this problem while
updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy
to trigger with:
#wipe the workdir
rm -rf *
hg update null
chrt -b 0 hg update default
I've also written the following program, which demonstrates the core problem:
from __future__ import print_function
import io
import os
import pickle
import time
obj = {"a": 1, "b": 2}
obj_data = pickle.dumps(obj)
assert len(obj_data) > 10
rfd, wfd = os.pipe()
pid = os.fork()
if pid == 0:
os.close(rfd)
for _ in range(4):
time.sleep(0.5)
print("First write")
os.write(wfd, obj_data[:10])
time.sleep(0.5)
print("Second write")
os.write(wfd, obj_data[10:])
os._exit(0)
try:
os.close(wfd)
rfile = os.fdopen(rfd, "rb", 0)
print("Reading")
while True:
try:
obj_copy = pickle.load(rfile)
assert obj == obj_copy
except EOFError:
break
print("Success")
finally:
os.kill(pid, 15)
The program reliably fails with Python 3.8 and succeeds with Python 2.7.
Providing the unpickler with a buffered reader fixes the issue, so let
"os.fdopen" create one.
https://bugzilla.mozilla.org/show_bug.cgi?id=1604486
Differential Revision: https://phab.mercurial-scm.org/D8051
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 01:32:28 -0500] rev 44154
packaging: lowercase the `contrib` and `templates` directories with Inno
I have no idea why these (and `contrib/vim`) were leading with uppercase with
Inno, but not WiX. It probably doesn't matter too much, but might be a problem
with `templates` if the user enabled case sensitivity on NTFS.
Differential Revision: https://phab.mercurial-scm.org/D8063
Matt Harbison <matt_harbison@yahoo.com> [Sun, 02 Feb 2020 00:56:40 -0500] rev 44153
packaging: merge the requirements.txt files for WiX and Inno
Now that the content is common, there's no need to have separate files. The
content still differs from the non-Windows platforms though.
Differential Revision: https://phab.mercurial-scm.org/D8066
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:58:34 -0500] rev 44152
packaging: bundle dulwich, keyring, and pywin32-ctypes with WiX too
TortoiseHg installs these, which is possibly where they originated (though I
would have thought it more likely to be in the WiX installer, given its
heritage). When I was working on the TortoiseHg app for Mac (which uses the
similar `py2app`), it wasn't possible to use the keyring extension (even
externally) without bundling this keyring package into the app. Assuming the
same principle applies here, these would enable some common extensions. One of
the things that the TortoiseHg packager on macOS does now is it adds the user's
local `site-packages` directory to `sys.path`. That would allow the user to
install these critical modules in cases like this. But that can probably wait
for py3 packaging.
The only difference in the installed packages that I see now is WiX also bundles
distutils for some reason. I suppose that's not harming anything, so I'm not
touching it.
The only orphans in the install directories when comparing WiX and Inno now is
the Copying.txt vs COPYING.rtf, the two uninstaller files for Inno, and a
`Mercurial.url` file in Inno. I have no idea what that is, and it has *.ini
syntax with a single field pointing to the Mercurial homepage.
Differential Revision: https://phab.mercurial-scm.org/D8062
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:48:08 -0500] rev 44151
packaging: bundle the default mercurial.ini template with Inno also
This is a step towards converging on the same installer content on Windows.
Differential Revision: https://phab.mercurial-scm.org/D8061
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:41:37 -0500] rev 44150
packaging: set the FileVersion field in the Inno installer executable
Previously, Properties > Details > File version showed "0.0.0.0". This appears
to be a longstanding issue, and not part of the refactoring this cycle.
Differential Revision: https://phab.mercurial-scm.org/D8060
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:32:46 -0500] rev 44149
packaging: move the version normalization function to the util module
This will be used with Inno as well. Since this module isn't platform specific,
rename to include that this is meant for Windows. (Mac has a different format.)
Differential Revision: https://phab.mercurial-scm.org/D8059
Matt Harbison <matt_harbison@yahoo.com> [Fri, 31 Jan 2020 22:20:39 -0500] rev 44148
resourceutil: account for the non-resource-like file hierarchy under py2exe
After 9e367157a990, config files for py2exe were expected to be in
C:\Program Files\Mercurial\mercurial\defaultrc because of the implied resource
structure of 'mercurial.defaultrc.*.rc', relative to the executable.
Accomodating this would require changes to the WIX and Inno scripts (and perhaps
the script that generates the WIX script), as well as 3rd party bundlers like
TortoiseHg. But these files aren't read as resources anyway- they fall back to
the filesystem APIs. (If we really wanted to carry on the charade, the
installer would have to also sprinkle various empty __init__.py files around.)
Instead, this simply prunes the 'mercurial.' portion of the resource name when
run with py2exe. (PyOxidizer uses the resources API, not the filesystem
fallback, so it is unaffected.) Since this hack only affects the py2 Windows
installers and is less risky, I think it's reasonable. We haven't needed to
load any 3rd party resource up to this point, and would have to make packaging
changes anyway to handle that.
Differential Revision: https://phab.mercurial-scm.org/D8058
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Jan 2020 19:37:06 -0500] rev 44147
wix: restore COPYING.rtf
This got truncated to 0 bytes in 0ab651b5f77c when the Phabricator extension
crashed because it's a binary file. That caused the license page in the WIX
installer to be empty. I don't remember if I needed to resubmit after the bug
was fixed, so let's try this again with the current stable. If this fails, I'll
retry with 5.1 to see if this is a regression in the API changeover last cycle.
Differential Revision: https://phab.mercurial-scm.org/D8052
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Jan 2020 23:48:45 -0500] rev 44146
resourceutil: correct the root path for file based lookup under py2exe
This silly copy/paste error caused "Mercurial" to be truncated from
"C:\Program Files". The fact that "helptext" and "defaultrc" are now in a
subpackage of "mercurial" added it back on, and everything seemed to work. But
that broke if not installed to the default directory, and also caused TortoiseHg
to look at Mercurial's config files instead of its own.
Differential Revision: https://phab.mercurial-scm.org/D8054