Martin von Zweigbergk <martinvonz@google.com> [Tue, 21 Jan 2020 15:45:06 -0800] rev 44231
tests: move non-collapse test out of test-rebase-collapse
The test case was added in 76630fbbf4fa (test-rebase-collapse: Add
test for rebase regression introduced in 12309c09d19a, 2012-01-23). I
think `hg rebase --collapse` was involved in either the regression or
in the fix that caused the regression, but the test that was added
doesn't use `--collapse`, so it doesn't seem to belong in
test-rebase-collapse.t. The test case is about copies, so I moved it
to test-rebase-rename.t.
Differential Revision: https://phab.mercurial-scm.org/D7968
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 22 Nov 2019 20:27:09 -0800] rev 44230
debugcommands: add Python implementation to debuginstall
This seems like a useful detail to print.
Differential Revision: https://phab.mercurial-scm.org/D7979
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 22 Nov 2019 20:12:10 -0800] rev 44229
run-tests: remove --py3-warnings
This Python 2 only mode was to help Python 2 alert when doing
things not supported on Python 3. Now that we have test coverage with
Python 3, I don't think we need it.
Differential Revision: https://phab.mercurial-scm.org/D7978
Georges Racinet <georges.racinet@octobus.net> [Wed, 22 Jan 2020 16:37:05 +0100] rev 44228
rust-node: binary Node ID and conversion utilities
The choice of type makes sure that a `Node` has the exact
wanted size. We'll use a different type for prefixes.
Added dependency: hexadecimal conversion relies on the
`hex` crate.
The fact that sooner or later Mercurial is going to need
to change its hash sizes has been taken strongly in
consideration:
- the hash length is a constant, but that is not directly
exposed to callers. Changing the value of that constant
is the only thing to do to change the hash length (even
in unit tests)
- the code could be adapted to support several sizes of hashes,
if that turned out to be useful. To that effect, only the
size of a given `Node` is exposed in the public API.
- callers not involved in initial computation, I/O and FFI
are able to operate without a priori assumptions on the hash
size. The traits `FromHex` and `ToHex` have not been directly
implemented, so that the doc-comments explaining these
restrictions would stay really visible in `cargo doc`
Differential Revision: https://phab.mercurial-scm.org/D7788
Georges Racinet <georges.racinet@octobus.net> [Wed, 22 Jan 2020 16:23:29 +0100] rev 44227
rust-nodemap: building blocks for nodetree structures
This is similar to `nodetreenode` in `revlog.c`. We give it
a higher level feeling for ease of handling in Rust context
and provide tools for tests and debugging.
The encoding choice is dictated by our ultimate goal in this
series, that is to make an append-only persistent version of
`nodetree`: the 0th Block must be adressed from other Blocks.
Differential Revision: https://phab.mercurial-scm.org/D7787
Matt Harbison <matt_harbison@yahoo.com> [Tue, 21 Jan 2020 10:13:08 -0500] rev 44226
lfs: move the initialization of the upload request into the try block
This (almost) guarantees that the file is closed in the case of an exception.
The one hole is if the `seek(SEEK_END)`/`tell()`/`seek(0)` sequence fails. But
that's going to go away when subclassing `httpconnection.httpsendfile` to fix
the worker problem, so I'm not going to worry too much. (And that class appears
to have the same problem.)
Differential Revision: https://phab.mercurial-scm.org/D7959
Matt Harbison <matt_harbison@yahoo.com> [Tue, 21 Jan 2020 09:55:35 -0500] rev 44225
lfs: drop an unnecessary r'' prefix
No longer necessary since the source transformer was removed.
# skip-blame for changing string prefixes
Differential Revision: https://phab.mercurial-scm.org/D7958
Matt Harbison <matt_harbison@yahoo.com> [Tue, 21 Jan 2020 09:51:39 -0500] rev 44224
lfs: explicitly close the file handle for the blob being uploaded
The previous code relied on reading the blob fully to close it. The obvious
problem is if an error occurs before that point. But there is also a problem
when using workers where the data may need to be re-read, which can't happen
once it is closed. This eliminates the surprising behavior before attempting to
fix the worker problem.
Differential Revision: https://phab.mercurial-scm.org/D7957
Matt Harbison <matt_harbison@yahoo.com> [Tue, 21 Jan 2020 09:40:40 -0500] rev 44223
lfs: drop the unused progressbar code in the `filewithprogress` class
This has been unused since f98fac24b757, which added worker based transfers for
concurrency, shifting the progressbar maintenance to the single thread waiting
on the worker to complete. Since the name no longer fits, rename the class.
Differential Revision: https://phab.mercurial-scm.org/D7956
Raphaël Gomès <rgomes@octobus.net> [Tue, 14 Jan 2020 16:58:07 +0100] rev 44222
rust-filepatterns: remove bridge code for filepatterns-related functions
These functions will be used internally by `hg-core` without needed to be
exposed to Python.
Differential Revision: https://phab.mercurial-scm.org/D7868
Raphaël Gomès <rgomes@octobus.net> [Tue, 14 Jan 2020 18:03:28 +0100] rev 44221
rust-utils: add Rust implementation of Python's "os.path.splitdrive"
I also wrote the NT version although I didn't mean to at first, so I thought
I would keep it, so that any further effort to get the Rust code working on
Windows is a little easier.
Differential Revision: https://phab.mercurial-scm.org/D7864
Alexander Pyhalov <apyhalov@gmail.com> [Mon, 13 Apr 2020 16:30:13 +0300] rev 44220
setup: link osutil.so to libsocket on Solaris/illumos (issue6299)
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 06 Apr 2020 00:24:57 +0200] rev 44219
discovery: avoid wrongly saying there are nothing to pull
We can get in a situation where a revision passed through `hg pull --rev REV`
are available on the server, but not a descendant of the advertised server
heads.
For example the server could lying be during heads advertisement, to hide some
pull request. Or obsolete/hidden content could be explicitly pulled.
So in this case the lookup associated to `REV` returned successfully, but the
normal discovery will find all advertised heads already known locally. This flip
a special boolean `anyinc` that will prevent any fetch attempt, preventing `REV`
to be pulled over.
We add three line of code to detect this case and make sure a pull actually
happens.
My main target is to make some third party extensions happy (I expect the
associated test to move upstream with the extension). However this fix already
make some of the `infinitepush` test happier.
Augie Fackler <raf@durin42.com> [Thu, 02 Apr 2020 12:05:41 -0400] rev 44218
Added signature for changeset 8fca7e8449a8
Augie Fackler <raf@durin42.com> [Thu, 02 Apr 2020 12:05:40 -0400] rev 44217
Added tag 5.3.2 for changeset 8fca7e8449a8
Kyle Lippincott <spectral@google.com> [Wed, 01 Apr 2020 14:14:55 -0700] rev 44216
histedit: add missing b prefix to a string
If i18n is disabled (such as via HGPLAIN=1), `_()` doesn't convert from str to
bytes, so this raises a TypeError on py3.
Differential Revision: https://phab.mercurial-scm.org/D8354
Martin von Zweigbergk <martinvonz@google.com> [Wed, 25 Mar 2020 18:50:40 -0700] rev 44215
py3: require values in changelog extras to be bytes
I don't know what happened here because b436059c1cca (py3: use
pycompat.bytestr() on extra values because it can be int, 2019-02-05)
came about b44a47214122 (py3: use string for "close" value in commit
extras, 2018-02-11). Whatever happened, we shouldn't need to convert
the values to bytes now. It's better to not convert because that might
cover up bugs where someone sets a unicode value in the extras and
that works until the unicode value happens to contain non-ascii (at
which point it will fail because `bytestr()` expects its argument to
be ascii if it's unicode).
Differential Revision: https://phab.mercurial-scm.org/D8332
Martin von Zweigbergk <martinvonz@google.com> [Wed, 25 Mar 2020 18:25:58 -0700] rev 44214
py3: make setup.py's hgcommand() consistently return bytes
Before this patch, it returned unicode when the command failed. That
made e.g. `make local PYTHON=python3` fail on an obsolete commit.
Differential Revision: https://phab.mercurial-scm.org/D8331
Kyle Lippincott <spectral@google.com> [Mon, 23 Mar 2020 14:38:00 -0700] rev 44213
darwin: use vim, not vi, to avoid data-loss inducing posix behavior
Apple's version of vim, available at
opensource.apple.com/release/macos-1015.html (for Catalina, but this behavior
has been there for a while) has several tweaks from the version of vim from
vim.org. Most of these tweaks appear to be for "Unix2003" compatibility.
One of the tweaks is that if any ex command raises an error, the entire
process will (when you exit, possibly minutes/hours later) also exit non-zero.
Ex commands are things like `:foo`.
Luckily, they only enabled this if vim was executed (via a symlink or copying
the binary) as `vi` or `ex`. If you start it as `vim`, it doesn't have this
behavior, so let's do that.
To see this in action, run the following two commands on macOS:
```
$ vi -c ':unknown' -c ':qa' ; echo $?
1
$ vim -c ':unknown' -c ':qa' ; echo $?
0
```
We don't want to start ignoring non-zero return types from the editor because
that will mean you can't use `:cquit` to intentionally exit 1 (which,
shows up as 2 if you combine an ex command error and a cquit, but only a 1 if
you just use cquit, so we can't differentiate between the two statuses). Since
we can't differentiate, we have to assume that all non-zero exit codes are
intentional and an indication of the user's desire to not continue with whatever
we're doing. If this was a complicated `hg split` or `hg histedit`, this is
especially disastrous :(
Differential Revision: https://phab.mercurial-scm.org/D8321
Matt Harbison <matt_harbison@yahoo.com> [Fri, 20 Mar 2020 10:04:13 -0400] rev 44212
cext: move variable declaration to the top of the block for C89 support
Not sure if we still care about C89 in general, but MSVC requires this style
too.
Differential Revision: https://phab.mercurial-scm.org/D8304
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Mar 2020 21:27:45 +0100] rev 44211
byteify-string: resolve symlink before byteifying
Otherwise the script turns symlinks into regular files.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 12 Mar 2020 15:23:47 +0100] rev 44210
cext-index: propagate inline_scan error in `index_deref`
Before this change, revlog index corruption could be silently ignored in some
situation.
Differential Revision: https://phab.mercurial-scm.org/D8276
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 12 Mar 2020 18:53:37 +0100] rev 44209
heptapod-ci: fix test paths in the listing file
Now what we run the test from the root, we need to list test name from the root.
Differential Revision: https://phab.mercurial-scm.org/D8275
Raphaël Gomès <rgomes@octobus.net> [Fri, 06 Mar 2020 10:38:37 +0100] rev 44208
hg-core: add a compilation error if trying to compile outside of Linux
For now, we can only provide support for Linux in `hg-core`, so we have to be
explicit about it in case anyone wonders why their Dirstate is suddenly broken,
or why the crate does not compile (on Windows for example).
Differential Revision: https://phab.mercurial-scm.org/D8246
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Mar 2020 18:54:44 +0100] rev 44207
gzip: use the stdlib version with python 3 (issue6284)
It turned out that the stdlib gained the feature we missed in python 3.1. We can
now use it directly.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Mar 2020 18:53:19 +0100] rev 44206
gzip: indent the custom Gzip code
We need a new conditional in the next changesets (Adding 3.8 support). We do the
large code move beforehand for clarity.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Mar 2020 18:57:49 +0100] rev 44205
gzip: rename the argument to `mtime` to match upstream python
Python gained the feature we missed in 3.1. The argument name is `mtime`. We
align our version for consistency.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Mar 2020 16:49:38 +0100] rev 44204
heptapod-ci: run the test from outside the test directory
This will help detecting case where this is broken.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Mar 2020 16:47:02 +0100] rev 44203
run-tests: fix conditional when tests are run outside of `tests`
The logic to find `hghave` was using `__file__` but… non absolute path in
`__file__` gets wrong as soon as we change directory (thanks python… you are
being helpful). The rest of the `run-tests.py` code already deal with this
fine. We simply reuse the `RUNTESTDIR` variable to fix it.
Manuel Jacob <me@manueljacob.de> [Mon, 09 Mar 2020 01:11:59 +0100] rev 44202
tests: fix isinstance test of wrong variable
3086a8627b2970cd7fbf49bc69413c08c68d5514 changed self._case to be a list, but
it was forgotten to adjust this line.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 06 Mar 2020 23:27:28 +0100] rev 44201
discovery: avoid wrong detection of multiple branch heads (issue6256)
This fix the code using obsolescence markers to remove "to be obsoleted" heads
during the detection of new head creation from push. The code turned out to not
use the branch information at all. This lead changeset from different branch to
be detected as new head on unrelated branch.
The code fix is actually quite simple. New tests have been added to covers
these cases.
Differential Revision: https://phab.mercurial-scm.org/D8259
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 06 Mar 2020 00:28:09 +0100] rev 44200
test: cleanly skip test-remotefilelog-datapack.py on policy that breaks it
That tests requires the pure module to be available.
Differential Revision: https://phab.mercurial-scm.org/D8245
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Mar 2020 09:26:45 +0100] rev 44199
rust-format: cleanup ancestors.rs to make rustfmt happy
Differential Revision: https://phab.mercurial-scm.org/D8224
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Mar 2020 15:47:22 +0100] rev 44198
heptapod-ci: use strict module policy
Without this, test can silently fallback to other compatible policy in some
cases.
Differential Revision: https://phab.mercurial-scm.org/D8234
Martin von Zweigbergk <martinvonz@google.com> [Thu, 05 Mar 2020 08:30:11 -0800] rev 44197
histedit: fix formatting after D8150
Differential Revision: https://phab.mercurial-scm.org/D8230
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Mar 2020 17:17:02 +0100] rev 44196
rust-format: make rustfmt happy
Differential Revision: https://phab.mercurial-scm.org/D8228
Augie Fackler <raf@durin42.com> [Wed, 04 Mar 2020 11:51:13 -0500] rev 44195
Added signature for changeset 6d121acbb82e
Augie Fackler <raf@durin42.com> [Wed, 04 Mar 2020 11:51:12 -0500] rev 44194
Added tag 5.3.1 for changeset 6d121acbb82e
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 19 Feb 2020 13:39:00 +0530] rev 44193
remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269)
This cause traceback on widening using narrow extension when remotefilelog
is enabled.
Differential Revision: https://phab.mercurial-scm.org/D8134
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 28 Feb 2020 11:22:47 +0100] rev 44192
remotefilelog-test: glob some flaky output line
This is similar to ee0959e7d435. The affected line is flaky underload, yet the
final result is correct. The command involves background pre-check of output,
these are not stable probably because they run in parallel in multiple process.
If it become useful to start testing precise internal details of the, they will
have to be tested in a more appropriate framework than `.t` tests.
Differential Revision: https://phab.mercurial-scm.org/D8179
Steve Fink <sfink@mozilla.com> [Tue, 25 Feb 2020 13:23:37 -0800] rev 44191
histedit: py3 fixes for curses mode
Differential Revision: https://phab.mercurial-scm.org/D8150
Manuel Jacob <me@manueljacob.de> [Sun, 01 Mar 2020 19:39:23 +0100] rev 44190
branch: make --force work even when specifying revs
The `hg branch` command accepts a `--force` parameter that allows to
"set branch name even if it shadows an existing branch". However, before this
patch, that didn’t work when specifying revs with `-r`.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 24 Feb 2020 00:20:47 -0500] rev 44189
setup: exclude the __index__ module from itself when generating
This module is generated on Windows to hold all of the extension names and the
help summaries, so that they are discoverable inside the py2exe zipfile. The
problem is this file is generated by dumping the disabled list, and that list
comes from walking the filesystem. So once an install from source into a
virtualenv created this module, then next build from source from that virtualenv
would also see __index__.py in the filesystem, and include it. Clearly that's
wrong because this isn't a real extension, so just filter it from the list when
generating it.
The Mercurial installer was unaffected by this, but the TortoiseHg package was.
In the final package, `hg help -v extensions` and the panel of extensions both
showed it.
Differential Revision: https://phab.mercurial-scm.org/D8142
Matt Harbison <matt_harbison@yahoo.com> [Mon, 24 Feb 2020 16:33:10 -0500] rev 44188
tests: stabilize test-amend.t on Windows
If $TESTTMP isn't quoted in this context, it ends up like
`C:Temphgtests.pikkoxchild1test-amend.t-obsstore-off`.
Differential Revision: https://phab.mercurial-scm.org/D8144
Yuya Nishihara <yuya@tcha.org> [Mon, 24 Feb 2020 13:28:49 +0900] rev 44187
py3: fix EOL detection in commandserver.channeledinput
This breaks TortoiseHg's email preview which sends b'\n' while readline
request is issued and the loop never ends. Spotted by Matt Harbison.
Kyle Lippincott <spectral@google.com> [Thu, 20 Feb 2020 16:21:00 -0800] rev 44186
shelve: fix ordering of merge labels
Differential Revision: https://phab.mercurial-scm.org/D8140
Kyle Lippincott <spectral@google.com> [Thu, 20 Feb 2020 17:06:01 -0800] rev 44185
shelve: add test clearly demonstrating that the conflict labels are backwards
Differential Revision: https://phab.mercurial-scm.org/D8139
Matt Harbison <matt_harbison@yahoo.com> [Sun, 16 Feb 2020 17:05:18 -0500] rev 44184
import: don't ignore `--secret` when `--bypass` is specified
Differential Revision: https://phab.mercurial-scm.org/D8126
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Feb 2020 13:46:10 -0500] rev 44183
phabricator: fix a phabsend crash when processing a renamed binary
This was a trivial fix, and some more tests are added to cover binary files.
Since the old filecontext is passed in, the old name is still available. But I
noticed some weirdness around what it marked as binary and not, and what is
viewable in Phabricator. Those things have been flagged, and will probably take
some digging.
Differential Revision: https://phab.mercurial-scm.org/D8133
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 13 Dec 2019 10:37:45 +0100] rev 44182
test: pin the number of CPU for issue4074 tests
On machine with an hundreds of CPUs, the "user" CPU time reported can be
inflated by the status steps. Since the test especially focus on the diff
computation, we restrict the number of CPU to avoid potential issues.
Differential Revision: https://phab.mercurial-scm.org/D8112
Raphaël Gomès <rgomes@octobus.net> [Wed, 12 Feb 2020 23:23:59 +0100] rev 44181
rust-dirstatemap: add `NonNormalEntries` class
This fix introduces the same encapsulation as the `copymap`. There is no easy
way of doing this any better for now.
`hg up -r null && time HGRCPATH= HGMODULEPOLICY=rust+c hg up tip` on Mozilla
Central, (not super recent, but it doesn't matter):
Before: 7:44,08 total
After: 1:03,23 total
Pretty brutal regression!
This is a graft on stable of cf1f8660e568
Differential Revision: https://phab.mercurial-scm.org/D8111
Raphaël Gomès <rgomes@octobus.net> [Thu, 30 Jan 2020 14:57:02 +0100] rev 44180
rust-dirstatemap: cache non normal and other parent set
Performance of `hg update` was significantly worse since the introduction of
the Rust `dirstatemap`. This regression was noticed by Valentin Gatien-Baron
when working on a large repository, as it goes unnoticed for smaller
repositories like Mercurial itself.
This fix introduces the same getter/setter mechanism at `hg-core` level as
for `set/get_dirs`.
While this technique is, as previously discussed, quite suboptimal, it fixes an
important enough problem. Refactoring `hg-core` to use the typestate
pattern could be a good approach to improving code quality in a future patch.
This is a graft of stable of 83b2b829c94e
Differential Revision: https://phab.mercurial-scm.org/D8110
Yuya Nishihara <yuya@tcha.org> [Tue, 11 Feb 2020 19:53:56 +0900] rev 44179
chgserver: spawn new process if schemes change
The schemes extension updates hg.schemes table. It's technically possible
for hg.repository() to look for e.g. ui.schemes instead of depending on
module-local table, but I don't think the change would make much sense
since [schemes] is usually specified in ~/.hgrc and thus it can be considered
static data.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 10 Feb 2020 15:52:52 -0800] rev 44178
tests: accept new bzr message about switching branches
The new version apparently prints "Switched to branch at " instead of
"Switched to branch: ".
Differential Revision: https://phab.mercurial-scm.org/D8106
Martin von Zweigbergk <martinvonz@google.com> [Mon, 10 Feb 2020 15:50:26 -0800] rev 44177
tests: add workaround for bzr bug
This started failing for me today. I guess my bzr was upgraded.
Differential Revision: https://phab.mercurial-scm.org/D8105
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 01 Feb 2020 09:14:36 +0100] rev 44176
test: simplify test-amend.t to avoid race condition
Insted on relying on sleep, we could simply have the editor do the file change.
This remove the reliance on "sleep" and avoid test failing on heavy load
machine.
To test this, I reverted the code change in 5558e3437872 and the test started
failing again.
This is a graft on stable of 141ceec06b55 which should have targeted for stable.
Differential Revision: https://phab.mercurial-scm.org/D8103
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 09 Feb 2020 01:34:37 +0100] rev 44175
remotefilelog-test: glob some flaky output line (issue6083)
The two following lines are flaky underload, yet the final result is correct.
The command involves background pre-check of output, these are not stable
probably because they run in parallel in multiple process.
I spent a couple of hours trying to understand the pattern and gave up. The
documented intend of these tests is safely guaranteed by checking the cache
content after the command.
If it become useful to start testing precise internal details of the, they will
have to be tested in a more appropriate framework than `.t` tests.
Differential Revision: https://phab.mercurial-scm.org/D8102
Augie Fackler <augie@google.com> [Mon, 03 Feb 2020 11:56:02 -0500] rev 44174
resourceutil: blacken
Kyle Lippincott <spectral@google.com> [Thu, 06 Feb 2020 15:46:55 -0800] rev 44173
py3: fully fix bundlepart.__repr__ to return str not bytes
My previous fix did not fully fix the issue: it would attempt to use
%-formatting to combine two strs into a bytes, which won't work. Let's just
switch the entire function to operating in strs. This can cause a small output
difference that will likely not be noticed since no one noticed that the method
wasn't working at all before: if `id` or `type` are not-None, they'll be shown
as `b'val'` instead of `val`. Since this is a debugging aid and these strings
shouldn't be shown to the user, slightly rough output is most likely fine and
it's likely not worthwhile to add the necessary conditionals to marginally
improve it.
Differential Revision: https://phab.mercurial-scm.org/D8091
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 17 Nov 2019 01:18:14 +0100] rev 44172
heptapod-ci: add a job to test the rust version of Mercurial
The rust version of Mercurial is not currently tested by anything else. So it
get quite important that developer runs it.
Differential Revision: https://phab.mercurial-scm.org/D8017
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 Nov 2019 12:26:54 +0100] rev 44171
heptapod-ci: run the --pure test too
These are usually rarely run by individual developper because they are slow.
However it is important that they stay happy.
Differential Revision: https://phab.mercurial-scm.org/D8016
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Jan 2020 14:56:36 +0100] rev 44170
heptapod-ci: run the normal test suite
The usual tests should be run too. We skip the "tests-check*.t" one because
their are already covered by another Ci step.
Differential Revision: https://phab.mercurial-scm.org/D8015
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 18 Nov 2019 09:38:40 +0100] rev 44169
heptapod-ci: also run the dedicated rust test for the rust code
The Rust code has various standard rust test that are fast to run. So let's run them.
Differential Revision: https://phab.mercurial-scm.org/D8014
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 Nov 2019 12:25:53 +0100] rev 44168
heptapod-ci: run test with python3 too
Python3 is the future^W present, it is important to run tests with it too.
Differential Revision: https://phab.mercurial-scm.org/D8013
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 Jan 2020 23:22:29 +0100] rev 44167
heptapod-ci: colorize output
The run result are nicer to read with color.
Differential Revision: https://phab.mercurial-scm.org/D8012
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 25 Jan 2020 17:57:40 +0100] rev 44166
heptapod-ci: add a basic file to be able to run tests with heptapod
Having this yaml file somewhere in the main mercurial repository makes it
trivial for contributors using heptapod to run CI on their in-progress work.
There are alot of different combination (python2/python3 pure/cext/rust/pypy)
to be tested and making sure all of them are covered manually is cumbersome.
Automatic CI runnig on draft really helps in that matters. We start small bu
later changesets will add more step testing more of the variants.
The series is targetted on stable to make it available to the widest amount of contribution possible.
The definition of the docker files used for this are available here:
https://dev.heptapod.net/octobus/ci-dockerfiles
Differential Revision: https://phab.mercurial-scm.org/D8011
Jan Alexander Steffens (heftig) <jan.steffens@gmail.com> [Tue, 04 Feb 2020 22:07:36 +0100] rev 44165
worker: manually buffer reads from pickle stream
My previous fix (D8051, cb52e619c99e, which added Python's built-in buffering
to the pickle stream) has the problem that the selector will ignore the buffer.
When multiple pickled objects are read from the pipe into the buffer at once,
only one object will be loaded.
This can repeat until the buffer is full and delays the processing of completed
items until the worker exits, at which point the pipe is always considered
readable and all remaining items are processed.
This changeset reverts D8051, removing the buffer again. Instead, on Python 3
only, we use a wrapper to modify the "read" provided to the Unpickler to behave
more like a buffered read. We never read more bytes from the pipe than the
Unpickler requests, so the selector behaves as expected.
Also add a test case for "pickle data was truncated" issue.
https://phab.mercurial-scm.org/D8051#119193
Differential Revision: https://phab.mercurial-scm.org/D8076
Kyle Lippincott <spectral@google.com> [Thu, 02 Jan 2020 11:04:18 -0800] rev 44164
py3: __repr__ needs to return str, not bytes
Differential Revision: https://phab.mercurial-scm.org/D8089
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:07:37 +0100] rev 44163
config: also respect HGRCSKIPREPO in the zeroconf extension
Differential Revision: https://phab.mercurial-scm.org/D8075
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:07:42 +0100] rev 44162
config: also respect HGRCSKIPREPO in hgwebdir_mod
Differential Revision: https://phab.mercurial-scm.org/D8074
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 Feb 2020 20:41:11 +0100] rev 44161
config: also respect HGRCSKIPREPO in `dispatch._getlocal`
For some reason, we are also reading the local config in that function.
Differential Revision: https://phab.mercurial-scm.org/D8073
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 Feb 2020 12:31:19 +0100] rev 44160
config: add a function in `rcutil` to abstract HGRCSKIPREPO
We wil need to respect this environment variable in more place.
Differential Revision: https://phab.mercurial-scm.org/D8072
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Feb 2020 20:12:47 -0500] rev 44159
packaging: make the path to Win32 requirements absolute when building WiX
Otherwise this broke automation when not launched from `contrib/packaging`.
Differential Revision: https://phab.mercurial-scm.org/D8068
Augie Fackler <raf@durin42.com> [Mon, 03 Feb 2020 11:07:34 -0500] rev 44158
Added signature for changeset 7f5410dfc8a6
Augie Fackler <raf@durin42.com> [Mon, 03 Feb 2020 11:07:33 -0500] rev 44157
Added tag 5.3 for changeset 7f5410dfc8a6
Raphaël Gomès <rgomes@octobus.net> [Wed, 29 Jan 2020 11:11:18 +0100] rev 44156
rust-dirstatemap: add missing @propertycache
While investigating a regression on `hg update` performance introduced by the
Rust `dirstatemap`, two missing `@propertycache` were identified when comparing
against the Python implementation. This adds back the first one, that has
no observable impact on behavior. The second one (`nonnormalset`) is going to
be more involved, as the caching has to be done from the Rust side of things.
Differential Revision: https://phab.mercurial-scm.org/D8047
Jan Alexander Steffens (heftig) <jan.steffens@gmail.com> [Thu, 30 Jan 2020 19:16:12 +0100] rev 44155
worker: Use buffered input from the pickle stream
On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError:
pickle data was truncated") when it gets a short read, i.e. it receives fewer
bytes than it requested.
On our build machine, Mercurial seems to frequently hit this problem while
updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy
to trigger with:
#wipe the workdir
rm -rf *
hg update null
chrt -b 0 hg update default
I've also written the following program, which demonstrates the core problem:
from __future__ import print_function
import io
import os
import pickle
import time
obj = {"a": 1, "b": 2}
obj_data = pickle.dumps(obj)
assert len(obj_data) > 10
rfd, wfd = os.pipe()
pid = os.fork()
if pid == 0:
os.close(rfd)
for _ in range(4):
time.sleep(0.5)
print("First write")
os.write(wfd, obj_data[:10])
time.sleep(0.5)
print("Second write")
os.write(wfd, obj_data[10:])
os._exit(0)
try:
os.close(wfd)
rfile = os.fdopen(rfd, "rb", 0)
print("Reading")
while True:
try:
obj_copy = pickle.load(rfile)
assert obj == obj_copy
except EOFError:
break
print("Success")
finally:
os.kill(pid, 15)
The program reliably fails with Python 3.8 and succeeds with Python 2.7.
Providing the unpickler with a buffered reader fixes the issue, so let
"os.fdopen" create one.
https://bugzilla.mozilla.org/show_bug.cgi?id=1604486
Differential Revision: https://phab.mercurial-scm.org/D8051
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 01:32:28 -0500] rev 44154
packaging: lowercase the `contrib` and `templates` directories with Inno
I have no idea why these (and `contrib/vim`) were leading with uppercase with
Inno, but not WiX. It probably doesn't matter too much, but might be a problem
with `templates` if the user enabled case sensitivity on NTFS.
Differential Revision: https://phab.mercurial-scm.org/D8063
Matt Harbison <matt_harbison@yahoo.com> [Sun, 02 Feb 2020 00:56:40 -0500] rev 44153
packaging: merge the requirements.txt files for WiX and Inno
Now that the content is common, there's no need to have separate files. The
content still differs from the non-Windows platforms though.
Differential Revision: https://phab.mercurial-scm.org/D8066
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:58:34 -0500] rev 44152
packaging: bundle dulwich, keyring, and pywin32-ctypes with WiX too
TortoiseHg installs these, which is possibly where they originated (though I
would have thought it more likely to be in the WiX installer, given its
heritage). When I was working on the TortoiseHg app for Mac (which uses the
similar `py2app`), it wasn't possible to use the keyring extension (even
externally) without bundling this keyring package into the app. Assuming the
same principle applies here, these would enable some common extensions. One of
the things that the TortoiseHg packager on macOS does now is it adds the user's
local `site-packages` directory to `sys.path`. That would allow the user to
install these critical modules in cases like this. But that can probably wait
for py3 packaging.
The only difference in the installed packages that I see now is WiX also bundles
distutils for some reason. I suppose that's not harming anything, so I'm not
touching it.
The only orphans in the install directories when comparing WiX and Inno now is
the Copying.txt vs COPYING.rtf, the two uninstaller files for Inno, and a
`Mercurial.url` file in Inno. I have no idea what that is, and it has *.ini
syntax with a single field pointing to the Mercurial homepage.
Differential Revision: https://phab.mercurial-scm.org/D8062
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:48:08 -0500] rev 44151
packaging: bundle the default mercurial.ini template with Inno also
This is a step towards converging on the same installer content on Windows.
Differential Revision: https://phab.mercurial-scm.org/D8061
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:41:37 -0500] rev 44150
packaging: set the FileVersion field in the Inno installer executable
Previously, Properties > Details > File version showed "0.0.0.0". This appears
to be a longstanding issue, and not part of the refactoring this cycle.
Differential Revision: https://phab.mercurial-scm.org/D8060
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Feb 2020 00:32:46 -0500] rev 44149
packaging: move the version normalization function to the util module
This will be used with Inno as well. Since this module isn't platform specific,
rename to include that this is meant for Windows. (Mac has a different format.)
Differential Revision: https://phab.mercurial-scm.org/D8059
Matt Harbison <matt_harbison@yahoo.com> [Fri, 31 Jan 2020 22:20:39 -0500] rev 44148
resourceutil: account for the non-resource-like file hierarchy under py2exe
After 9e367157a990, config files for py2exe were expected to be in
C:\Program Files\Mercurial\mercurial\defaultrc because of the implied resource
structure of 'mercurial.defaultrc.*.rc', relative to the executable.
Accomodating this would require changes to the WIX and Inno scripts (and perhaps
the script that generates the WIX script), as well as 3rd party bundlers like
TortoiseHg. But these files aren't read as resources anyway- they fall back to
the filesystem APIs. (If we really wanted to carry on the charade, the
installer would have to also sprinkle various empty __init__.py files around.)
Instead, this simply prunes the 'mercurial.' portion of the resource name when
run with py2exe. (PyOxidizer uses the resources API, not the filesystem
fallback, so it is unaffected.) Since this hack only affects the py2 Windows
installers and is less risky, I think it's reasonable. We haven't needed to
load any 3rd party resource up to this point, and would have to make packaging
changes anyway to handle that.
Differential Revision: https://phab.mercurial-scm.org/D8058
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Jan 2020 19:37:06 -0500] rev 44147
wix: restore COPYING.rtf
This got truncated to 0 bytes in 0ab651b5f77c when the Phabricator extension
crashed because it's a binary file. That caused the license page in the WIX
installer to be empty. I don't remember if I needed to resubmit after the bug
was fixed, so let's try this again with the current stable. If this fails, I'll
retry with 5.1 to see if this is a regression in the API changeover last cycle.
Differential Revision: https://phab.mercurial-scm.org/D8052
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Jan 2020 23:48:45 -0500] rev 44146
resourceutil: correct the root path for file based lookup under py2exe
This silly copy/paste error caused "Mercurial" to be truncated from
"C:\Program Files". The fact that "helptext" and "defaultrc" are now in a
subpackage of "mercurial" added it back on, and everything seemed to work. But
that broke if not installed to the default directory, and also caused TortoiseHg
to look at Mercurial's config files instead of its own.
Differential Revision: https://phab.mercurial-scm.org/D8054
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 24 Jan 2020 20:27:59 -0800] rev 44145
wix: use original version string for MSI filename
Version string normalization is mostly to placate MSI requirements.
I think it makes sense to use the original version string in
filenames.
Since we can have distinct versions normalizing to the same MSI
version string, this will allow us to distinguish between different
actual version strings based on the filename.
Differential Revision: https://phab.mercurial-scm.org/D8005
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 24 Jan 2020 20:24:29 -0800] rev 44144
wix: always normalize version string
Before, it was possible to pass in a custom version string
which would not be valid in MSI. So we always normalize the
version string.
While we're here, also print when we normalize the version string,
for better visibility.
Differential Revision: https://phab.mercurial-scm.org/D8004
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 24 Jan 2020 20:21:53 -0800] rev 44143
wix: more robust normalization of RC version components
MSI has strict version requirements where the format is
`A.B.C[.D]` and all fields must be numeric
(https://docs.microsoft.com/en-us/windows/win32/msi/productversion?redirectedfrom=MSDN).
Only the first 3 are used by the installer itself.
Mercurial's version strings can have `rcN` and an optional
`+<commit>-<date>` fragment at the end.
This commit teaches the MSI version normalization to handle
both of these more robustly.
Before, we would throw away the `.rcN` component completely.
e.g. `5.3rc1` would get normalized to `5.3.0`. And worse,
`5.3rc0+5-abcdef` would get normalized to `5.3.5`.
After this commit, presence of an `.rcN` provides the
value for a 4th field. e.g. `5.3rc1` -> `5.3.0.1`. In
addition, the commit count from the `+` suffix gets
normalized into the 4th version component, but only if
the original version string didn't have a 4th version
component or if no `rcN` is present. e.g. `5.3+5-abcdef`
is `5.3.0.5`.
Differential Revision: https://phab.mercurial-scm.org/D8003
Matt Harbison <matt_harbison@yahoo.com> [Sat, 25 Jan 2020 00:16:04 -0500] rev 44142
copyright: update to 2020
Differential Revision: https://phab.mercurial-scm.org/D8006
Matt Harbison <matt_harbison@yahoo.com> [Sat, 25 Jan 2020 01:06:46 -0500] rev 44141
phabricator: fix a crash when submitting binaries (issue6260)
I think this assumed that `p1()` returned the changectx instead of the previous
filelog entry.
Differential Revision: https://phab.mercurial-scm.org/D8010
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 22 Jan 2020 20:01:38 -0800] rev 44140
packaging: add configparser to inno requirements file
This dependency is missing and pip complains about it in
strict hashing mode. How this was missed, I have no clue.
Differential Revision: https://phab.mercurial-scm.org/D7973
Augie Fackler <raf@durin42.com> [Fri, 24 Jan 2020 11:02:59 -0500] rev 44139
Added signature for changeset e4344e463c0c
Augie Fackler <raf@durin42.com> [Fri, 24 Jan 2020 11:02:58 -0500] rev 44138
Added tag 5.3rc1 for changeset e4344e463c0c
Matt Harbison <matt_harbison@yahoo.com> [Fri, 24 Jan 2020 01:37:18 -0500] rev 44137
packaging: rename hgrc.d to defaultrc for Windows config files next to the exe
The code and the help still says that it will read hgrc.d next to the
executable. But this directory needs to exist to read the resource based config
files. Otherwise even `hg version` errors out:
$ /c/Program\ Files/Mercurial/hg.exe version
Traceback (most recent call last):
File "hg", line 43, in <module>
File "mercurial\dispatch.pyc", line 110, in run
File "mercurial\dispatch.pyc", line 226, in dispatch
File "mercurial\ui.pyc", line 308, in load
File "mercurial\rcutil.pyc", line 99, in rccomponents
File "mercurial\rcutil.pyc", line 69, in default_rc_resources
File "mercurial\utils\resourceutil.pyc", line 84, in contents
WindowsError: [Error 3] The system cannot find the path specified: 'c:\\Program Files\\mercurial\\defaultrc\\*.*'
Differential Revision: https://phab.mercurial-scm.org/D7981
Matt Harbison <matt_harbison@yahoo.com> [Fri, 24 Jan 2020 01:11:19 -0500] rev 44136
resourceutil: ensure `_rootpath` is defined under py2exe
Can't even run `hg version` without this.
Differential Revision: https://phab.mercurial-scm.org/D7980
Kyle Lippincott <spectral@google.com> [Tue, 21 Jan 2020 17:15:34 -0800] rev 44135
crecord: fix a concatenation of bytes and str on py3
Differential Revision: https://phab.mercurial-scm.org/D7970
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 22 Jan 2020 14:11:11 -0500] rev 44134
recover: fix typos
Differential Revision: https://phab.mercurial-scm.org/D7971
Martin von Zweigbergk <martinvonz@google.com> [Tue, 21 Jan 2020 10:27:39 -0800] rev 44133
relnotes: copy "next" to "5.3" and clear "next"
This is the same thing as we've done for the last two releases.
Differential Revision: https://phab.mercurial-scm.org/D7955
Kyle Lippincott <spectral@google.com> [Tue, 21 Jan 2020 12:10:35 -0800] rev 44132
cext: change two more vars to Py_ssize_t in manifest.c
D7913 fixed a compiler warning with a signedness conflict in a ternary operator
by changing the types of some variables to be Py_ssize_t instead of size_t or
int. That commit missed these two cases since they aren't warned about (at least
on my compiler).
Both of these variables are produced by operations on variables that are
themselves Py_ssize_t now/already, so they should keep the same type.
Differential Revision: https://phab.mercurial-scm.org/D7964
Augie Fackler <raf@durin42.com> [Tue, 21 Jan 2020 13:16:30 -0500] rev 44131
Added signature for changeset 84a0102c05c7
Augie Fackler <raf@durin42.com> [Tue, 21 Jan 2020 13:16:29 -0500] rev 44130
Added tag 5.3rc0 for changeset 84a0102c05c7
Augie Fackler <augie@google.com> [Tue, 21 Jan 2020 13:14:51 -0500] rev 44129
merge to stable for 5.3 release freeze
Matt Harbison <matt_harbison@yahoo.com> [Fri, 17 Jan 2020 16:56:49 -0500] rev 44128
phabricator: use .arcconfig for `phabricator.url` if not set locally
This setting is also per repo; see the previous commit for details.
The existing `conduit_uri` setting is the previous name of `phabricator.uri`[1]
and while it could easily be queried before the latter for compatibility, the
config in this repo has '/api' appended. That's already done in `callconduit()`,
which would clearly end up giving the wrong result. It looks like the path of
the URL is now ignored in user configs[2], so add the modern setting without it
to this repo's .arcconfig.
Sadly, we still need to have contributors configure `auth.hg.phabtoken` (and
therefore `auth.hg.prefix` to link it to `phabricator.url`) in order to submit
patches, but at least now it's localized to a single section.
[1] https://secure.phabricator.com/book/phabricator/article/arcanist_new_project/
[2] https://github.com/phacility/arcanist/blob/cc850163f30c4697e925df0d6212469679600a2c/scripts/arcanist.php#L271
Differential Revision: https://phab.mercurial-scm.org/D7935
Matt Harbison <matt_harbison@yahoo.com> [Fri, 17 Jan 2020 14:21:40 -0500] rev 44127
phabricator: use .arcconfig for the callsign if not set locally (issue6243)
This makes things easier for people working with more than one repository
because this file can be committed to each repository. The bug report asks to
read <repo>/.arcrc, but AFAICT, that file lives in ~/ and holds the credentials.
And we already track an .arcconfig file. Any callsign set globally is still
used if that is all that is present, but .arcconfig will override it if
available. The idea behind letting the local hgrc override .arcconfig is that
the developer may need to do testing against another server, and not dirty the
working directory.
Originally I was going to just try to read the callsign in `getrepophid()` if it
wasn't present in the hg config. That works fine, but I think it also makes
sense to read the URL from this file too. That would have worked less well
because `readurltoken()` doesn't have access to the repo object to know where to
find the file. Supplimenting the config mechanism is less magical because it
reports the source and value of the properties used, and it doesn't need to read
the file twice.
Invalid hgrc files generally cause the program to abort. I only flagged it as a
warning here because it's not our config file, not crucial to the whole program
operating, and really shouldn't be corrupt in the typical case where it is
checked into the repo.
Differential Revision: https://phab.mercurial-scm.org/D7934
Matt Harbison <matt_harbison@yahoo.com> [Fri, 17 Jan 2020 13:29:47 -0500] rev 44126
config: add a function to insert non-file based, but overridable settings
This will be used in the next patch.
Until relatively recently (473510bf0575), there was no official way for
extensions to inject per-repo config data, so it probably makes sense that
`ui.setconfig()` items are sticky, and not affected by loading more config
files. But that makes it cumbersome if the extension wants to allow the data it
might add to be overridden by any data in the local hgrc file. The only thing I
could get to work was to load the local hgrc first, and then check if the source
for the config item that should be overridden was *not* the local hgrc file
name. But that's brittle because in addition to the file name, the source
contains the line number, there are the usual '\' vs '/' platform differences,
etc.
Differential Revision: https://phab.mercurial-scm.org/D7933
Matt Harbison <matt_harbison@yahoo.com> [Thu, 16 Jan 2020 19:48:01 -0500] rev 44125
tests: restore phabricator tests and regenerate the recordings
These contain the new API chatter. Most of the changes are because some new
commits were created, but they're pretty obviously equivalent. I have no idea
why the last recording contains real data, whereas it previously looked fake.
Differential Revision: https://phab.mercurial-scm.org/D7920
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 07 Jan 2020 11:24:05 +0100] rev 44124
hgrc: introduce HGRCSKIPREPO to skip reading the repository's hgrc
We had a way to change the behavior regarding reading the global and user
config, but we had nothing regarding the repository hgrc itself.
This option is useful in situation where scripts need to be able to work around
strange configuration set by the user in his repository. (and were HGPLAIN is
not enough).
Differential Revision: https://phab.mercurial-scm.org/D7807
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:37:14 -0800] rev 44123
debugcommands: move away from line buffered output on binary stream
Line buffering on binary file objects is apparently undefined behavior
in Python and emits a RuntimeWarning on Python 3.8. See
https://bugs.python.org/issue32236.
This commit changes the I/O logging file descriptor from line
buffered to unbuffered to work around this. I'm no fan of
unbuffered I/O for performance reasons. But I don't think it
is an issue here given the nature of the code.
With this change, test-ssh-proto.t now passes on Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D7948
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:43:52 -0800] rev 44122
py3: conditionalize test-lfs-serve-access.t for Python 3.8
This is another case where Python 3.8's traceback printing
varies subtly from older Python versions. Again, I'm not sure
why. But this is apparently the new behavior.
With this change, test-lfs-serve-access.t now passes on Python
3.8!
Differential Revision: https://phab.mercurial-scm.org/D7947
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:27:03 -0800] rev 44121
py3: add extra traceback line present on Python 3.8
I'm not sure why Python 3.8 is outputting this line. It
appears to be a change in behavior of formatting tracebacks on
Python 3.8. So let's add it to expected output.
With this change, test-hook.t now passes on Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D7946
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:12:41 -0800] rev 44120
py3: conditionalize test-flagprocessor.t on Python 3.8
For reasons I don't understand, Python 3.8 is outputting a different
lint in the traceback than prior Pythons.
The lines in question are:
flagutil.addflagprocessor(
REVIDX_NOOP, (noopdonothingread, noopdonothing, validatehash,)
)
Python <3.8 prints the 2nd line but 3.8 the first line. Perhaps Python
changed its traceback logic to always print the first line of a
multiple line expression?
Whatever the case, with this change, the test now passes on
Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D7945
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:21:45 -0800] rev 44119
tests: conditionalize test-hightlight.t on pygments version
Output changed slightly in pygments 2.5.
I thought it was easier to copy the line and add a version
check than to add a regular expression because the line has some
special characters. I also like tests explicitly calling out where
they vary.
Differential Revision: https://phab.mercurial-scm.org/D7943
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 20 Jan 2020 23:51:25 -0800] rev 44118
hgdemandimport: apply lazy module loading to sys.meta_path finders
Python's `sys.meta_path` finders are the primary objects whose job it
is to find a module at import time. When `import` is called, Python
iterates objects in this list and calls `o.find_spec(...)` to find
a `ModuleSpec` (or None if the module couldn't be found by that
finder). If no meta path finder can find a module, import fails.
One of the default meta path finders is `PathFinder`. Its job is to
import modules from the filesystem and is probably the most important
importer. This finder looks at `sys.path` and `sys.path_hooks` to do
its job.
The `ModuleSpec` returned by `MetaPathImporter.find_spec()` has a
`loader` attribute, which defines the concrete module loader to use.
`sys.path_hooks` is a hook point for teaching `PathFinder` to
instantiate custom loader types.
Previously, we injected a custom `sys.path_hook` that told `PathFinder`
to wrap the default loaders with a loader that creates a module object
that is lazy.
This approach worked. But its main limitation was that it only applied
to the `PathFinder` meta path importer. There are other meta path
importers that are registered. And in the case of PyOxidizer loading
modules from memory, `PathFinder` doesn't come into play since
PyOxidizer's own meta path importer was handling all imports.
This commit changes our approach to lazy module loading by proxying
all meta path importers. Specifically, we overload the `find_spec()`
method to swap in a wrapped loader on the `ModuleSpec` before it
is returned. The end result of this is all meta path importers should
be lazy.
As much as I would have loved to utilize .__class__ manipulation to
achieve this, some meta path importers are implemented in C/Rust
in such a way that they cannot be monkeypatched. This is why we
use __getattribute__ to define a proxy.
Also, this change could theoretically open us up to regressions in
meta path importers whose loader is creating module objects which
can't be monkeypatched. But I'm not aware of any of these in the
wild. So I think we'll be safe.
According to hyperfine, this change yields a decent startup time win of
5-6ms:
```
Benchmark #1: ~/.pyenv/versions/3.6.10/bin/python ./hg version
Time (mean ± σ): 86.8 ms ± 0.5 ms [User: 78.0 ms, System: 8.7 ms]
Range (min … max): 86.0 ms … 89.1 ms 50 runs
Time (mean ± σ): 81.1 ms ± 2.7 ms [User: 74.5 ms, System: 6.5 ms]
Range (min … max): 77.8 ms … 90.5 ms 50 runs
Benchmark #2: ~/.pyenv/versions/3.7.6/bin/python ./hg version
Time (mean ± σ): 78.9 ms ± 0.6 ms [User: 70.2 ms, System: 8.7 ms]
Range (min … max): 78.1 ms … 81.2 ms 50 runs
Time (mean ± σ): 73.4 ms ± 0.6 ms [User: 65.3 ms, System: 8.0 ms]
Range (min … max): 72.4 ms … 75.7 ms 50 runs
Benchmark #3: ~/.pyenv/versions/3.8.1/bin/python ./hg version
Time (mean ± σ): 78.1 ms ± 0.6 ms [User: 70.2 ms, System: 7.9 ms]
Range (min … max): 77.4 ms … 80.9 ms 50 runs
Time (mean ± σ): 72.1 ms ± 0.4 ms [User: 64.4 ms, System: 7.6 ms]
Range (min … max): 71.4 ms … 74.1 ms 50 runs
```
Differential Revision: https://phab.mercurial-scm.org/D7954
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 20 Jan 2020 23:42:19 -0800] rev 44117
hgdemandimport: disable on Python 3.5
The demand importer functionality isn't working at all on Python 3.5.
I'm not sure what's wrong.
Since it isn't working, let's disable it completely.
```
$ HGRCPATH= hyperfine -w 1 -r 50 -- "~/.pyenv/versions/3.5.9/bin/python ./hg version" \
"HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version"
Benchmark #1: ~/.pyenv/versions/3.5.9/bin/python ./hg version
Time (mean ± σ): 163.7 ms ± 2.2 ms [User: 148.5 ms, System: 15.7 ms]
Range (min … max): 161.0 ms … 170.2 ms 50 runs
Benchmark #2: HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version
Time (mean ± σ): 164.3 ms ± 1.4 ms [User: 148.2 ms, System: 16.6 ms]
Range (min … max): 161.4 ms … 169.8 ms 50 runs
```
Differential Revision: https://phab.mercurial-scm.org/D7953
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 11:13:01 -0800] rev 44116
py3: suppress unraisable exceptions in test-worker.t
Python 3.8 calls sys.unraisablehook when an unraisable
exception is encountered. The default behavior is to print a
warning.
test-worker.t was triggering this hook due to a race between
a newly forked process exiting and that process's
_os.register_at_fork handlers running. I was seeing the
stdlib's random module in the stack re-seeding itself. Although
there could be other after-fork handlers in the mix.
This commit defines sys.unraisablehook to effectively no-op.
This suppresses the warning and makes test output on Python 3.8
consistent with prior versions. test-worker.t now passes on
Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D7949
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Mon, 20 Jan 2020 18:28:46 -0500] rev 44115
rust: add a README
In particular to explain how to build any of the rust. It's neither
obvious, nor easy to find out, nor easy to determine if you did it
right without some documentation.
Differential Revision: https://phab.mercurial-scm.org/D7952
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Mon, 20 Jan 2020 17:44:03 -0500] rev 44114
rust: move hgcli's README out of the way
My understanding is that it's not meant to be used in the current
form.
Differential Revision: https://phab.mercurial-scm.org/D7951
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Jan 2020 01:54:17 -0500] rev 44113
verify: avoid spurious integrity warnings in verbose mode (issue6172)
The issue seems to revolve around renames in filtered commits, and only occurred
in verbose mode. The problem occurs in the `# check renames` stage, around line
577. Without using the unfiltered repo, this test would have printed:
$ hg verify -v
repository uses revlog format 1
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
foo@25: checking rename of 71ec0570c325: filtered revision '25'
foobar@26: checking rename of 1b549296015b: filtered revision '26'
checked 28 changesets with 16 changes to 11 files
2 integrity errors encountered!
(first damaged changeset appears to be 25)
[1]
Differential Revision: https://phab.mercurial-scm.org/D7950
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Jan 2020 22:31:47 -0800] rev 44112
py3: glob over exception in test-check-py3-compat.t
Python 3.6+ raise ModuleNotFoundError and older versions raise
ImportError. Glob over the exception differences.
For whatever reason, we were already doing this for one failure.
But not all occurrences of ModuleNotFoundError were changed.
Who knows.
This test should now pass on all Python versions (although I didn't
check Windows).
Differential Revision: https://phab.mercurial-scm.org/D7939