Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 20 Aug 2021 11:23:52 +0200] rev 47889
dirstate: forward `remove` call to newer `API`
The `_remove` method was only called in the deprecated `remove` function. We
merge the two and express it in terms of call to new API methods.
Differential Revision: https://phab.mercurial-scm.org/D11314
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 30 Aug 2021 23:40:43 +0530] rev 47888
Added signature for changeset
86a60679cf61
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 30 Aug 2021 23:40:37 +0530] rev 47887
Added tag 5.9.1 for changeset
86a60679cf61
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Aug 2021 13:51:44 -0700] rev 47886
fix: again allow formatting the working copy while merging
I forgot about unfinished merges (I think I was thinking only about
unfinished merge conflicts) when I wrote
https://phab.mercurial-scm.org/D11209. As a coworker (hg contributor
dploch) reported to me, this led to `hg fix --working-dir` failing
when you have an uncommitted merge. The fix is trivial: just move the
assertion to just before the call to `scmutil.movedirstate()` where it
actually matters.
Differential Revision: https://phab.mercurial-scm.org/D11376
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 26 Aug 2021 21:40:21 -0700] rev 47885
pyoxidizer: add arch to PyOxidizer MSIs
This is the proper filename pattern. This filename format
wasn't properly implemented when PyOxidizer took over Python 3
MSI generation in
603efb3845ba.
Differential Revision: https://phab.mercurial-scm.org/D11359
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 26 Aug 2021 20:05:12 -0700] rev 47884
pyoxidizer: pass arch to WiXInstaller()
We just upgraded Windows automation to PyOxidizer 0.17.0. This
version of PyOxidizer changed the behavior of WiX installers so
installer architecture defaulted to "x64" (before it defaulted
to the target of the pyoxidizer.exe binary). To allow controlling
the architecture of the installer, the `arch` argument was added
to `WiXInstaller`. This commit passes that argument in.
Differential Revision: https://phab.mercurial-scm.org/D11358
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 26 Aug 2021 20:01:01 -0700] rev 47883
contrib: install PyOxidizer 0.17.0
This pulls in some changes we want to improve Windows MSI installers.
This will need separate enhancements to the pyoxidizer.bzl. But those will
be handled in a separate changeset.
Differential Revision: https://phab.mercurial-scm.org/D11357
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 26 Aug 2021 17:39:11 -0700] rev 47882
packaging: reference proper output directory
9438e9b7321a changed the name of the PyOxidizer target, which changed
the name of the output directory. The code changed by this patch
wasn't properly updated by that changeset. This resulted in a run-time
failure due to trying to read from a non-existent directory.
This change should fix the building of Python 3 Inno installers.
Differential Revision: https://phab.mercurial-scm.org/D11356
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 25 Aug 2021 19:33:25 -0700] rev 47881
packaging: pass extra_pyoxidizer_vars only to pyoxidizer
Before, we would attempt to call a function (build_installer_py2exe)
that didn't accept this keyword argument. This was preventing the
building of py2exe installers.
Differential Revision: https://phab.mercurial-scm.org/D11355
Matt Harbison <matt_harbison@yahoo.com> [Thu, 26 Aug 2021 11:04:14 -0400] rev 47880
hg: don't attempt to extend `sys.path` with the user site without `APPDATA`
This variable is created by the system and *should* be available, but
test-lfs-bundle.t has a test where it is explicitly unset. It wasn't caught
before because prior to
95af358fcdfe, it was limited to the py2exe binary. As a
precaution, fix both that and the pyoxidizer case that was causing the test to
fail.
Differential Revision: https://phab.mercurial-scm.org/D11354
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 26 Aug 2021 09:49:09 +0200] rev 47879
ci: only run the phabricator step if the previous on succeeded
It seems like
f6879956a386 regressed the intended behavior.
Differential Revision: https://phab.mercurial-scm.org/D11342
Raphaël Gomès <rgomes@octobus.net> [Wed, 25 Aug 2021 15:15:19 +0200] rev 47878
branching: merge stable into default
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Mon, 02 Aug 2021 08:05:13 -0400] rev 47877
store: return just one filename in walk functions
Various walk functions return `(revlog_type, decoded, encoded)` where
decoded could be None. But no-one cares about `encoded` and expects
`unencoded` to be present, except verify (because this can only happen
with old repo formats).
Simplify all this by either failing outright if a decoding a filename
fails (instead of almost certainly failing with a type error due to
treating None as a bytes), or skipping the filename but providing in
an out argument for hg verify.
Differential Revision: https://phab.mercurial-scm.org/D11248
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 01 Aug 2021 10:57:21 -0400] rev 47876
tests: rename test-clone-uncompressed.t
as clone --uncompressed is deprecated in favor of --stream
Differential Revision: https://phab.mercurial-scm.org/D11237
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Fri, 06 Aug 2021 16:27:17 -0400] rev 47875
debugrebuildfncache: add a cheaper option to rebuild the fncache
On my repository, debugrebuildfncache takes 5-10min with the lock.
With the flag added in this commit, it takes 10s. The tradeoff is that
it only recovers from certain kinds of corruptions. It is intended to
to recover faster from fncaches broken by a revlog split during a
transaction that ends up being rolled back.
Differential Revision: https://phab.mercurial-scm.org/D11265
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Fri, 06 Aug 2021 16:17:17 -0400] rev 47874
test: reduce noise, so the important bits stand out
Differential Revision: https://phab.mercurial-scm.org/D11264
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Tue, 24 Aug 2021 17:27:16 +0200] rev 47873
wireprotov1peer: update all rpcs to use the new batchable scheme
If desired, we could keep the future class and the function that
upgrades an old style rpc instead of a new style, for extensions.
Differential Revision: https://phab.mercurial-scm.org/D11212
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Tue, 24 Aug 2021 17:27:16 +0200] rev 47872
wireprotov1peer: simplify the way batchable rpcs are defined
The scheme with futures/generator is confusing due to the way
communication is done by side effects, especially with two different
"future" objects. Just returning a request and a function to read the
response is easier to understand.
There are tests failures with the largefiles extension due to it
aliasing one rpc to another one, which gets fixed in the next commit.
Differential Revision: https://phab.mercurial-scm.org/D11211
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Aug 2021 11:32:49 +0200] rev 47871
clone: properly create target directories during local clone (
issue6581)
The store encoding was mudding the water. This lead to local clone crashing for
file with long filename as their destination directory needed to be encoded.
Differential Revision: https://phab.mercurial-scm.org/D11340
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Aug 2021 10:00:55 +0200] rev 47870
clone: add a file demonstrating
issue6581 in test-clone-stream.t
This is quite noisy so we adds it in its own changeset. Fixes for the issue are
coming in the next patch.
Differential Revision: https://phab.mercurial-scm.org/D11339
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Aug 2021 10:42:09 +0200] rev 47869
clone: verify the local clone in test-clone-stream.t
This will help detecting corruption.
Differential Revision: https://phab.mercurial-scm.org/D11338
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Aug 2021 10:08:37 +0200] rev 47868
clone: automatically glob stream clone output in test
Touching `test-clone-stream.t` is very painful otherwise.
Differential Revision: https://phab.mercurial-scm.org/D11337
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Aug 2021 09:32:21 +0200] rev 47867
clone: fix a comment in test-clone-stream.t
Differential Revision: https://phab.mercurial-scm.org/D11336
Matt Harbison <matt_harbison@yahoo.com> [Sun, 22 Aug 2021 17:59:21 -0400] rev 47866
windows: degrade to py2 behavior when reading a non-symlink as a symlink
While waiting for the push to hg-committed in WSL to complete, I ran a
`phabimport` from Windows and got this traceback:
$ hg phabimport 11313
** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" (version N/A)
** which supports versions unknown of Mercurial.
** Please disable "mercurial_keyring" and try your action again.
** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues
** Python 3.9.5 (default, May 6 2021, 17:29:31) [MSC v.1928 64 bit (AMD64)]
** Mercurial Distributed SCM (version 5.9rc1+hg32.
0e2f5733563d)
** Extensions loaded: absorb, blackbox, evolve 10.3.3, extdiff, fastannotate, fix, mercurial_keyring, mq, phabblocker
20210126, phabricator, rebase, show, strip, topic 0.22.3
Traceback (most recent call last):
File "mercurial.lock", line 279, in _trylock
File "mercurial.vfs", line 202, in makelock
File "mercurial.util", line 2147, in makelock
FileExistsError: [WinError 183] Cannot create a file when that file already exists: b'hp-omen:78348' -> b'C:\\Users\\Matt\\hg/.hg/store/lock'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 24, in <module>
File "mercurial.dispatch", line 144, in run
File "mercurial.dispatch", line 250, in dispatch
File "mercurial.dispatch", line 294, in _rundispatch
File "mercurial.dispatch", line 470, in _runcatch
File "mercurial.dispatch", line 480, in _callcatch
File "mercurial.scmutil", line 153, in callcatch
File "mercurial.dispatch", line 460, in _runcatchfunc
File "mercurial.dispatch", line 1273, in _dispatch
File "mercurial.dispatch", line 918, in runcommand
File "mercurial.dispatch", line 1285, in _runcommand
File "mercurial.dispatch", line 1271, in <lambda>
File "mercurial.util", line 1886, in check
File "mercurial.util", line 1886, in check
File "hgext.mq", line 4239, in mqcommand
File "mercurial.util", line 1886, in check
File "mercurial.util", line 1886, in check
File "hgext.phabricator", line 314, in inner
File "hgext.phabricator", line 2222, in phabimport
File "hgext.phabricator", line 2123, in readpatch
File "hgext.phabricator", line 2199, in _write
File "mercurial.localrepo", line 2956, in lock
File "mercurial.localrepo", line 2918, in _lock
File "mercurial.lock", line 152, in trylock
File "mercurial.lock", line 283, in _trylock
File "mercurial.lock", line 314, in _readlock
File "mercurial.vfs", line 221, in readlock
File "mercurial.util", line 2163, in readlock
File "mercurial.windows", line 619, in readlink
ValueError: not a symbolic link
Both exceptions look accurate (the file exists, and the Windows side can't read
WSL side symlinks). I didn't try to reproduce this entirely within the Windows
side, but we can do better than a cryptic stacktrace. With this change, the
same scenario results in this abort:
abort: C:\Users\Matt\hg/.hg/store/lock: The file cannot be accessed by the system
When both the `push` and `phabimport` are done on the Windows side, it prints a
message about waiting for the lock, and successfully applies the patch after the
push completes.
I'm not sure if there's enough info to be able to convert the abort into the
wait scenario. As it stands now, we don't support symlinks on Windows, which
requires either a UAC Administrator level process or an opt-in in developer
mode, and there are several places where the new symlink on Windows support in
py3 was explicitly disabled in order to get tests to pass quicker.
Differential Revision: https://phab.mercurial-scm.org/D11333
Matt Harbison <matt_harbison@yahoo.com> [Wed, 18 Aug 2021 21:59:55 -0400] rev 47865
pyoxidizer: add user-site to `sys.path` on Windows
This is a port of
53221078e0de to Windows to allow pip-installed extensions to
be loaded without specifying a path. It's a major headache to have an hg.exe on
`PATH` that needs to have the path to the extensions specified, because WSL
doesn't see the same path.
This is only for Windows for now, to match the currently shipping py2 behavior.
There is a better solution with using the `site` package, but this needs support
in PyOxidizer[1].
[1] https://github.com/indygreg/PyOxidizer/issues/430
Differential Revision: https://phab.mercurial-scm.org/D11308
Matt Harbison <matt_harbison@yahoo.com> [Wed, 18 Aug 2021 14:58:42 -0400] rev 47864
zeroconf: fix an issue concatenating bytes and str
`bytes(length)` doesn't do what we want, so use `str`.
There appear to be a ton more issues in this extension, including:
- globals()[b'_GLOBAL_DONE'] using bytes as the key
- `__author__` and similar using bytes
- `BadDomainName` is feeding bytes to the Exception constructor
- DNSRecord.toString() has the wrong signature (should be str, not bytes)
Differential Revision: https://phab.mercurial-scm.org/D11303
Julien Cristau <jcristau@debian.org> [Tue, 24 Aug 2021 23:42:35 +0200] rev 47863
revlog: fix more type confusion in index_replace_sidedata_info (
issue6580)
We were telling python that "rev" was a Py_ssize_t (via the "n" format),
but it was actually an int.
Differential Revision: https://phab.mercurial-scm.org/D11335
Julien Cristau <jcristau@mozilla.com> [Tue, 24 Aug 2021 12:44:34 +0200] rev 47862
revlog: fix type confusion with sidedata_comp_len (
issue6580)
The format string uses "i" (int) for sidedata_comp_len, so we shouldn't
be passing a pointer to Py_ssize_t to PyArg_ParseTuple. On 64-bit
big-endian, this would result in python only writing to the upper 32
bits, and things go downhill from there.
Differential Revision: https://phab.mercurial-scm.org/D11334
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 20 Aug 2021 11:23:41 +0200] rev 47861
dirstate: directly call the dirstatemap in `set_untracked`
This function is only called in two places: the deprecated "remove" method and
in the new `set_untracked` method.
So we simply inline the appropriate content in `set_untracked`, paving the way
to dropping the deprecated code and its associated function.
Differential Revision: https://phab.mercurial-scm.org/D11313
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Aug 2021 17:42:55 +0200] rev 47860
resources: narrow the try:except clause to minimum
Otherwise this mind hides other import or attribute errors.
Differential Revision: https://phab.mercurial-scm.org/D11312
Matt Harbison <matt_harbison@yahoo.com> [Sun, 22 Aug 2021 16:32:06 -0400] rev 47859
merge with stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Aug 2021 17:47:27 +0200] rev 47858
resources: stop important a non existent FileNotFoundError
The error was hidden by the wide try/except
Differential Revision: https://phab.mercurial-scm.org/D11311
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Aug 2021 17:46:46 +0200] rev 47857
template: FileNotFoundError is actually a built in exception
However it is python3 only.
Differential Revision: https://phab.mercurial-scm.org/D11310
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Aug 2021 18:04:14 +0200] rev 47856
check-code: drop the camelcase checks
Fro about 2 years we have been using CamelCase is class names. That rules gets
in the way of assigning class or exception in compatibility layers.
I think it is safe to drop it now that we started using CamelCase for some case.
My motivation for this is the need to assign `FileNotFoundError` in `pycompat`.
Differential Revision: https://phab.mercurial-scm.org/D11309
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 20 Aug 2021 00:23:49 +0530] rev 47855
Added signature for changeset
53221078e0de
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 20 Aug 2021 00:23:40 +0530] rev 47854
Added tag 5.9 for changeset
53221078e0de
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 16 Aug 2021 18:39:58 +0200] rev 47853
pyoxidizer: always run the associated CI jobs
Now that all tests are passing, we can enable this by default.
Differential Revision: https://phab.mercurial-scm.org/D11306
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 21:28:36 +0200] rev 47852
pyoxidizer: skip for ZeroConf related test for now
These test are failing for obscure reason. They are fairly minor and Since they
are the only thing between us and enabling pyoxidizer testing in the CI by
default, we just shallow the bad output for now.
Differential Revision: https://phab.mercurial-scm.org/D11305
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 21:05:46 +0200] rev 47851
pyoxidizer: disable the test for disabled & broken extensions help
The test is a bit fragile and the feature is significantly broken for 3rd party
extension already. So we disable that section with a comment about why and what
is needed.
Differential Revision: https://phab.mercurial-scm.org/D11304
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 21:37:55 +0200] rev 47850
pyoxidizer: re-install PYTHONPATH behavior
Hooks and extensions can rely on PYTHONPATH value. Yet the pyoxidized binary
ignore the variable. So we manually reinstall the behavior to fix various tests
and restore the expected behavior.
This positively affects:
- test-extension.t
- test-hook.t
Differential Revision: https://phab.mercurial-scm.org/D11302
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 16:48:32 +0200] rev 47849
windows: add pytest-vcr to the dependencies
This will lead to pyoxidizer including it in its binary.
This fix test-phabricator.t
Differential Revision: https://phab.mercurial-scm.org/D11301
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 16:02:46 +0200] rev 47848
pyoxidizer: wrap the pyoxidizer script on multiple line
This is clearer and will help with future edition of the script.
Differential Revision: https://phab.mercurial-scm.org/D11300
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 15:05:25 +0200] rev 47847
pyoxidized: silence the fuzzywuzzy warning about python-Levenshtein
I don't think we need the fast implementation for the test so lets ignore it for
now.
Differential Revision: https://phab.mercurial-scm.org/D11299
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 13:16:04 +0200] rev 47846
pyoxidized: install fuzzywuzzy too
This is necessary for some of the release note tooling.
Differential Revision: https://phab.mercurial-scm.org/D11298
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 14:57:34 +0200] rev 47845
pyoxidized: adapt output of test-install.t
The python lib and template directories are in different location that requires
a different patterns.
Differential Revision: https://phab.mercurial-scm.org/D11297
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 13:02:08 +0200] rev 47844
pyoxidized: disable part of `test-install.t` related to pip and virtualenv
This part are testing manual installation of Mercurial, this is not relevant
for the pyoxidizer variant.
Differential Revision: https://phab.mercurial-scm.org/D11296
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 12:53:40 +0200] rev 47843
pyoxidized: adapt output of test-bad-extension.t
The file path in the traceback become module name.
Differential Revision: https://phab.mercurial-scm.org/D11295
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 12:49:55 +0200] rev 47842
pyoxidized: adapt output of test-flagprocessor.t
The file path in the traceback become module name.
Differential Revision: https://phab.mercurial-scm.org/D11294
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 12:42:16 +0200] rev 47841
pyoxidized: adapt output of `test-phases.t`
We are getting one less line. I am not sure why, but it seems fairly minor.
Differential Revision: https://phab.mercurial-scm.org/D11293
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:27:45 +0200] rev 47840
pyoxidized: add a copy of the `doc` directory in the right location
This get the hacky setup closer to how an actual install is setup.
Differential Revision: https://phab.mercurial-scm.org/D11292
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:26:25 +0200] rev 47839
pyoxidized: add a copy of the `contrib` directory in the right location
This get the hacky setup closer to how an actual install is setup.
Differential Revision: https://phab.mercurial-scm.org/D11291
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:18:34 +0200] rev 47838
pyoxidized: add a copy of the `defaultrc` file in the right location
This get the hacky setup closer to how an actual install is setup.
Differential Revision: https://phab.mercurial-scm.org/D11290
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 02:08:03 +0200] rev 47837
pyoxidized: add a copy of the `helptext` directory in the right location
This get the hacky setup closer to how an actual install is setup.
Differential Revision: https://phab.mercurial-scm.org/D11289
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:41:02 +0200] rev 47836
template: handle missing resource in `_readmapfile`
This fix the remaining issue in test-template-map.t.
Differential Revision: https://phab.mercurial-scm.org/D11288
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:10:40 +0200] rev 47835
pyoxidized: add a copy of the `templates` directory in the right location
This fix about 10 extra test bring the number of currently failing test to about 12.
Differential Revision: https://phab.mercurial-scm.org/D11287
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 18 Aug 2021 01:07:38 +0200] rev 47834
pyoxidized: add a dedicated target in the tests
We will need to setup more items to make the "quick" version work on stable,
having a dedicated operation will allow to contains change to that operation.
Differential Revision: https://phab.mercurial-scm.org/D11286
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Aug 2021 20:48:52 +0200] rev 47833
pyoxidized: adapt test-devel-warnings.t
Differential Revision: https://phab.mercurial-scm.org/D11284
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Aug 2021 21:57:59 +0200] rev 47832
test-extension: adapt output to pyoxidizer
The traceback use module name instead of filename.
The rest of the test is still on fire for various other reasons. However that
specific output mismatch is now fixed.
Differential Revision: https://phab.mercurial-scm.org/D11285
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Aug 2021 20:48:34 +0200] rev 47831
pyoxidized: add a `pyoxidizer` hghave keyword for line matching
Some output line can be affected by pyoxidizer, for example the source file path
are replaced by the module name. We introduce a new condition keyword to cope
with this.
Differential Revision: https://phab.mercurial-scm.org/D11283
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Aug 2021 14:38:25 +0200] rev 47830
templater: swap `\` with `/` in more location
This is similar to the previous changeset, but apply that logic to
_readmapfile.
Differential Revision: https://phab.mercurial-scm.org/D11282
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Aug 2021 13:22:06 +0200] rev 47829
templater: swap `\` with `/` to allow the resource logic to kicks in
Without this change our wrapper to the resource logic will fails to details the
nesting and try to pass `directory/file` entry to the resource module, leading
to a crash.
Ideally, we should track down all caller passing `\` instead of `/`, but that
is far more intrusive and we would like to have a working pyoxidized version
working for 5.9 It will be easy to add a warning to the above code to track
down "bad" caller during the 6.0 cycle.
This reduce the number of failing test with pyoxidizer from 52 to 23
Differential Revision: https://phab.mercurial-scm.org/D11281
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Aug 2021 18:38:42 +0200] rev 47828
heptapod-ci: extend timeout for pytype checking
This test routinely times out for loaded or older machines.
Differential Revision: https://phab.mercurial-scm.org/D11258
Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 00:49:50 -0400] rev 47827
ci: run --pyoxidized tests on Windows
They still have numerous failure, but at least we can start fixing them now.
Differential Revision: https://phab.mercurial-scm.org/D11278
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Aug 2021 12:56:32 +0200] rev 47826
run-tests: introduce a --pyoxidized option
This options make it possible to use the pyoxidizer version to run the tests.
This is a first basic version that is windows only.
The test needs a working python, with Mercurial installed. However the
pyoxidizer product is "self contains" without a "usable" Python. There have been
discussion to have a fully functional `hg admin::python` command providing a
fully functional python interpreter, but nothing is of the sort is ready yet. In
In the meantime we use an hybrid approach, similar to what we do for testing
`rhg`. We install a full "normal" Mercurial, but also the pyxodizer product as
the official `hg binary`. That way, we use the pyoxidizer version or everything,
but test that needs to run python have it available, with a fully functional
Mercurial package.
This first version is pretty basic (Windows only, no --local, not
--with-pyoxidized), but it runs, various bug that we will have to fix.
Differential Revision: https://phab.mercurial-scm.org/D11277
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 10 Aug 2021 11:45:43 +0200] rev 47825
ci: add a "all" template to easily control "when" test run
When debugging some job (usually windows one) it is handy to have a way to pass
all other jobs to manual execution.
We add a "root" template to control just that.
Differential Revision: https://phab.mercurial-scm.org/D11276
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 10 Aug 2021 19:19:53 +0530] rev 47824
Added signature for changeset
2813d406b036
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 10 Aug 2021 19:19:47 +0530] rev 47823
Added tag 5.9rc1 for changeset
2813d406b036
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 07 Aug 2021 16:51:47 +0200] rev 47822
issue6528: add a config option to control the fixing on the fly
This will allow people who know to be safe to avoid any performance overhead
(and other potential issue).
Differential Revision: https://phab.mercurial-scm.org/D11271
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 07 Aug 2021 14:12:28 +0200] rev 47821
issue6528: also filter delta on the fly when applying a changegroup
This ensure that corrupted clone does not spread corruption to "fixed" version.
This might come at a performance cost, we will had a config option to control
this behavior in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D11270
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 07 Aug 2021 14:13:53 +0200] rev 47820
filelog: open the writing context a bit earlier in `addgroup`
This is a small change made ahead of the next patch for clarification.
Differential Revision: https://phab.mercurial-scm.org/D11269
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 07 Aug 2021 12:39:01 +0200] rev 47819
issue6528: implement _is_revision_affected_fast using callback
The delta comming from a bundle/stream does not exists in the revlog yet, so we
will need other way to retrieve the same information.
To prepare for this we split the function to use callbacks in the core logic.
Differential Revision: https://phab.mercurial-scm.org/D11268
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 07 Aug 2021 12:38:48 +0200] rev 47818
issue6528: implement _is_revision_affected using callback
The delta comming from a bundle/stream does not exists in the revlog yet, so we
will need other way to retrieve the same information.
To prepare for this we split the function to use callbacks in the core logic.
Differential Revision: https://phab.mercurial-scm.org/D11267
Raphaël Gomès <rgomes@octobus.net> [Fri, 06 Aug 2021 12:10:36 +0200] rev 47817
debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
See justification inline.
Differential Revision: https://phab.mercurial-scm.org/D11263
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Aug 2021 17:00:03 +0200] rev 47816
repair: improve performance of detection of revisions affected by
issue6528
Explanations inside the patch. I've tested this on Mozilla-Central and it's
5 times faster than the naive approach on my laptop.
Differential Revision: https://phab.mercurial-scm.org/D11262
Raphaël Gomès <rgomes@octobus.net> [Tue, 27 Jul 2021 21:45:27 +0200] rev 47815
debugcommands: introduce a debug command to repair repos affected by
issue6528
This command is quite basic and slow, it will loop over the entirety of the
filelogs in the repository and check each revision for corruption, then fixes
the affected filelogs. It takes under 25 minutes for Mozilla-Central on my
not-top-of-the-line laptop, using the `--to-report` and `--from-report` options
will make this pretty tolerable to use, I think.
This change also introduces a test for the fix.
Differential Revision: https://phab.mercurial-scm.org/D11239
Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 19:49:57 -0400] rev 47814
contrib: switch the Windows bootstrap environment to py3.9
Use the built in `venv` module instead of `virtualenv` for simplicity, and
upgrade to a modern Mercurial that supports py3.
One issue here is that `venv` doesn't copy `python3{,Y}.dll` into the `Scripts`
subdirectory, so running the `hg.exe` that gets installed immediately fails on a
clean system because Python isn't in `PATH`. There is code in `python.exe` to
detect when it is in a venv and add the original python install to the DLL
lookup path, which we don't do in `hg.exe` yet. The simple workaround for now is
to run the `hg` script with `python.exe`. Typically `PYTHONLEGACYWINDOWSSTDIO`
must be set in the environment on Windows, but the clone process works without
it.
Differential Revision: https://phab.mercurial-scm.org/D11275
Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 19:36:26 -0400] rev 47813
contrib: comment out the 64-bit py38 dependency installation on Windows
Not sure what is going on here, but it appears to not install py3.8 x64 in the
usual `C:\hgdev` directory. The x32 installer works fine. I'm assuming this is
a bug in this version of the installer, but didn't look into it too much.
Differential Revision: https://phab.mercurial-scm.org/D11274
Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 19:32:44 -0400] rev 47812
contrib: drop python2.7 from the Windows dependency install script
While we aren't quite ready to drop py27 yet, the MS compiler is no longer
available from MS (there is a copy on github if it's really needed), and that
causes the script to fail.
Differential Revision: https://phab.mercurial-scm.org/D11273
Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 19:24:46 -0400] rev 47811
contrib: log the command and args for every process installing windows deps
This is a little noisier, but makes it simple to debug when things fail.
Differential Revision: https://phab.mercurial-scm.org/D11272
Raphaël Gomès <rgomes@octobus.net> [Fri, 23 Jul 2021 10:47:12 +0200] rev 47810
windows-ci: run Windows CI automatically alongside the others
This will enable us to make Windows Python 3 a first-class citizen for the next
6.0 cycle. We will probably get some flaky tests and we're missing others that
are skipped, but we'll turn them on it future patches.
Differential Revision: https://phab.mercurial-scm.org/D11256
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Aug 2021 12:53:44 +0200] rev 47809
vfs: always use / as file separator (
issue6546)
Various part of vfs already enforce `/` usage and using `\` confuse the encoded
vfs. So we simply use `/` all the time.
Differential Revision: https://phab.mercurial-scm.org/D11260
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Aug 2021 12:53:36 +0200] rev 47808
subrepo: compare normalised vfs path
Otherwise the realpath call can turn `/` into `\` on windows confusing the
check.
(We probably needs this in more location)
Differential Revision: https://phab.mercurial-scm.org/D11259
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Aug 2021 18:25:35 +0200] rev 47807
pager: account for flakiness in Windows output
This test case is cursed and probably not worth losing more time over. This
makes apparent what the intended behavior is while still removing the flakiness
from the CI.
Differential Revision: https://phab.mercurial-scm.org/D11257
Raphaël Gomès <rgomes@octobus.net> [Fri, 23 Jul 2021 10:45:08 +0200] rev 47806
windows-ci: clean up the Heptapod CI file now that the baseline is solid
Enough work has been done one the CI side, this now works with little effort
on our side. The next patch will remove the manual switch.
Differential Revision: https://phab.mercurial-scm.org/D11254
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 21:22:02 +0200] rev 47805
test-nointerrupt: make "sure" the handler "might" trigger (
issue6558)
We are sure that the signal got sent in the right time frame, however, we still
have race, so either the code is actually buggy or we need some security to make
sure the signal get processed.
We might be affected by https://bugs.python.org/
issue43406 ?
Differential Revision: https://phab.mercurial-scm.org/D11251
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 19:26:26 +0200] rev 47804
testing: make sure write_file is "atomic"
This make sure viewer cannot see the new file with partial content.
This was likely the cause of some flakiness in `test-nointerrupt.t`
Differential Revision: https://phab.mercurial-scm.org/D11250
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 04 Aug 2021 19:45:13 +0200] rev 47803
test: disable test-subrepo-git.t in python2 + chg
I am a couple of days in try to debug that at it seems minor enough with enough
other priority to simply disable it for now.
Differential Revision: https://phab.mercurial-scm.org/D11253
Georges Racinet <georges.racinet@octobus.net> [Tue, 20 Jul 2021 17:20:19 +0200] rev 47802
hgwebdir: avoid systematic full garbage collection
Forcing a systematic full garbage collection upon each request
can serioulsy harm performance. This is reported as
https://bz.mercurial-scm.org/show_bug.cgi?id=6075
With this change we're performing the full collection according
to a new setting, `experimental.web.full-garbage-collection-rate`.
The default value is 1, which doesn't change the behavior and will
allow us to test on real use cases. If the value is 0, no full garbage
collection occurs.
Regardless of the value of the setting, a partial garbage collection
still occurs upon each request (not attempting to collect objects from
the oldest generation). This should be enough to take care of
reference cycles that have been created by the last request
(assessment of this requires changing the setting, not to be 1).
In my experience chasing memory leaks in Mercurial servers,
the full collection never reclaimed any memory, but this is with
Python 3 and biased towards small repositories.
On the other hand, as explained in the Python developer docs [1],
frequent full collections are very harmful in terms of performance if
lots of objects survive the collection, and hence stay in the
oldest generation. Note that `gc.collect()` is indeed trying to
collect the oldest generation [2]. This happens usually in two cases:
- unwanted lingering objects (i.e., an actual memory leak that
the GC cannot do anything about). Sadly, we have lots of those
these days.
- desireable long-term objects, typically in caches (not inner caches
carried by repositories, which should be collected with them). This
is a subject of interest for the Heptapod project.
In short, the flat rate that this change still permits is
probably a bad idea in most cases, and the default value can
be tweaked later on (or even be set to 0) according to experiments
in the wild.
The test is inspired from test-hgwebdir-paths.py
[1] https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation
[2] https://docs.python.org/3/library/gc.html#gc.collect
Differential Revision: https://phab.mercurial-scm.org/D11204
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 18:29:31 +0200] rev 47801
check-module-imports: ignore non-stdlib module installed by distribution
Previously, the check script would detect breezy as part of the stdlib if
installed using the debian package manager.
This silence the following complains:
tests/test-convert-bzr.t:117: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr.t:117: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
tests/test-convert-bzr-ghosts.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr-ghosts.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
tests/test-convert-bzr-treeroot.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr-treeroot.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
Differential Revision: https://phab.mercurial-scm.org/D11249
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Mon, 02 Aug 2021 08:06:27 -0400] rev 47800
remotefilelog: fix what looks like a wrong refactoring
when various store functions started returning a revlog type as the
first element of the tuple.
Differential Revision: https://phab.mercurial-scm.org/D11243
Georges Racinet <georges.racinet@octobus.net> [Sun, 01 Aug 2021 14:39:38 +0200] rev 47799
rust-nodemap: falling back to C impl as mitigation
This is a mitigation for https://bz.mercurial-scm.org/show_bug.cgi?id=6554
We see sometimes almost all data except the most recent revisions
removed from the persistent nodemap, but we don't know how to
reproduce yet.
This has sadly repercussions beyond just needing to reconstruct the
persistent nodemap: for instance, this automatically filters out
all bookmarks pointing to revisions that the nodemap cannot resolve.
If such filtering happens in a transaction, the update of the
bookmarks file that happens at the end of transaction loses all
bookmarks that have been affected. There may be similar consequences
for other data.
So this is a data loss, something that we have to prevent as soon as
possible.
As a mitigation measure, we will now fallback to the C implementation
in case nodemap lookups failed. This will add some latency, e.g., in
discovery, yet less than disabling the persistent nodemap entirely.
We considered implementing the fallback directly on the Python
side, but `revlog.get_rev()` is not systematically used, there are
also several direct calls to the index method (`self.index.rev()` for
a `revlog` instance). It is therefore more direct to implement the
mitigation in the rust-cpython wrapper.
Differential Revision: https://phab.mercurial-scm.org/D11238
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:07 +0300] rev 47798
obsolete: disable other evolution config options if createmarkers is off
We used to raise an abort in this case, but recent changes to local clone
command (
377d8fc20e34) resulted in destrepo both caring about
experimental.evolution config options and not initializing extensions.
So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local
clone stopped working after
377d8fc20e34 because evolve sets
experimental.evolution=all, but only on srcrepo, for destrepo the extension is
not initialized. It's possible to make local cloning work by initializing
extensions for destrepo in some cases, but in other cases (e.g. allowdivergence
in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail.
In a discussion with Pierre-Yves David it was decided to simply force other
evolution options to be false if createmarkers is not enabled.
Differential Revision: https://phab.mercurial-scm.org/D11223
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:47:21 +0300] rev 47797
fix: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11222
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:41 +0300] rev 47796
rebase: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11221
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:07 +0300] rev 47795
obsolete: disable other evolution config options if createmarkers is off
We used to raise an abort in this case, but recent changes to local clone
command (
377d8fc20e34) resulted in destrepo both caring about
experimental.evolution config options and not initializing extensions.
So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local
clone stopped working after
377d8fc20e34 because evolve sets
experimental.evolution=all, but only on srcrepo, for destrepo the extension is
not initialized. It's possible to make local cloning work by initializing
extensions for destrepo in some cases, but in other cases (e.g. allowdivergence
in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail.
In a discussion with Pierre-Yves David it was decided to simply force other
evolution options to be false if createmarkers is not enabled.
Differential Revision: https://phab.mercurial-scm.org/D11223
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:47:21 +0300] rev 47794
fix: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11222