Pulkit Goyal <7895pulkit@gmail.com> [Thu, 18 Jun 2020 17:54:39 +0530] rev 44987
tests: use proctutil.stdout.write() instead of print() in test-extension.t
I was debugging this test failure on python3 + chg. I get the following hunk as
test failure:
```
@@ -206,6 +206,18 @@ Check normal command's load order of ext
4) bar uipopulate
5) foo reposetup
5) bar reposetup
+ 4) foo uipopulate (chg !)
+ 4) bar uipopulate (chg !)
+ 4) foo uipopulate (chg !)
+ 4) bar uipopulate (chg !)
+ 4) foo uipopulate (chg !)
+ 4) bar uipopulate (chg !)
+ 4) foo uipopulate (chg !)
+ 4) bar uipopulate (chg !)
+ 4) foo uipopulate (chg !)
+ 4) bar uipopulate (chg !)
+ 5) foo reposetup (chg !)
+ 5) bar reposetup (chg !)
0:
c24b9ac61126
```
After hours of debugging and head scracthing, I figured out that something is
wrong with output flushing. I initially switched the print() statements to
ui.warn() but thanks to Yuya who suggested using procutil.stdout.write()
instead.
Augie Fackler <augie@google.com> [Wed, 08 Jan 2020 11:33:41 -0500] rev 44986
fuzz: tell manifest fuzzer about longer node hashes
Differential Revision: https://phab.mercurial-scm.org/D8374
Manuel Jacob <me@manueljacob.de> [Mon, 01 Jun 2020 20:57:14 +0200] rev 44985
absorb: preserve changesets which were already empty
Most commands in Mercurial (commit, rebase, absorb itself) don’t create empty
changesets or drop them if they become empty. If there’s a changeset that’s
empty, it must be a deliberate choice of the user. At least it shouldn’t be
absorb’s responsibility to prune them. The fact that changesets that became
empty during absorb are pruned, is unaffected by this.
This case was found while writing patches which make it possible to configure
absorb and rebase to not drop empty changesets. Even without having such config
set, I think it’s valuable to preserve changesets which were already empty.
Manuel Jacob <me@manueljacob.de> [Mon, 01 Jun 2020 11:07:33 +0200] rev 44984
absorb: preserve branch-closing changesets even if empty
This makes the behavior consistent with 'hg commit', which allows to create
otherwise empty changesets if they close the branch. A lost branch closure can
inadvertently re-open a branch, so it should be preserved.
Manuel Jacob <me@manueljacob.de> [Mon, 01 Jun 2020 10:33:00 +0200] rev 44983
absorb: preserve branch-changing changesets even if empty
This makes the behavior consistent with 'hg commit', which allows to create
otherwise empty changesets if the branch changes compared to the parent. A
branch change can denote important information, so it should be preserved.
Antoine Cezar <antoine.cezar@octobus.net> [Mon, 22 Jun 2020 15:19:35 +0530] rev 44982
rhg: add Command trait for subcommands implemented by rhg
Normalizes the interface of the cli's commands
Differential Revision: https://phab.mercurial-scm.org/D8611
Differential Revision: https://phab.mercurial-scm.org/D8648
Antoine Cezar <antoine.cezar@octobus.net> [Fri, 05 Jun 2020 10:28:58 +0200] rev 44981
rhg: add rhg crate
The goal of rhg is to speedup some of hg's commands when possible by bypassing
python entirely for the time being.
It is by no means a replacement for hg as it will not support extentions or
configuration and implement only a subset of hg's commands and options.
Only use rhg if you understand what the tradeoffs are.
Differential Revision: https://phab.mercurial-scm.org/D8610
Antoine Cezar <antoine.cezar@octobus.net> [Fri, 05 Jun 2020 08:48:09 +0200] rev 44980
hg-core: add FindRoot operation to find repository root path
Differential Revision: https://phab.mercurial-scm.org/D8609
Joerg Sonnenberger <joerg@bec.de> [Tue, 23 Jun 2020 16:07:18 +0200] rev 44979
share: provide a more useful text for hg help
Differential Revision: https://phab.mercurial-scm.org/D8651
Martin von Zweigbergk <martinvonz@google.com> [Thu, 25 Jun 2020 00:06:23 -0700] rev 44978
procutil: make recent fix for zombies compatible with py2
The fix in
ed684a82e29b (procutil: always waiting on child processes
to prevent zombies with 'hg serve', 2020-05-07) works only on Python 3
because it passes a `daemon` argument to `threading.Thread()`. Python
2 requires you to assign to the `.daemon` property instead. Python 3
also seems to support that, so this patch fixes the code by
unconditionally using the old form.
Differential Revision: https://phab.mercurial-scm.org/D8657
Manuel Jacob <me@manueljacob.de> [Wed, 24 Jun 2020 04:25:34 +0200] rev 44977
pycompat: fix crash when default locale is unknown
Instead, fall back to the filesystem encoding if the default locale is unknown.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 22 Jun 2020 22:15:57 -0700] rev 44976
help: document meaning of '%' in graphlog output
I added the feature in
14d0e89520a2 (graphlog: use '%' for other
context in merge conflict, 2020-01-28), but I didn't think to look for
documentation until today (because an internal user asked about it).
Differential Revision: https://phab.mercurial-scm.org/D8649
Kyle Lippincott <spectral@google.com> [Wed, 17 Jun 2020 16:11:11 -0700] rev 44975
py3: fix broken man page generation, it was generating `(default: NUL*)`
`bytes(default)` was producing things like `(default: \x00)` when handed
non-bytes values such as `1`, `10`, or `True`. The man page generation would
apparently ignore these bytes and produce man pages that had the string
`(default: )`.
Test Plan:
- Ran `cd doc; python3 gendoc.py "hg.1.gendoc"` and grepped for bad output
- Ran `make deb`, extracted the deb, manually inspected `hg.1` file.
Differential Revision: https://phab.mercurial-scm.org/D8639
Antoine Cezar <antoine.cezar@octobus.net> [Fri, 05 Jun 2020 08:46:35 +0200] rev 44974
hg-core: add Operation interface for high-level hg operations
A distinction is made between operations and commands.
An operation is a high-level function of mercurial whereas a command is what is exposed by the cli.
A single command can use several operations to achieve its goal.
Differential Revision: https://phab.mercurial-scm.org/D8608
Raphaël Gomès <rgomes@octobus.net> [Mon, 15 Jun 2020 18:26:40 +0200] rev 44973
rust: do a clippy pass
This is the result of running `cargo clippy` on hg-core/hg-cpython and fixing
the lints that do not require too much code churn (and would warrant a separate
commit/complete refactor) and only come from our code (a lot of warnings in
hg-cpython come from `rust-cpython`).
Most of those were good lints, two of them was the linter not being smart
enough (or compiler to get up to `clippy`'s level depending on how you see it).
Maybe in the future we could have `clippy` be part of the CI.
Differential Revision: https://phab.mercurial-scm.org/D8635
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:38:50 +0200] rev 44972
py3: fix comparison between int and None
If stop is None, the condition was always false on Python 2, as None compares
smaller than ints. Therefore we make the condition false if stop is None.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:33:49 +0200] rev 44971
py3: pass regex as bytes
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:32:10 +0200] rev 44970
py3: avoid using %r format on bytes
Before the patch, the 'b' prefix appeared in the formatted string. Wrapping the
bytes as pycompat.bytestr solves this problem.
Eventually, I think that we should move away from using %r (like
975e517451a6
and
4d6019c0e0ef did), but that would change output of non-ASCII bytes on
Python 2, so we can’t do it on the stable branch. Also, many places continue to
use %r, so it would be a good idea to do the change all at once.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:03:00 +0200] rev 44969
py3: use `%d` for int in % formatting
On Python 3, `%s` is an alias to `%b`, which requires that the object implements
`__bytes__()`, which is not the case for `int`.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:00:20 +0200] rev 44968
py3: pass native string to urlreq.url2pathname()
Of course, I’m not happy with the warning, but it’s better than crashing.
Solving the problem properly is hard, and non-UTF-8 percent-encoded bytes in
file URLs seem rare enough to block solving that all file URLs (even if not
SVN-specific) will cause a crash.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 12:59:45 +0200] rev 44967
py3: suppress DeprecationWarning about deprecated base64 module aliases
base64.encodestring() / base64.decodestring() were renamed to
base64.encodebytes() / base64.decodebytes() in Python 3. The old names still
worked, but raised a DeprecationWarning.
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:38:02 +0200] rev 44966
py3: use `pycompat.ziplist()`
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:34:23 +0200] rev 44965
py3: use `%d` for int in % formatting
On Python 3, `%s` is an alias to `%b`, which requires that the object implements
`__bytes__()`, which is not the case for `int`.
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:30:24 +0200] rev 44964
py3: fix bytes iteration
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:09:55 +0200] rev 44963
py3: unbyteify arguments to warnings.filterwarnings()
This fixes a crash when trying to import the convert extension on Python 3.
Augie Fackler <augie@google.com> [Mon, 15 Jun 2020 15:14:16 -0400] rev 44962
fuzz: add config knob for PYTHON_CONFIG_FLAGS
I'll clean this up once we get oss-fuzz to use Python 3.8 instead of
2.7, but for now we need a way to evolve the flags passed to
python-config in lockstep with the Python version. Yuck.
Differential Revision: https://phab.mercurial-scm.org/D8637
Augie Fackler <augie@google.com> [Mon, 15 Jun 2020 15:13:01 -0400] rev 44961
pyutil: this has taken so long to fix, I'm using 3.8 now
Differential Revision: https://phab.mercurial-scm.org/D8636
Augie Fackler <augie@google.com> [Mon, 15 Jun 2020 12:00:15 -0400] rev 44960
merge with stable
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:15:11 +0800] rev 44959
tests: adjust to the new format in pyflakes output
According to the pyflakes' NEWS.rst, the default output format changed
recently:
2.2.0 (2020-04-08)
- Include column information in error messages
So the lines now read:
contrib/perf.py:149:15 undefined name 'xrange'
mercurial/hgweb/server.py:427:13 undefined name 'reload'
mercurial/util.py:2862:24 undefined name 'file'
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8630
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:12:49 +0800] rev 44958
tests: consistently use pyflakes as a Python module
We check availability of pyflakes as a module, and also running it for real as
a module. Only fair to test filterpyflakes.py working correctly when using
pyflakes as a module too.
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8629
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:19:27 +0800] rev 44957
tests: skip pyflakes for mercurial/thirdparty/
The current version of pyflakes (2.2.0) correctly detects one issue:
mercurial/thirdparty/selectors2.py:335:40 '...'.format(...) has unused arguments at position(s): 1
But we're not interested in fixing lint errors in third-party code, so we need
to exclude at least selectors2.py. And in the discussion for this patch it was
decided to just skip the entire thirdparty directory.
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8628
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 13 Jun 2020 11:06:22 +0200] rev 44956
zeroconf: fix non existant formatting in the vendored zeroconf module
On Tue Mar 1st 2016 at 09:33:39 timeless decided to wrap long line in
`hgext/zeroconf/Zeroconf.py`. Doing so, he fat fingered a "%w" instead of a "%s"
in a string. %w does not exists, 4 year later, pyflakes (rightfully) complains
about it. So I am fixing it.
Differential Revision: https://phab.mercurial-scm.org/D8627
Adam Hull <adam@hmlad.com> [Fri, 12 Jun 2020 14:22:34 -0700] rev 44955
ignore: note debugignore on ignore man page
It took me a long time to find debugignore. I found the ignore man page
quickly. This change adds a debugging section to the ignore man page
letting people know there is a debug command.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 13 Jun 2020 11:57:58 +0200] rev 44954
nodemap: fix validity checking when revlog is too short
We cannot check the nodeid of a revision that is not even there. We add a simple
fix and simple test.
Manuel Jacob <me@manueljacob.de> [Fri, 12 Jun 2020 23:43:56 +0200] rev 44953
tests: remove unused creation of file and outdated text
It was forgotten to remove this in
fb0de0bcd297.
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:19:27 +0800] rev 44952
tests: skip pyflakes for mercurial/thirdparty/
The current version of pyflakes (2.2.0) correctly detects one issue:
mercurial/thirdparty/selectors2.py:335:40 '...'.format(...) has unused arguments at position(s): 1
But we're not interested in fixing lint errors in third-party code, so we need
to exclude at least selectors2.py. And in the discussion for this patch it was
decided to just skip the entire thirdparty directory.
Differential Revision: https://phab.mercurial-scm.org/D8619
Augie Fackler <augie@google.com> [Tue, 09 Jun 2020 17:13:26 -0400] rev 44951
git: add debug logging when there's a mismatch in the cached heads list
The dag rebuild can be expensive, so let's try and avoid bugs where it
transparently rebuilds all the time for no reason. This would have
prevented the issue fixed in D8622.
Differential Revision: https://phab.mercurial-scm.org/D8625
Manuel Jacob <me@manueljacob.de> [Wed, 10 Jun 2020 13:02:39 +0200] rev 44950
py3: make stdout line-buffered if connected to a TTY
Status messages that are to be shown on the terminal should be written to the
file descriptor before anything further is done, to keep the user updated.
One common way to achieve this is to make stdout line-buffered if it is
connected to a TTY. This is done on Python 2 (except on Windows, where libc,
which the CPython 2 streams depend on, does not properly support this).
Python 3 rolls it own I/O streams. On Python 3, buffered binary streams can't be
set line-buffered. The previous code (added in
227ba1afcb65) incorrectly
assumed that on Python 3, pycompat.stdout (sys.stdout.buffer) is already
line-buffered. However the interpreter initializes it with a block-buffered
stream or an unbuffered stream (when the -u option or the PYTHONUNBUFFERED
environment variable is set), never with a line-buffered stream.
One example where the current behavior is unacceptable is when running
`hg pull https://www.mercurial-scm.org/repo/hg` on Python 3, where the line
"pulling from https://www.mercurial-scm.org/repo/hg" does not appear on the
terminal before the hg process blocks while waiting for the server.
Various approaches to fix this problem are possible, including:
1. Weaken the contract of procutil.stdout to not give any guarantees about
buffering behavior. In this case, users of procutil.stdout need to be
changed to do enough flushes. In particular,
1. either ui must insert enough flushes for ui.write() and friends, or
2. ui.write() and friends get split into flushing and fully buffered
methods, or
3. users of ui.write() and friends must flush explicitly.
2. Make stdout unbuffered.
3. Make stdout line-buffered. Since Python 3 does not natively support that for
binary streams, we must implement it ourselves.
(2.) is problematic because using unbuffered I/O changes the performance
characteristics significantly compared to line-buffered (which is used on
Python 2) and this would be a regression.
(1.2.) and (1.3) are a substantial amount of work. It’s unclear whether the
added complexity would be justified, given that raw performance doesn’t matter
that much when writing to a terminal much faster than the user could read it.
(1.1.) pushes complexity into the ui class instead of separating the concern of
how stdout is buffered. Other users of procutil.stdout would still need to take
care of the flushes.
This patch implements (3.). The general performance considerations are very
similar to (1.1.). The extra method invocation and method forwarding add a
little more overhead if the class is used. In exchange, it doesn’t add overhead
if not used.
For the benchmarks, I compared the previous implementation (incorrect on Python
3), (1.1.), (3.) and (2.). The command was chosen so that the streams were
configured as if they were writing to a TTY, but actually write to a pager,
which is also the default:
HGRCPATH=/dev/null python3 ./hg --cwd ~/vcs/mozilla-central --time --pager yes --config pager.pager='cat > /dev/null' status --all
previous:
time: real 7.880 secs (user 7.290+0.050 sys 0.580+0.170)
time: real 7.830 secs (user 7.220+0.070 sys 0.590+0.140)
time: real 7.800 secs (user 7.210+0.050 sys 0.570+0.170)
(1.1.) using Yuya Nishihara’s patch:
time: real 9.860 secs (user 8.670+0.350 sys 1.160+0.830)
time: real 9.540 secs (user 8.430+0.370 sys 1.100+0.770)
time: real 9.830 secs (user 8.630+0.370 sys 1.180+0.840)
(3.) using this patch:
time: real 9.580 secs (user 8.480+0.350 sys 1.090+0.770)
time: real 9.670 secs (user 8.480+0.330 sys 1.170+0.860)
time: real 9.640 secs (user 8.500+0.350 sys 1.130+0.810)
(2.) using a previous patch by me:
time: real 10.480 secs (user 8.850+0.720 sys 1.590+1.500)
time: real 10.490 secs (user 8.750+0.750 sys 1.710+1.470)
time: real 10.240 secs (user 8.600+0.700 sys 1.590+1.510)
As expected, there’s no difference on Python 2, as exactly the same code paths
are used:
previous:
time: real 6.950 secs (user 5.870+0.330 sys 1.070+0.770)
time: real 7.040 secs (user 6.040+0.360 sys 0.980+0.750)
time: real 7.070 secs (user 5.950+0.360 sys 1.100+0.760)
this patch:
time: real 7.010 secs (user 5.900+0.390 sys 1.070+0.730)
time: real 7.000 secs (user 5.850+0.350 sys 1.120+0.760)
time: real 7.000 secs (user 5.790+0.380 sys 1.170+0.710)
Yuya Nishihara <yuya@tcha.org> [Tue, 02 Jun 2020 21:44:57 +0900] rev 44949
simplemerge: rewrite flag merging loop as expression
I feel binary operations are more readable.
Yuya Nishihara <yuya@tcha.org> [Tue, 02 Jun 2020 21:40:49 +0900] rev 44948
simplemerge: leverage pycompat function to convert byte string to set
Yuya Nishihara <yuya@tcha.org> [Tue, 02 Jun 2020 21:39:07 +0900] rev 44947
simplemerge: fix function name that tests if ctx is not null revision
Hollis Blanchard <hollis_blanchard@mentor.com> [Tue, 09 Jun 2020 13:18:21 -0700] rev 44946
git: decode node IDs back into Python strings (
issue6349)
db.text_factory = bytes, so the database contains only strings. The object IDs
we get from pygit2 are Python strings. b'foo' != 'foo'
This change allows the "don't reindex" optimization to work by allowing the
"cur_cache_heads == cache_heads" comparison a few lines down to succeed.
Differential Revision: https://phab.mercurial-scm.org/D8622
Sushil khanchi <sushilkhanchi97@gmail.com> [Tue, 09 Jun 2020 22:02:09 +0530] rev 44945
phabricator: make it clear what happen when no response
Differential Revision: https://phab.mercurial-scm.org/D8621
Sushil khanchi <sushilkhanchi97@gmail.com> [Mon, 08 Jun 2020 11:43:07 +0530] rev 44944
tests: make it clear what happen when no response entered
Differential Revision: https://phab.mercurial-scm.org/D8620
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 18 Jan 2020 10:07:07 -0800] rev 44943
localrepo: handle ValueError during repository opening
Python 3.8 can raise ValueError on attempt of an I/O operation
against an illegal path. This was causing test-remotefilelog-gc.t
to fail on Python 3.8.
This commit teaches repository opening to handle ValueError
and re-raise an Abort on failure.
An arguably better solution would be to implement this logic
in the vfs layer. But that seems like a bag of worms and I don't
want to go down that rabbit hole. Until users report uncaught
ValueError exceptions in the wild, I think it is fine to patch
this at the only occurrence our test harness is finding it.
Differential Revision: https://phab.mercurial-scm.org/D7944
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 May 2020 12:56:13 +0200] rev 44942
metadata: filter the `removed` set to only contains relevant data
The `files` entry can be bogus and contains too many entries. This can badly
combines with the computation of `removed` inflating the set size. The can lead
to the changesets centric rename computation to process much more data than
needed, slowing it down (and increasing space taken by data storage).
In practice newer commits already that reduced set, this applies this "fix" to
older changeset.
Differential Revision: https://phab.mercurial-scm.org/D8589
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 May 2020 12:45:39 +0200] rev 44941
files: extract code for extra filtering of the `removed` entry into copies
We want to reduce the set of `removed` files that to the set of files actually
removed. That `removed` set is used as of the changeset centric algorithm,
having smaller sets means less processing and faster computation.
In this changeset we extract the code to be a function of it own. We will make
use of it in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D8588
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 May 2020 12:26:08 +0200] rev 44940
metadata: move computation related to files touched in a dedicated module
This was suggested by Yuya Nishihara a while back. Since I am about to add more
metadata related computation, lets create a new repositories.
Differential Revision: https://phab.mercurial-scm.org/D8587