Manuel Jacob <me@manueljacob.de> [Fri, 03 Jul 2020 23:25:19 +0200] rev 44840
py3: fix crash when server address is 0.0.0.0 (issue6362)
`socket.getfqdn()` assumes that the name is passed as `str` on Python 3 and
always returns `str` in this case. Mercurial passed `bytes` (but still expected
a `str` result), which worked by chance in many cases, except for e.g.
b'0.0.0.0', which was returned unchanged, breaking later code.
Instead of calling `socket.getfqdn()`, we can also use `self.server_name` from
the base `HTTPServer` class, which already stores the FQDN of the locally-bound
socket name (see `BaseHTTPServer.py` in the Python 2 stdlib and
`http/server.py` in the Python 3 stdlib).
Axel Hecht <axel@pike.org> [Wed, 01 Jul 2020 15:43:15 +0200] rev 44839
run-tests: find python binary on Python 3 (issue6361)
Return strings from _findprogram as all callers expect
unicode strings.
Previously the check in _usecorrectpython agains sysexecutable
was always false on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D8674
Joerg Sonnenberger <joerg@bec.de> [Mon, 29 Jun 2020 16:36:53 +0200] rev 44838
tests: ignore possible diagnostics from gpg 2.2
Differential Revision: https://phab.mercurial-scm.org/D8672
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 01 Jul 2020 23:30:47 +0530] rev 44837
Added signature for changeset 0ea9c86fac89
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 01 Jul 2020 23:30:39 +0530] rev 44836
Added tag 5.4.2 for changeset 0ea9c86fac89
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 07:23:29 +0200] rev 44835
convert: handle percent-encoded bytes in file URLs like Subversion
75b59d221aa3 added most of the code that gets removed by this patch. It helped
making progress on Python 3, but the reasoning was wrong in many ways. I tried
to retract it while it was queued, but it was too late.
Back then, I was asssuming that what happened on Python 2 (preserving bytes) is
correct and my Python 3 change is a hack. However it turned out that Subversion
interprets percent-encoded bytes as UTF-8. Accepting the same format as
Subversion is a good idea.
Consistency with urlreq.pathname2url() (as described in the removed comment)
doesn’t matter because that function is only used for passing paths to urllib.
This is not a backwards-incompatible change because before 5c0d5b48e58c,
non-ASCII filenames didn’t work at all on Python 2.
When the locale encoding is ISO-8859-15, `svn` accepts `file:///tmp/a%E2%82%AC`
for `/tmp/a€`. Before this patch, this was the case for this extension on
Python 3, but not on Python 2. This patch makes it work like with `svn` on both
Python 2 and Python 3.
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 16:39:45 +0200] rev 44834
convert: add docstring on convert.subversion.geturl()
The function is unusual for a bytes-handling function in Mercurial because it
can’t handle arbitrary bytes. Therefore we should document this fact.
Pointed out by Yuya Nishihara while reviewing e3b19004087a.
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 01:32:17 +0200] rev 44833
tests: use path inside test dir
This will make the diff for the next patch less noisy.
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 05:30:47 +0200] rev 44832
convert: convert URLs to UTF-8 for Subversion
Preamble: for comprehension, note that the `path` of geturl() would better be
called `path_or_url` (the argument of the call of getsvn() is called `url`).
For HTTP(S) URLs, the changes don’t make a difference, as they are restricted to
ASCII.
For file URLs, the reasoning is the same as for paths: we have to roundtrip with
what Subversion is doing.
When the locale encoding is ISO-8859-15, trying to convert a SVN repo
`file:///tmp/a€` failed before like this:
file:///tmp/a%A4 does not look like a Subversion repository to libsvn version 1.14.0
Decoding the path using the locale encoding can fail. In this case, we have to
bail out, as Subversion won’t be able to do anything useful with the path.
Manuel Jacob <me@manueljacob.de> [Mon, 29 Jun 2020 15:03:36 +0200] rev 44831
convert: correctly convert paths to UTF-8 for Subversion
The previous code using encoding.tolocal() only worked by chance in these
situations:
* The string is ASCII: The fast path was triggered and the string was returned
unmodified.
* The local encoding is UTF-8: The source and target encoding is the same.
* The string is not valid UTF-8 and the native encoding is ISO-8859-1: If the
string doesn’t decode using UTF-8, ISO-8859-1 is tried as a fallback. During
`hg convert`, the local encoding is always UTF-8. The irony is that in this
case, encoding.tolocal() behaves like what someone would expect the reverse
function, encoding.fromlocal(), to do.
When the locale encoding is ISO-8859-15, trying to convert a SVN repo `/tmp/a€`
failed before like this:
file:///tmp/a%C2%A4 does not look like a Subversion repository to libsvn version 1.14.0
The correct URL is `file:///tmp/a%E2%82%AC`.
Unlike previously (with the ISO-8859-1 fallback), decoding the path using the
locale encoding can fail. In this case, we have to bail out, as Subversion
won’t be able to do anything useful with the path.
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 05:04:36 +0200] rev 44830
py3: pass URL as str
Before the patch, HTTP(S) URLs were never recognized as a Subversion repository
on Python 3.
Manuel Jacob <me@manueljacob.de> [Tue, 30 Jun 2020 04:55:52 +0200] rev 44829
convert: bail out in Subversion source if encountering non-ASCII HTTP(S) URL
Before this patch, in the tested case, urllib raised `httplib.InvalidURL: URL
can't contain control characters. '/\xff/!svn/ver/0/.svn' (found at least
'\xff')`, which resulted in that the URL was never recognized as a Subversion
repository.
This patch adds a check that bails out if the URL contains non-ASCII characters.
The warning is not overly user-friendly, but giving the user something to type
into a search engine is definitively better than not explaining why the
repository was not recognized.
We could support non-ASCII chracters by quoting them before passing them to
urllib. However, we would want to be compatible with what the `svn` command
does, which converts the URL from the locale encoding to UTF-8, percent-encodes
it and sends it to the server. If the locale encoding is not UTF-8, the
behavior is IMHO not very intuitive, as the `svn` command may send different
(percent-encoded) octets than what was passed on the console. Instead of
copying this behavior, we better leave it forbidden.
Manuel Jacob <me@manueljacob.de> [Mon, 29 Jun 2020 02:05:12 +0200] rev 44828
run-tests: fix escapes with conditions
Before this fix, escapes with conditions in tests failed like this on Python 3:
$ $PYTHON -c 'from mercurial.utils.procutil import stdout; stdout.write(b"\xff")'
- \xff (no-eol) (esc) (true !)
+ \xff (no-eol) (esc)
The unicode_escape encoding decodes br'\xff' to u'\xff'. To convert the first
256 code points to bytes with the same ordinal, the latin-1 encoding must be
used.
Escapes without conditions already worked before on Python 3, but not through
`el == l` a few lines below the changed line in run-tests.py. I didn’t
investigate further.
Manuel Jacob <me@manueljacob.de> [Sun, 28 Jun 2020 18:02:45 +0200] rev 44827
convert: set LC_CTYPE around calls to Subversion bindings
The Subversion bindings require that LC_CTYPE is set. However, we don’t want to
set it all the time, as it changes the behavior of str methods on Python 2. The
taken approach is hopefully fine-grained enough to not trigger any
locale-specfic behavior of the str methods and coarse-grained enough to not
clutter the code.
Emulating the with-statement behavior in before() and after() should be safe, as
after() is always called when before() is called. hgext.convert.hg takes a
similar approach.
Manuel Jacob <me@manueljacob.de> [Sun, 28 Jun 2020 18:02:45 +0200] rev 44826
curses: do not initialize LC_ALL to user settings (issue6358)
701341f57ceb moved the setlocale() call to right before curses was used. This
didn’t fully solve the problem it was supposed to solve (locale-dependent
functions, like date formatting/parsing and str methods on Python 2), but only
postponed it.
Initializing LC_CTYPE seems to be sufficient for curses to work correctly.
Therefore LC_CTYPE is set while curses is used and reset afterwards. Some
locale-dependent str methods might behave differently on Python 2 while curses
is used, but that shouldn’d be a problem.
Manuel Jacob <me@manueljacob.de> [Thu, 25 Jun 2020 03:46:07 +0200] rev 44825
hgweb: encode WSGI environment like OS environment
Previously, the WSGI environment keys and values were encoded using latin-1.
This resulted in a crash if a WSGI environment key or value could not be encoded
using latin-1.
On Unix, the OS environment is byte-based. Therefore we should do the reverse of
what Python does for os.environ.
On Windows, there’s no native byte-based OS environment. Therefore we should do
the same as what mercurial.encoding does with the OS environment.
Manuel Jacob <me@manueljacob.de> [Thu, 25 Jun 2020 03:10:13 +0200] rev 44824
hgweb: deduplicate code
A following patch will change the way keys and values are encoded. To reduce the
diff, I’ve split off the uninteresting part.
Joerg Sonnenberger <joerg@bec.de> [Tue, 23 Jun 2020 16:07:18 +0200] rev 44823
share: provide a more useful text for hg help
Differential Revision: https://phab.mercurial-scm.org/D8651
Martin von Zweigbergk <martinvonz@google.com> [Thu, 25 Jun 2020 00:06:23 -0700] rev 44822
procutil: make recent fix for zombies compatible with py2
The fix in ed684a82e29b (procutil: always waiting on child processes
to prevent zombies with 'hg serve', 2020-05-07) works only on Python 3
because it passes a `daemon` argument to `threading.Thread()`. Python
2 requires you to assign to the `.daemon` property instead. Python 3
also seems to support that, so this patch fixes the code by
unconditionally using the old form.
Differential Revision: https://phab.mercurial-scm.org/D8657
Manuel Jacob <me@manueljacob.de> [Wed, 24 Jun 2020 04:25:34 +0200] rev 44821
pycompat: fix crash when default locale is unknown
Instead, fall back to the filesystem encoding if the default locale is unknown.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 22 Jun 2020 22:15:57 -0700] rev 44820
help: document meaning of '%' in graphlog output
I added the feature in 14d0e89520a2 (graphlog: use '%' for other
context in merge conflict, 2020-01-28), but I didn't think to look for
documentation until today (because an internal user asked about it).
Differential Revision: https://phab.mercurial-scm.org/D8649
Kyle Lippincott <spectral@google.com> [Wed, 17 Jun 2020 16:11:11 -0700] rev 44819
py3: fix broken man page generation, it was generating `(default: NUL*)`
`bytes(default)` was producing things like `(default: \x00)` when handed
non-bytes values such as `1`, `10`, or `True`. The man page generation would
apparently ignore these bytes and produce man pages that had the string
`(default: )`.
Test Plan:
- Ran `cd doc; python3 gendoc.py "hg.1.gendoc"` and grepped for bad output
- Ran `make deb`, extracted the deb, manually inspected `hg.1` file.
Differential Revision: https://phab.mercurial-scm.org/D8639
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:38:50 +0200] rev 44818
py3: fix comparison between int and None
If stop is None, the condition was always false on Python 2, as None compares
smaller than ints. Therefore we make the condition false if stop is None.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:33:49 +0200] rev 44817
py3: pass regex as bytes
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:32:10 +0200] rev 44816
py3: avoid using %r format on bytes
Before the patch, the 'b' prefix appeared in the formatted string. Wrapping the
bytes as pycompat.bytestr solves this problem.
Eventually, I think that we should move away from using %r (like 975e517451a6
and 4d6019c0e0ef did), but that would change output of non-ASCII bytes on
Python 2, so we can’t do it on the stable branch. Also, many places continue to
use %r, so it would be a good idea to do the change all at once.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:03:00 +0200] rev 44815
py3: use `%d` for int in % formatting
On Python 3, `%s` is an alias to `%b`, which requires that the object implements
`__bytes__()`, which is not the case for `int`.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 14:00:20 +0200] rev 44814
py3: pass native string to urlreq.url2pathname()
Of course, I’m not happy with the warning, but it’s better than crashing.
Solving the problem properly is hard, and non-UTF-8 percent-encoded bytes in
file URLs seem rare enough to block solving that all file URLs (even if not
SVN-specific) will cause a crash.
Manuel Jacob <me@manueljacob.de> [Tue, 16 Jun 2020 12:59:45 +0200] rev 44813
py3: suppress DeprecationWarning about deprecated base64 module aliases
base64.encodestring() / base64.decodestring() were renamed to
base64.encodebytes() / base64.decodebytes() in Python 3. The old names still
worked, but raised a DeprecationWarning.
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:38:02 +0200] rev 44812
py3: use `pycompat.ziplist()`
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:34:23 +0200] rev 44811
py3: use `%d` for int in % formatting
On Python 3, `%s` is an alias to `%b`, which requires that the object implements
`__bytes__()`, which is not the case for `int`.
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:30:24 +0200] rev 44810
py3: fix bytes iteration
Manuel Jacob <me@manueljacob.de> [Mon, 15 Jun 2020 03:09:55 +0200] rev 44809
py3: unbyteify arguments to warnings.filterwarnings()
This fixes a crash when trying to import the convert extension on Python 3.
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:15:11 +0800] rev 44808
tests: adjust to the new format in pyflakes output
According to the pyflakes' NEWS.rst, the default output format changed
recently:
2.2.0 (2020-04-08)
- Include column information in error messages
So the lines now read:
contrib/perf.py:149:15 undefined name 'xrange'
mercurial/hgweb/server.py:427:13 undefined name 'reload'
mercurial/util.py:2862:24 undefined name 'file'
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8630
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:12:49 +0800] rev 44807
tests: consistently use pyflakes as a Python module
We check availability of pyflakes as a module, and also running it for real as
a module. Only fair to test filterpyflakes.py working correctly when using
pyflakes as a module too.
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8629
Anton Shestakov <av6@dwimlabs.net> [Sat, 06 Jun 2020 19:19:27 +0800] rev 44806
tests: skip pyflakes for mercurial/thirdparty/
The current version of pyflakes (2.2.0) correctly detects one issue:
mercurial/thirdparty/selectors2.py:335:40 '...'.format(...) has unused arguments at position(s): 1
But we're not interested in fixing lint errors in third-party code, so we need
to exclude at least selectors2.py. And in the discussion for this patch it was
decided to just skip the entire thirdparty directory.
This is a graft of a similar fix that ended up on default.
Differential Revision: https://phab.mercurial-scm.org/D8628
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 13 Jun 2020 11:06:22 +0200] rev 44805
zeroconf: fix non existant formatting in the vendored zeroconf module
On Tue Mar 1st 2016 at 09:33:39 timeless decided to wrap long line in
`hgext/zeroconf/Zeroconf.py`. Doing so, he fat fingered a "%w" instead of a "%s"
in a string. %w does not exists, 4 year later, pyflakes (rightfully) complains
about it. So I am fixing it.
Differential Revision: https://phab.mercurial-scm.org/D8627
Adam Hull <adam@hmlad.com> [Fri, 12 Jun 2020 14:22:34 -0700] rev 44804
ignore: note debugignore on ignore man page
It took me a long time to find debugignore. I found the ignore man page
quickly. This change adds a debugging section to the ignore man page
letting people know there is a debug command.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 13 Jun 2020 11:57:58 +0200] rev 44803
nodemap: fix validity checking when revlog is too short
We cannot check the nodeid of a revision that is not even there. We add a simple
fix and simple test.
Manuel Jacob <me@manueljacob.de> [Tue, 09 Jun 2020 05:24:45 +0200] rev 44802
resourceutil: fix location of line comments
These comments slipped out of position when the sources where formatted with
black in 2372284d9457.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Apr 2020 00:33:00 -0400] rev 44801
rebase: avoid clobbering wdir() with --dry-run or --confirm (issue6291)
See 56d3e0b499df for the source of adding originalwd to the list of things that
cause wdir to be updated. That change didn't come with tests, and attempts to
recreate the scenario described have thus far failed.
Differential Revision: https://phab.mercurial-scm.org/D8489
Matt Harbison <matt_harbison@yahoo.com> [Thu, 30 Apr 2020 00:12:11 -0400] rev 44800
tests: show that rebase --dry-run and --confirm wipeout uncommitted changes
It looks like the carnage is limited to rebasing something that is not an
ancestor of wdir(), as both of these abort in a preflight check for that case
with a dirty working directory.
Differential Revision: https://phab.mercurial-scm.org/D8488
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 06 Jun 2020 00:51:36 +0530] rev 44799
Added signature for changeset 065704cbdbdb
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 06 Jun 2020 00:51:28 +0530] rev 44798
Added tag 5.4.1 for changeset 065704cbdbdb
Manuel Jacob <me@manueljacob.de> [Fri, 05 Jun 2020 06:40:15 +0200] rev 44797
py3: update comment to account for Python 2 and Python 3 differences
Manuel Jacob <me@manueljacob.de> [Fri, 05 Jun 2020 07:20:52 +0200] rev 44796
py3: add warning about buffering behavior of pycompat.{stdout,stderr}
Manuel Jacob <me@manueljacob.de> [Fri, 05 Jun 2020 04:10:37 +0200] rev 44795
tests: fix indentation
Yuya Nishihara <yuya@tcha.org> [Tue, 02 Jun 2020 20:40:06 +0900] rev 44794
graft: fix --base value to be saved in state file
'True' just works because it is treated as an integer revision '1' and
only the truthiness of the basectx is important. If multiple source revisions
were supported with --base, the resumed graft operation would go wrong.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:38:53 +0200] rev 44793
flags: also test merge with executable bit removed
This might catch more bug in the future.
Differential Revision: https://phab.mercurial-scm.org/D8536
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:38:42 +0200] rev 44792
flags: also test the removal of the exec flag
Differential Revision: https://phab.mercurial-scm.org/D8535
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:38:31 +0200] rev 44791
flags: read flag from dirstate/disk for workingcopyctx (issue5743)
In 491855ea9d62, various piece of code are moved from committablectx to
workingctx. The reason given is "These read from the dirstate, so they shouldn't
be used in other subclasses."
At least for `flags` this change introduce a bug, because the value flags end up being
read from `_manifest` disregarding the actual state in the working copy (ie: on
disk). When merging exec flag change with renames, this means a new files (the
local content, renamed) is properly written on disk, with the right flags, but
the flags part is later ignored when actually reading flags during merge.
It is not clear to me why the `flags` function was moved, because the code does
not actually hit the dirstate (the reason given in the changeset description).
So I am moving it back to were it comes from and we use a simpler version of
that code (that hit the dirstate everytime) in workingcopyctx. This fix the last
know bug with merging rename and executable byte changes.
Other similar bug might be lurking in 491855ea9d62, but I have not investigated
them.
Differential Revision: https://phab.mercurial-scm.org/D8534
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:38:19 +0200] rev 44790
flags: actually merge flags in simplemerge
Since b86fc43e4b73, the local flag were blindly taken. This resulted in bug when
rename are involved. exec flag change are now properly merged (when merged from
the rename side).
Another bug is affecting this when merging from the side without the rename.
Differential Revision: https://phab.mercurial-scm.org/D8533
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:38:07 +0200] rev 44789
flags: add a test for merging exec flag change with rename and file change
Changing the file activate other code path that also have bugs… There are two
distinct bugs depending of which side of the merge you stand on. They both
leading to exec flag loss.
We add tests for both, the fix are coming in later changesets.
Differential Revision: https://phab.mercurial-scm.org/D8532
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:37:56 +0200] rev 44788
flags: account for flag change when tracking rename relevant to merge
There are some logic filtering rename to the one relevant to the merge. That
logic was oblivious of flag change, leading to exec flag being dropped when
merged with a renamed.
There are two others bugs affecting this scenario. This patch fix the was where
there is not modification involved except for the flag change. Fixes for the
other bug are coming in later changesets.
Differential Revision: https://phab.mercurial-scm.org/D8531
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:37:44 +0200] rev 44787
flags: also test merging a rename with and exec flag change
This case is currently buggy and was not tested. This is probably a quite old
regression. The next changeset fix this case. Move exec+rename related bug will
gain a test later.
To highlight the expected behavior the currently missing line are marked with (false !)
and the bad one with (true !)
note: we should probably gain explicit "test bool" for this usecases.
Differential Revision: https://phab.mercurial-scm.org/D8530
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 May 2020 20:37:33 +0200] rev 44786
flags: introduce explicit testing for merging change to exec flag
It turns out that we do not seems to test the simple case for merging exec flag
changes. More advanced case are test (merging exec flag without a common
ancestors, merging with a symlink, etc…) but not the basic.
We are about introduce various fixes to merging flag change across renames,
having the most basic case tested first seems useful.
note: We are only testing "adding" an exec flag here, not removing it. We
introduce basic test on stable and will consolidate them on default.
Differential Revision: https://phab.mercurial-scm.org/D8529
Charles Chamberlain <cchamberlain@janestreet.com> [Tue, 26 May 2020 11:14:07 -0400] rev 44785
graft-state: save --base in graft's state, fixing bug with graft --continue
Without this change, running graft --continue after grafting a merge commit using --base
(and encountering conflicts) will output "skipping ungraftable merge revision" even though
we specified a base in the initial graft command.
Graft's improve behaviour is reflected in test-graft.t.
Differential Revision: https://phab.mercurial-scm.org/D8578