Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:38:34 -0400] rev 43231
sidedatacopies: only store an entry if it has values
This will make for a shorter storage and help use to write faster code in simple
case. This change already provided a speed boost for copy tracing.
For example here is combined time of running copies tracing on a 6989 pairs of
revision on the pypy repos:
before: 771s
after: 631s - 18%
This also has a very positive impact on changelog size. For example here are the
number for the `00changelog.d` file of pypy.
before: 30449712 Bytes
after: 24973718 Bytes - 18%
This give an overall quite acceptable overhead for storing copies into the
changelog:
filelog-only: 23370586
sidedata: 24973718 Bytes + 7%
Differential Revision: https://phab.mercurial-scm.org/D7068
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:16:30 -0400] rev 43230
sidedatacopies: only read from copies when in this mode
If we know we expect data from sidedata, we read them from sidedata and nothing
else.
This avoid looking into extra for revision without sidedata entries. Such
revision will be introduced in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D7067
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 20:06:09 +0900] rev 43229
dirs: remove mutable string optimization at all
As far as I can see, the optimization trick has been dead since 42e89b87ca79
"dirs: speed up by storing number of direct children per dir". After
42e89b87ca79, the key variable is cleared to NULL at each iteration.
Denis Laxalde <denis.laxalde@logilab.fr> [Tue, 15 Oct 2019 12:14:44 +0200] rev 43228
py3: prevent comparison with None in curses histedit (issue6196)
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:30:25 -0700] rev 43227
fix: match patterns relative to root
I was surprised fixer patterns (used to determine which fixers to run)
are applies to the parent directory, not the repo root
directory. Danny Hooper (the author of the extension) seemed to agree
that it's better to apply them to the repo root, so that's what this
patch does.
Differential Revision: https://phab.mercurial-scm.org/D7101
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:30:20 -0700] rev 43226
tests: add test showing that fixer patterns are currently relative to $PWD
Differential Revision: https://phab.mercurial-scm.org/D7100
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:55:40 -0400] rev 43225
black: also ignore grey.py
Differential Revision: https://phab.mercurial-scm.org/D7090
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:54:49 -0400] rev 43224
black: use multiline exclude definition
Our excludes list is pretty complicated, so let's give ourselves some
readability. Black treats multiline exclude definitions as verbse
regular expressions, so we can split this up and make it easier to
consume.
Differential Revision: https://phab.mercurial-scm.org/D7089
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:52:33 -0400] rev 43223
black: move remaining config knobs into toml file
Differential Revision: https://phab.mercurial-scm.org/D7088
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:46:32 -0400] rev 43222
black: rename pyproject.toml to black.toml
Black won't read this automatically (you'll have to specify --config),
but having a pyproject.toml *at all* puts pip in PEP 517/518 mode
which breaks us for obscure reasons I don't understand. Rather than
waste a ton of time fighting with pip, let's just do this.
Differential Revision: https://phab.mercurial-scm.org/D7087
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 15:12:54 -0700] rev 43221
fix: remove a never-true check for unset pattern in Fixer.affects()
We don't create an instance of a Fixer if the pattern is None, so we
don't need to check that in affects().
Differential Revision: https://phab.mercurial-scm.org/D7086
Martin von Zweigbergk <martinvonz@google.com> [Fri, 26 Jul 2019 08:43:57 -0700] rev 43220
fix: warn when a fixer doesn't have a configured command
It seems we currently produce an empty command line and then decide to
not run it, but it seems better to skip that part too.
Differential Revision: https://phab.mercurial-scm.org/D7085
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:00:20 -0700] rev 43219
fix: make Fixer initialization more explicit for clarity
I found it quite confusing that Fixer accessed fields that seemed like
they didn't exist.
Differential Revision: https://phab.mercurial-scm.org/D7084
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:16:19 -0700] rev 43218
fix: don't pass in default value when looking up config
The defaults are now registered correctly so we don't need to pass in
the default value when we look up a config value.
Differential Revision: https://phab.mercurial-scm.org/D7083
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:13:55 -0700] rev 43217
fix: fix registration of config item defaults
Before this patch, because of the "(:<name>)?", all registered
patterns would match and the default value would not be the one we
thought we had registered (maybe it just took the default value for
the first match?). This didn't matter because we didn't care about the
default value; we used our own, intended default value in getfixers()
anyway.
We also have to look up each config item individually in order to not
get developer warnings.
Differential Revision: https://phab.mercurial-scm.org/D7082
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 12 Oct 2019 12:41:53 -0700] rev 43216
automation: capture additional exception when formatting
I started encountering this for some reason.
Differential Revision: https://phab.mercurial-scm.org/D7078
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 12 Oct 2019 11:36:26 -0700] rev 43215
contrib: update to latest Windows package versions
Let's keep modern.
Differential Revision: https://phab.mercurial-scm.org/D7077
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 21:47:05 +0900] rev 43214
rust-cpython: drop direct dependency on python(27|3)_sys
We no longer use it.
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:07:44 +0900] rev 43213
rust-cpython: leverage upstreamed py_capsule_fn!() macro
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:05:09 +0900] rev 43212
rust-cpython: bump cpython crates to 0.3
Unblocks py_capsule_fn!().
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:02:26 +0900] rev 43211
rust-cpython: turn inline comments into non-doc comments
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:01:10 +0900] rev 43210
rust-cpython: fix signature of make_dirstate_tuple()
Fortunately, the layout of PyObject {} is compatible with a C pointer, but
we shouldn't rely on that. This also fixes the handling of NULL pointer.
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 16:58:15 +0900] rev 43209
rust-cpython: mark capsule function as unsafe
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 16:55:17 +0900] rev 43208
rust-cpython: add wrapper around decapsule_make_dirstate_tuple()
There are a couple of safety issues. First, the returned function pointer
must be unsafe. Second, its return value must be a raw pointer (i.e.
python27_sys::PyObject), not a cpython::PyObject. The wrapper function will
address these issues.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:10:26 +0200] rev 43207
eol: don't fallback to use .hgeol from tip (BC)
If no .hgeol were found in the current working directory, eol would fallback to
use the one in tip. That could in some cases give very confusing or wrong
behaviour when it applied wrong filters.
It might be convenient to have plain 'clone' immediately apply 'native'
encoding patterns in the cloned repo. But it is wrong to assume that this
revision is tip, and even more wrong to also apply it when not cloning - for
example when updating between history revisions. The encoding should always
match the content of the current .hgeol . It should never use anything else.
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 01:42:24 +0200] rev 43206
eol: tweak test-eol-clone.t with better descriptions and logging
Expose impact of changes coming next ...
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:15:07 +0200] rev 43205
eol: fix update - don't use and apply removed .hgeol patterns
'hg up -C' to revisions with different .hgeol patterns could leave dirty
changes in the working directory. That could make deployment of new .hgeol
filters tricky: they would "occasionally" apply also in branches where they
shouldn't.
Fixed by dropping all old patterns before applying new ones.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:11:33 +0200] rev 43204
eol: cache needs update, also if it has same timestamp as the source
Ignoring same timestamp could (in theory?) cause changes to not be detected.
It might happen quite often that the cache is populated right after .hgeol has
been updated and they thus have the same time stamp second. But we want
correctness, and if it populates the cache so fast, then it can also not be a
big problem to run it again next time when the timestamp has moved on.
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 01:33:18 +0200] rev 43203
eol: update isbinary filter to work without compat wrapper
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:05:19 +0200] rev 43202
localrepo: fix variable binding in handling of old filters
The lambda was referencing oldfn in outer scope without binding the current
value. If oldfn function were reassigned before use, wrong filters could be
used.
Fixed by having oldfn as named parameter default value of the lambda.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 14:40:00 +0200] rev 43201
localrepo: debug log of filter name when filtering through a function
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 00:09:25 +0200] rev 43200
eol: test-eol-update.t coverage around update --clean using filters ... badly
This will reveal problems and track their fixes.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 17:18:46 +0200] rev 43199
copies: drop the findlimit logic
We don't use the limit anymore so we should stop computing that limit.
I did not bother measuring the potential performance gain. I am assuming that
not running any code will be faster that doing some computation and not using
the result.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 03:49:33 +0200] rev 43198
pathcopies: give up any optimization based on `introrev`
Between 8a0136f69027 and d98fb3f42f33, we sped up the search for the
introduction revision during path copies. However, further checking show that
finding the introduction revision is still expensive and that we are better off
without it. So we simply drop it and only rely on the linkrev optimisation.
I ran `perfpathcopies` on 6989 pair of revision in the pypy
repository (`hg perfhelper-pathcopies`. The result is massively in favor of
dropping this condition. The result of the copy tracing are unchanged.
Attempt to use a smaller changes preserving linkrev usage were unsuccessful, it
can return wrong result. The following changesets broke test-mv-cp-st-diff.t
- if not f.isintroducedafter(limit):
+ if limit >= 0 and f.linkrev() < limit:
return None
Here are various numbers (before this changeset/after this changesets)
source destination before after saved-time ratio
worth cases e66f24650daf 695dfb0f493b 1.062843 1.246369 -0.183526 1.172675
c979853a3b6a 8d60fe293e79 1.036985 1.196414 -0.159429 1.153743
22349fa2fc33 fbb1c9fd86c0 0.879926 1.038682 -0.158756 1.180420
682b98f3e672 a4878080a536 0.909952 1.063801 -0.153849 1.169074
5adabc9b9848 920958a93997 0.993622 1.147452 -0.153830 1.154817
worse 1% dbfbfcf077e9 aea8f2fd3593 1.016595 1.082999 -0.066404 1.065320
worse 5% c95f1ced15f2 7d29d5e39734 0.453694 0.471156 -0.017462 1.038488
worse 10% 3e144ed1d5b7 2aef0e942480 0.035140 0.037535 -0.002395 1.068156
worse 25% 321fc60db035 801748ba582a 0.009267 0.009325 -0.000058 1.006259
median 2088ce763fc2 e6991321d78b 0.000665 0.000651 0.000014 0.978947
best 25% 915631a97de6 385b31354be6 0.040743 0.040363 0.000380 0.990673
best 10% ad495c36a765 19c10384d3e7 0.431658 0.411490 0.020168 0.953278
best 5% d13ae7d283ae 813c99f810ac 1.141404 1.075346 0.066058 0.942126
best 1% 81593cb4a496 99ae11866969 1.833297 0.063823 1.769474 0.034813
best cases c3b14617fbd7 743a0fcaa4eb 1101.811740 2.735970 1099.075770 0.002483
c3b14617fbd7 9ba6ab77fd29 1116.753953 2.800729 1113.953224 0.002508
058b99d6e81f 57e249b7a3ea 1246.128485 3.042762 1243.085723 0.002442
9a8c361aab49 0354a250d371 1253.111894 3.085796 1250.026098 0.002463
442dbbc53c68 3ec1002a818c 1261.786294 3.138607 1258.647687 0.002487
As one can see, the average case is not really impacted. However, the worth case
we get after this changeset are much better than the one we had before it. We
have 30 pairs where improvements are above 10 minutes.
This reflect in the combined time for all pairs
before: 26256s
after: 1300s (-95%)
If we remove these pathological 30 cases, we still see a significant improvements:
before: 1631s
after: 1245s (-24%)
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 04:28:23 -0400] rev 43197
perf: introduce a `--contains` flag to the `perfdirstate` command
The new flag benchmark a large amount of `filepath in dirstate` call. This will
be useful to compare the Python and Rust implementation of the dirstatemap.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 04:23:04 -0400] rev 43196
perf: introduce a `--iteration` to `perfdirstate`
This flag benchmark an iteration over all the file in the dirstate. This
will be useful to compare the Python and the Rust implementation of the
dirstate.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 13 Oct 2019 00:15:36 -0400] rev 43195
tests: open file in binary mode in test-upgrade-repo.t
Otherwise the file sizes printed in the output diverge from other platforms.
Differential Revision: https://phab.mercurial-scm.org/D7081
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Oct 2019 22:21:51 -0400] rev 43194
hghave: use a native string to invoke the `black` command
Windows can't create a process based on bytes. Apparently only the output test
in `matchoutput` wants bytes.
Differential Revision: https://phab.mercurial-scm.org/D7080
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Oct 2019 22:22:17 -0400] rev 43193
py3: convert cwd to native string when running `fix`
This prevents a bunch of failures on Windows.
Differential Revision: https://phab.mercurial-scm.org/D7079
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 18:56:59 -0700] rev 43192
format: format commands.py, which recently regressed
# skip-blame just reformatting
Differential Revision: https://phab.mercurial-scm.org/D7064
Ian Moody <moz-ian@perix.co.uk> [Fri, 04 Oct 2019 19:36:21 +0100] rev 43191
py3: fix test-bad-extension expectations for py35
ModuleNotFoundError was introduced in 3.6, so the tests need to expect to see
ImportError with 3.5.
Differential Revision: https://phab.mercurial-scm.org/D7063
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 16:57:26 +0100] rev 43190
phabricator: add addremoved and addmodified functions
These are relatively simple so add them together. As the name indicates, these
add phabchange objects for removed and modified files to the phabdiff.
Differential Revision: https://phab.mercurial-scm.org/D7049
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:37:13 +0100] rev 43189
phabricator: add makebinary and addoldbinary functions
These populate the phabchange with the data for a binary file, much as maketext
does for text files.
Differential Revision: https://phab.mercurial-scm.org/D7048
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:16:47 +0100] rev 43188
phabricator: add the uploadfile function
This is needed to be able to submit binary files such as images in commits (and
also non-UTF-8 text files). One of the primary disadvantages of the current
use of createrawdiff is that Phabricator simply drops any binary diffs included
in the patch, but not the files, so if such a diff is then committed it
corrupts any binary files in it.
Differential Revision: https://phab.mercurial-scm.org/D7047
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:04:54 +0100] rev 43187
phabricator: add the uploadchunks function
Differential Revision: https://phab.mercurial-scm.org/D7046
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:54:46 +0100] rev 43186
phabricator: add the maketext function
This add the diff data for a text file to a phabchange.
Differential Revision: https://phab.mercurial-scm.org/D7045
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:53:03 +0100] rev 43185
phabricator: add the phabdiff data structure
This holds all the data about a commit, and is passed to the
differential.creatediff API.
Differential Revision: https://phab.mercurial-scm.org/D7044
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:08:03 +0100] rev 43184
phabricator: add the phabchange data structure
These store data about individual files in a commit.
Differential Revision: https://phab.mercurial-scm.org/D7043
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 13:55:04 +0100] rev 43183
phabricator: add the phabhunk data structure
These store the actual diff data (for UTF-8 text files anyway) and are
equivalent to hunks in a patch file.
Differential Revision: https://phab.mercurial-scm.org/D7042
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 13:50:32 +0100] rev 43182
phabricator: add the DiffChangeType and DiffFileType constants
These are used in Phabricator change objects.
There are more values but so far as I can tell we don't need them.
Differential Revision: https://phab.mercurial-scm.org/D7041
Rodrigo Damazio <rdamazio@google.com> [Sun, 06 Oct 2019 03:30:00 -0400] rev 43181
help: adding a help category to narrow and remotefilelog commands
Differential Revision: https://phab.mercurial-scm.org/D6998
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 17:02:31 +0900] rev 43180
rust-cpython: drop self.borrow_mut() in favor of PySharedRef wrapper
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:59:50 +0900] rev 43179
rust-cpython: drop self.leak_immutable() in favor of PySharedRef wrapper
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:50:48 +0900] rev 43178
rust-cpython: add safe wrapper representing shared data borrowed from PyObject
PySharedRef is a tempoary wrapper around PySharedRefCell. It provides safe
functions for each shared data. $shared_accessor implements a safe method
to construct PySharedRefCell.
This allows us to add more than once PySharedRefCell to a Python object.
Yuya Nishihara <yuya@tcha.org> [Sun, 08 Sep 2019 20:26:55 +0900] rev 43177
rust-cpython: move $leaked struct out of macro
It wasn't easy to hack the $leaked struct since errors in macro would
generate lots of compile errors. Let's make it a plain struct so we can
easily extend it.
PyLeakedRef keeps a more generic PyObject instead of the $name struct
since it no longer has to call any specific methods implemented by
the $name class. $leaked parameter in py_shared_iterator!() is kept
for future change.
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:04:45 +0900] rev 43176
rust-cpython: store leaked reference to PySharedState in $leaked struct
I want to move it out of the macro, and allow multiple sharable objects
per PyObject.
Yuya Nishihara <yuya@tcha.org> [Tue, 17 Sep 2019 07:59:25 +0900] rev 43175
rust-cpython: mark PySharedState as Sync so &'PySharedState can be Send
The goal is to store &'static PySharedState in $leaked struct, which allows
us to move the $leaked struct out of the macro. Currently, it depends on
inner.$data_member(py), which can't be generalized.
PySharedState is Sync because any mutation or read operation is synchronized
by the Python GIL, py: Python<'a>, which should guarantee that &'PySharedState
can be sent to another thread.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Sep 2019 23:17:19 +0900] rev 43174
rust-cpython: move borrow_mut() to PySharedRefCell
PySharedRefCell() will host almost all py_shared public functions. This change
is the first step.
borrow_mut() can be safely implemented since PySharedRefCell knows its inner
object is managed by its own py_shared_state.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Sep 2019 23:01:51 +0900] rev 43173
rust-cpython: move py_shared_state to PySharedRefCell object
The goal of this series is to encapsulate more "py_shared" thingy and
reduce the size of the macro, which is hard to debug.
Since py_shared_state manages the borrowing state of the object owned by
PySharedRefCell, this change makes more sense. If a PyObject has more than
one data to be leaked into Python world, each PySharedState should incref
the parent PyObject, and keep track of the corresponding borrowing state.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:37:12 +0200] rev 43172
py3: decode username and password before SMTP login
smtplib.SMTP.login() requires str on Python 3.
For 'password', we only need to decode when value comes from config as
getpass() returns the correct type already.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:30:44 +0200] rev 43171
py3: use socket.makefile() instead of dropped smtplib.SSLFakeFile
The latter (undocumented internal) class got removed in Python 3.3. Use
socket.makefile() as suggested:
https://docs.python.org/3/whatsnew/3.3.html#porting-python-code
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:05:34 +0200] rev 43170
py3: call SMTP.docmd() with an str
Otherwise, this always returns (502, '5.5.2 Error: command not
recognized').
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:00:13 +0200] rev 43169
py3: call SMTP.has_extn() with an str
Passing a bytes on Python 3 always returns False, thus starttls is not
properly detected.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 20:27:34 +0200] rev 43168
py3: fix sorting of obsolete markers when building bundle
Last item of marker tuple (parents) is either None or tuple. Comparison
thus fails on Python 3 with:
TypeError: '<' not supported between instances of 'tuple' and 'NoneType'
Fixing this by coercing None to the empty tuple when sorting markers in
exchange._getbundleobsmarkerpart().