Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 23 Oct 2019 17:18:16 +0200] rev 43320
packaging: ship only a single binary Debian package
We merge the mercurial and mercurial-common binary packages into a
single mercurial package. This is essentially to ease installation (and
upgrade) using a simple "dpkg -i" command. This also simplifies
debian/rules by removing arch (in)dependent cleanups during
installation.
We have the mercurial binary Breaks: and Replaces: mercurial-common so
that the latter will be removed upon upgrade.
Also note the change from "override_dh_install" to
"override_dh_auto_install" in debian/rules: this is because we do not
want "make install" to be run automatically as we need the
--install-layout=deb of "setup.py install" (otherwise, files would end
up in $DESTDIR/usr/local).
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 23 Oct 2019 17:18:57 +0200] rev 43319
packaging: avoid running bare "make install" in debian/rules
We change the "override_dh_install" target to "override_dh_auto_install"
in debian/rules (see dh_auto_install(1) for details). This is because we
do not want "make install" to be run automatically as we need the
--install-layout=deb of "setup.py install". Otherwise, files end up in
$DESTDIR/usr/local. At the moment, this is not a problem since files
installed in debian/tmp (the default location) are not packed into
binary packages (because there are debian/mercurial and
debian/mercurial-common directories). This is cleaner to avoid running
make more than needed.
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 23 Oct 2019 16:35:27 +0200] rev 43318
packaging: distinguish clean and build steps from install in Debian
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 23 Oct 2019 16:25:41 +0200] rev 43317
packaging: also move Debian .buildinfo file in output directory
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 21 Oct 2019 19:53:30 -0700] rev 43316
packaging: upgrade packages distributed with Windows installers
We like to use the latest versions of things.
I added pywin32-ctypes to the explicit list of packages so running
on !Windows will pull in the dependency.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 21 Oct 2019 19:28:23 -0700] rev 43315
automation: install Python 2.7.17, 3.7.5, and PyPy 7.2.0
These were all recently released and we should use them in
automation.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 21 Oct 2019 19:25:06 -0700] rev 43314
contrib: install Python 2.7.17 and 3.7.5 in Windows environment
These are the latest Python versions and we should use them.
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 11:48:59 +0200] rev 43313
packaging: use /usr/bin/python3 shebang for scripts in Debian
"hg" script is already correct because it is handled by setup.py but
"hg-ssh" will be rewritten by dh_python into "/usr/bin/python" which is
not wanted as we target Python 3. By passing --shebang=/usr/bin/python3
to dh_python3, we force shebangs to be set with this value.
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 11:32:54 +0200] rev 43312
packaging: fix path where .deb files are looked for
In builddeb script, the path where .deb files are looked for should
simply be $OUTPUTDIR since the previous instruction moves those files
there.
This fixes "make deb".
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 09:52:31 +0200] rev 43311
packaging: let Debian package depends on sensible-utils
This fixes the following Lintian error:
E: mercurial-common: missing-depends-on-sensible-utils etc/mercurial/hgrc.d/default-tools.rc
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 10:59:54 +0200] rev 43310
packaging: upgrade Debian packaging to build with Python 3
Also drop the explicit "Depends: python" as debhelper will add it.
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 11:26:41 +0200] rev 43309
py3: decode bytes before logging in run-tests.py
Avoids messages like "Found prerequisite b'diff' at b'/usr/bin/diff'"
under Python 3.
Augie Fackler <raf@durin42.com> [Mon, 21 Oct 2019 11:14:28 -0400] rev 43308
Added signature for changeset 59338f956109
Augie Fackler <raf@durin42.com> [Mon, 21 Oct 2019 11:14:27 -0400] rev 43307
Added tag 5.2rc0 for changeset 59338f956109
Augie Fackler <augie@google.com> [Mon, 21 Oct 2019 11:09:48 -0400] rev 43306
merge default into stable for 5.2 release
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 18 Oct 2019 11:18:41 -0400] rev 43305
grep: update docs to reflect new --all-files default
These docs no longer even mention the old weird behaviour that was
poorly understood and underutilised. I think this undocumentation
effectively deprecates it.
Sushil khanchi <sushilkhanchi97@gmail.com> [Sun, 06 Oct 2019 11:06:10 -0400] rev 43304
grep: enable all-files by default (BC)
This patch deprecates the `--all-files` flag and make the all-files
behaviour as default.
In test-grep.t, I removed '--all-files' from every command where it was
used, to reflect that all-files behaviour is default and there is no
change even after the removal.
And other changes in test files are because of changed behaviour.
Differential Revision: https://phab.mercurial-scm.org/D7000
.. bc::
`hg grep` now searches working copy file contents by default. We
recognize this is a significant change from past behavior, but
surveys of large bodies of users indicated nobody used (and almost
nobody understood) the previous no-flags behavior of `hg
grep`. The new behavior aligns with the behavior most users
expected (including hg's maintainers), which also happens to be
the behavior of `git grep`. Given that the old behavior was
confusing to the point of being unusable, we were comfortable
changing this behavior.
Denis Laxalde <denis@laxalde.org> [Mon, 21 Oct 2019 11:08:23 +0200] rev 43303
lfs: dedent documentation section about .hglfs file
The leading ".hglfs::" is interpreted as a macro in generated man page
and, as it is unknown, the whole section does not render. Also, having
the section marked as preformatted (::) makes it render verbatim in
HTML, which is not desired as the text contains formatting markers. So
we just dedent the section and remove the ".hglfs::" line. The example
file is still indented and rendered preformatted.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 17 Oct 2019 16:06:06 +0200] rev 43302
copies: simplify chain loop
This is a mostly gratuitous change to make the code more direct.
Differential Revision: https://phab.mercurial-scm.org/D7131
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 17 Oct 2019 02:17:24 +0200] rev 43301
sidedatacopies: only fetch information once for merge
Before this change, merge would result in reading the data from revlog twice.
With this change, we keep the information in memory until we encounter the other
parent.
When looking at pypy, I see about 1/3 of the changesets with copy information
being merge. Not doing duplicated fetch for them provide a significant speedup.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 0.767042 comb 0.760000 user 0.750000 sys 0.010000 (median of 11)
after: ! wall 0.671162 comb 0.670000 user 0.650000 sys 0.020000 (median of 13)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 1.170169 comb 1.170000 user 1.130000 sys 0.040000 (median of 10)
after: ! wall 1.030596 comb 1.040000 user 1.010000 sys 0.030000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.209846 comb 0.200000 user 0.200000 sys 0.000000 (median of 46)
after: ! wall 0.170981 comb 0.170000 user 0.170000 sys 0.000000 (median of 56)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013248 comb 0.010000 user 0.010000 sys 0.000000 (median of 223)
after: ! wall 0.013295 comb 0.020000 user 0.020000 sys 0.000000 (median of 222)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.001672 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001666 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8010)
after: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8007)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.168599 comb 0.160000 user 0.160000 sys 0.000000 (median of 58)
after: ! wall 0.133316 comb 0.140000 user 0.140000 sys 0.000000 (median of 73)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.036052 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)
after: ! wall 0.032558 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)
Differential Revision: https://phab.mercurial-scm.org/D7127
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Oct 2019 18:23:34 +0200] rev 43300
copies: move from a copy on branchpoint to a copy on write approach
Before this changes, any branch points results in a copy of the dictionary containing the
copy information. This can be very costly for branchy history with few rename
information. Instead, we take a "copy on write" approach. Copying the input data
only when we are about to update them.
In practice we where already doing the copying in half of these case (because
`_chain` makes a copy), so we don't add a significant cost here even in the
linear case. However the speed up in branchy case is very significant. Here are
some timing on the pypy repository.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 1.399863 comb 1.400000 user 1.370000 sys 0.030000 (median of 10)
after: ! wall 0.766453 comb 0.770000 user 0.750000 sys 0.020000 (median of 11)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 1.876748 comb 1.890000 user 1.870000 sys 0.020000 (median of 10)
after: ! wall 1.167223 comb 1.170000 user 1.150000 sys 0.020000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.242457 comb 0.240000 user 0.240000 sys 0.000000 (median of 39)
after: ! wall 0.211476 comb 0.210000 user 0.210000 sys 0.000000 (median of 45)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013193 comb 0.020000 user 0.020000 sys 0.000000 (median of 224)
after: ! wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (median of 222)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.001673 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001677 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8023)
after: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 7997)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.201898 comb 0.210000 user 0.200000 sys 0.010000 (median of 48)
after: ! wall 0.167415 comb 0.170000 user 0.160000 sys 0.010000 (median of 58)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.036820 comb 0.040000 user 0.040000 sys 0.000000 (median of 100)
after: ! wall 0.035797 comb 0.040000 user 0.040000 sys 0.000000 (median of 100)
The extra cost in the linear case can be reclaimed later with some extra logic.
Differential Revision: https://phab.mercurial-scm.org/D7124
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Oct 2019 18:35:14 +0200] rev 43299
copies: compute the exact set of revision to walk
This change make the code clearer by removing the revision queue. It comes
without very noticeable performance impact. However the simpler code will be
easier to update in later changesets.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 1.430082 comb 1.430000 user 1.390000 sys 0.040000 (median of 10)
after: ! wall 1.405192 comb 1.410000 user 1.390000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 1.971366 comb 1.970000 user 1.950000 sys 0.020000 (median of 10)
after: ! wall 1.892541 comb 1.890000 user 1.870000 sys 0.020000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.252594 comb 0.250000 user 0.240000 sys 0.010000 (median of 38)
after: ! wall 0.240075 comb 0.240000 user 0.240000 sys 0.000000 (median of 40)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013100 comb 0.010000 user 0.010000 sys 0.000000 (median of 226)
after: ! wall 0.013247 comb 0.010000 user 0.010000 sys 0.000000 (median of 223)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.001633 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001670 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000078 comb 0.000000 user 0.000000 sys 0.000000 (median of 11984)
after: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 7982)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.207093 comb 0.210000 user 0.210000 sys 0.000000 (median of 47)
after: ! wall 0.201551 comb 0.200000 user 0.200000 sys 0.000000 (median of 48)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.038462 comb 0.040000 user 0.040000 sys 0.000000 (median of 100)
after: ! wall 0.036578 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)
Differential Revision: https://phab.mercurial-scm.org/D7076
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Oct 2019 17:53:29 +0200] rev 43298
copies: add an explicit test using multiple roots
We explicitly add a test checking for copies of connected revision but involving
a different roots. This make sure the copies coming purely from the new roots
does not interfere with the merge.
Differential Revision: https://phab.mercurial-scm.org/D7075
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 18:16:02 -0400] rev 43297
sidedatacopies: fast path data fetching if revision has no sidedata
When using the side data mode, we know their won't be any copy information
sidedata. Skipping revision restoration give an important speed boost.
In the future, there will be other user of sidedata, reducing the efficiency of
this. We should consider adding a dedicated flag in revlog V2 to preserve this
optimisation. The current situation is good enough for now.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 2.401569 comb 2.400000 user 2.390000 sys 0.010000 (median of 10)
after: ! wall 1.429294 comb 1.430000 user 1.410000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 3.519140 comb 3.520000 user 3.470000 sys 0.050000 (median of 10)
after: ! wall 1.963332 comb 1.960000 user 1.960000 sys 0.000000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.593880 comb 0.600000 user 0.590000 sys 0.010000 (median of 15)
after: ! wall 0.251679 comb 0.250000 user 0.250000 sys 0.000000 (median of 38)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013414 comb 0.020000 user 0.020000 sys 0.000000 (median of 220)
after: ! wall 0.013222 comb 0.020000 user 0.020000 sys 0.000000 (median of 223)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.002711 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001631 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000077 comb 0.000000 user 0.000000 sys 0.000000 (median of 12208)
after: ! wall 0.000078 comb 0.000000 user 0.000000 sys 0.000000 (median of 12012)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.410067 comb 0.410000 user 0.410000 sys 0.000000 (median of 23)
after: ! wall 0.207786 comb 0.200000 user 0.200000 sys 0.000000 (median of 46)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.097004 comb 0.090000 user 0.090000 sys 0.000000 (median of 100)
after: ! wall 0.038495 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)
Differential Revision: https://phab.mercurial-scm.org/D7074
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Oct 2019 15:16:57 -0700] rev 43296
largefiles: use context manager for setting "lfstatus" on subrepos too
It's safe since we now reset to the previous value.
Differential Revision: https://phab.mercurial-scm.org/D7138
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Oct 2019 15:14:51 -0700] rev 43295
largefiles: reset "lfstatus" attribute to previous value in "finally"
We were resetting it to False, which means nesting of these overrides
would be incorrect.
Differential Revision: https://phab.mercurial-scm.org/D7137
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Oct 2019 14:40:50 -0700] rev 43294
largefiles: add context manager for setting/clearing "lfstatus" attribute
Differential Revision: https://phab.mercurial-scm.org/D7136
Denis Laxalde <denis.laxalde@logilab.fr> [Fri, 18 Oct 2019 16:51:26 +0200] rev 43293
py3: adjust expected traceback in test-hook.t
In Python 3, traceback.format_exception() displays the chain of
exceptions so we get extra results from our grep. Also,
ModuleNotFoundError is raised instead of ImportError from Python 3.6.
Denis Laxalde <denis.laxalde@logilab.fr> [Fri, 18 Oct 2019 16:38:37 +0200] rev 43292
tests: use non-reverse grep in traceback in test-hook.t
This will prepare for updating test output for Python 3.
Denis Laxalde <denis.laxalde@logilab.fr> [Fri, 18 Oct 2019 11:44:44 +0200] rev 43291
py3: add Python 3 exception output to test-hook.t
This is similar to, e.g., 3e9c6cef949b.
Denis Laxalde <denis.laxalde@logilab.fr> [Fri, 18 Oct 2019 12:12:56 +0200] rev 43290
py3: ajust abort message in test-hook.t
Yuya Nishihara <yuya@tcha.org> [Mon, 14 Oct 2019 16:15:19 +0900] rev 43289
rust-cpython: prepare for writing tests that require libpython
What I wanted is to disable the "cpython/extension-module<ver>" feature
while building tests executable, but that seems not doable. Instead,
this patch adds new features dedicated for tests.
The make rule is extracted so that we can easily run cargo tests.
Added a minimal test of ref-sharing as an example. More tests will follow.
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 10:21:34 -0400] rev 43288
rust-cpython: make inner functions and structs of ref_sharing private
Most of these methods were public because they had to be accessible from
macro-generated functions. Some "unsafe" can be removed since we can
guarantee the data consistency across non-public operations.
Yuya Nishihara <yuya@tcha.org> [Sat, 21 Sep 2019 17:05:01 +0900] rev 43287
rust-cpython: keep Python<'a> token in PyRefMut
This just clarifies that the GIL is obtained while PyRefMut is dereferenced,
so there's no need of extra acquire_gil() to drop the reference.
Yuya Nishihara <yuya@tcha.org> [Sat, 21 Sep 2019 17:15:50 +0900] rev 43286
rust-cpython: require GIL to borrow immutable reference from PySharedRefCell
Since the inner value may be leaked, we probably need GIL to guarantee that
there's no data race.
inner(py).borrow() is replaced with inner_shared(py).borrow(), which basically
means any PySharedRefCell data should be accessed through PySharedRef wrapper.
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 22:19:10 +0900] rev 43285
rust-cpython: make PyLeakedRef operations relatively safe
This patch encapsulates the access to the leaked reference to make most
leaked-ref operations safe. The only exception is leaked_ref.map(). I
couldn't figure out how to allow arbitrary map operation safely over an
unsafe static reference. See the docstring and inline comment for details.
Now leak_immutable() can be safely implemented as the PyLeakedRef owns
its inner data.
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 22:06:19 +0900] rev 43284
rust-cpython: put leaked reference in PyLeakedRef
The next patch will make PyLeakedRef manage the lifetime of the underlying
object. leak_handle.data.take() will be removed soon.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 17 Oct 2019 20:40:12 -0700] rev 43283
run-tests: make code coverage work on Python 3
This code path was obviously not tested on Python 3 because it
blew up in several places due to str/bytes mismatch.
For internal code, we normalize paths to bytes.
For code calling into `coverage`, we normalize paths to str,
which is what `coverage` seems to expect.
After this, `run-tests.py -H` works on Python 3!
Differential Revision: https://phab.mercurial-scm.org/D7133
Ian Moody <moz-ian@perix.co.uk> [Thu, 17 Oct 2019 22:40:24 +0100] rev 43282
py3: don't index into bytes in phabricator's _tokenize()
`phabread`ing a stack using `hg phabread :D1234` under py3 will currently die
with a KeyError because it will index into `b':D1234'` and return `58` instead
of `b':'` as a token.
Differential Revision: https://phab.mercurial-scm.org/D7129
Raphaël Gomès <rgomes@octobus.net> [Thu, 17 Oct 2019 14:20:11 +0200] rev 43281
rust-dirstate-status: use fast-path even with fsmonitor and sparse extensions
When I initially ran the tests on my series, there were test failures related
to those extensions. Now that the initial series has landed, I felt like going
back to those issues because people with performance issues will often want to
use fsmonitor and sparse.
Either because of flaky tests or because the series has changed so much, I
can't seem to reproduce these issues... let's widen the scope of the fast-path.
Differential Revision: https://phab.mercurial-scm.org/D7128
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 10:14:26 +0200] rev 43280
rust-dirstatemap: remove additional lookups in dirstatemap
We're copying this shortcut from the Python implementation, pretty standard
for this codebase.
Differential Revision: https://phab.mercurial-scm.org/D7117
Denis Laxalde <denis@laxalde.org> [Thu, 17 Oct 2019 21:37:21 +0200] rev 43279
tests: avoid chmod on windows in hgrc tests
Follow up on changeset d201a637c971 introducing this test, which fails
on Windows.
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 17 Oct 2019 13:34:21 +0200] rev 43278
py3: fix sorting of obsolete markers during push
This fixes similar errors as in 01e8eefd9434:
TypeError: '<' not supported between instances of 'NoneType' and 'tuple'
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 17 Oct 2019 13:30:58 +0200] rev 43277
exchange: extract a function to sort obsolete markers
Follows up on changeset 01e8eefd9434, several other occurrences of
sorted() need to be fixed. The _sortedmarkers() handles sorting
obsmarkers with a None value as last element on Python 3.
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 17 Oct 2019 12:10:42 +0200] rev 43276
py3: encode underlying error message during parse error of %include
Ian Moody <moz-ian@perix.co.uk> [Fri, 11 Oct 2019 18:08:50 +0100] rev 43275
convert: convert os.devnull to bytes before trying to join it with other bytes
Together with the previous commit relating to emailparser this gets
test-convert-tla.t passing under py3.
Differential Revision: https://phab.mercurial-scm.org/D7062
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 20:34:56 +0200] rev 43274
rust-dirstate-status: add call to rust-fast path for `dirstate.status`
The reasoning behind this patch is explained in the first patch of the series.
Differential Revision: https://phab.mercurial-scm.org/D7060
Raphaël Gomès <rgomes@octobus.net> [Tue, 08 Oct 2019 08:45:55 +0200] rev 43273
rust-dirstate-status: rust-cpython bindings for `dirstate.status`
The ref-sharing mechanism has improved, but its ergonomics still left a bit
to be desired, as expected.
Differential Revision: https://phab.mercurial-scm.org/D7059
Raphaël Gomès <rgomes@octobus.net> [Tue, 15 Oct 2019 21:26:56 +0200] rev 43272
rust-refsharing: add missing lifetime parameter in ref_sharing
Differential Revision: https://phab.mercurial-scm.org/D7110
Raphaël Gomès <rgomes@octobus.net> [Fri, 11 Oct 2019 13:39:57 +0200] rev 43271
rust-dirstate-status: add first Rust implementation of `dirstate.status`
Note: This patch also added the rayon crate as a Cargo dependency. It will
help us immensely in making Rust code parallel and easy to maintain. It is
a stable, well-known, and supported crate maintained by people on the Rust
team.
The current `dirstate.status` method has grown over the years through bug
reports and new features to the point where it got too big and too complex.
This series does not yet improve the logic, but adds a Rust fast-path to speed
up certain cases.
Tested on mozilla-try-2019-02-18 with zstd compression:
- `hg diff` on an empty working copy:
- c: 1.64(+-)0.04s
- rust+c before this change: 2.84(+-)0.1s
- rust+c: 849(+-)40ms
- `hg commit` when creating a file:
- c: 5.960s
- rust+c before this change: 5.828s
- rust+c: 4.668s
- `hg commit` when updating a file:
- c: 4.866s
- rust+c before this change: 4.371s
- rust+c: 3.855s
- `hg status -mard`
- c: 1.82(+-)0.04s
- rust+c before this change: 2.64(+-)0.1s
- rust+c: 896(+-)30ms
The numbers are clear: the current Rust `dirstatemap` implementation is super
slow, its performance needs to be addressed.
This will be done in a future series, immediately after this one, with the goal
of getting Rust to be at least to the speed of the Python + C implementation
in all cases before the 5.2 freeze. At worse, we gate dirstatemap to only be used
in those cases.
Cases where the fast-path is not executed:
- for commands that need ignore support (`status`, for example)
- if subrepos are found (should not be hard to add, but winter is coming)
- any other matcher than an `alwaysmatcher`, like patterns, etc.
- with extensions like `sparse` and `fsmonitor`
The next step after this is to rethink the logic to be closer to
Jane Street's Valentin Gatien-Baron's Rust fast-path which does a lot less
work when possible.
Differential Revision: https://phab.mercurial-scm.org/D7058
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 06 Oct 2019 20:18:54 +0300] rev 43270
share: unmark --relative as EXPERIMENTAL
Differential Revision: https://phab.mercurial-scm.org/D7001
Georges Racinet <georges.racinet@octobus.net> [Mon, 30 Sep 2019 16:31:53 -0400] rev 43269
rust-cpython: removed now useless py_set() conversion
In rust-cpython 0.3.0, HashSets implement the appropriate
ToPythonObject, we can therefore get rid of this hacky conversion.
There still remains an inefficiency in `MissingAncestors.bases()`:
we have to clone, because `to_py_object()` requires full ownership.
However:
- the only use case outside of unit tests used to be from
`setdiscovery.partialdiscovery` which is now fully implemented
in Rust.
- it's not worse than what `py_set()` used to do
Differential Revision: https://phab.mercurial-scm.org/D7120
Matt Harbison <matt_harbison@yahoo.com> [Tue, 15 Oct 2019 22:02:34 -0400] rev 43268
tests: stabilize test-eol-update on Windows
These appeared in the recent eol fix series. I'm not sure about the identity
mismatch warning- it seems like a general timing issue, because it happened on
the first run but not when run with `--loop`. The buildbot also flagged this.
Differential Revision: https://phab.mercurial-scm.org/D7112
Matt Harbison <matt_harbison@yahoo.com> [Tue, 15 Oct 2019 21:51:37 -0400] rev 43267
tests: opt into python3 when testing virtualenv installation
Otherwise it complains that Mercurial requires '~=2.7'. Since the existing
linux py3 test doesn't flag this, I'm assuming that virtualenv simply isn't
installed on that system.
Differential Revision: https://phab.mercurial-scm.org/D7111
Navaneeth Suresh <navaneeths1998@gmail.com> [Sun, 06 Oct 2019 15:17:34 +0530] rev 43266
grep: put --diff in the first line of usage
Differential Revision: https://phab.mercurial-scm.org/D7003
Joerg Sonnenberger <joerg@bec.de> [Thu, 12 Sep 2019 03:40:54 +0200] rev 43265
ui: option to preserve the progress bar
Some interface like the fast-import format of git are meant for pipe
processing. It can be still desirable to have progress bars in this
case, but clearing the bar after each output operation is not desirable.
Differential Revision: https://phab.mercurial-scm.org/D6843
Ian Moody <moz-ian@perix.co.uk> [Thu, 10 Oct 2019 22:11:39 +0100] rev 43264
phabricator: update hgmatcher to cope with the new data format
The new conduit format can't be matched by the existing matcher since it shifts
all the data into an urlencoded string of JSON, the order of which isn't stable
between runs. Instead detect JSON values of params and load them into python
dicts, which python will then naturally deep-equal compare.
Differential Revision: https://phab.mercurial-scm.org/D7055
Ian Moody <moz-ian@perix.co.uk> [Thu, 10 Oct 2019 22:05:28 +0100] rev 43263
phabricator: treat non-utf-8 text files as binary as phabricator requires
Phabricator can't cope with text files that are not UTF-8, so requires them to
be submitted as binary files instead. This has the unfortunate effect of
making them practically unreviewable in Phabricator since it will only display
the separate versions of the file in other views, not a diff. `phabread`ing
such submissions are similar, since it will just output the binary patch, but
`hg import` copes with it fine and `hg diff` afterwards will show the actual
changes. It is still a marked improvement over trying to submit them as text,
which just leads to corruption (Phabricator will either output ? or HTML
entities for non-UTF-8 characters, depending on context).
Running decode on the whole file like this seems slightly unfortunate, but I'm
not aware of a better way.
Needs to be done to p1() version as well to detect conversions to UTF-8.
Differential Revision: https://phab.mercurial-scm.org/D7054
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 18:23:06 +0100] rev 43262
phabricator: switch to the creatediff endpoint
This lets the extension submit binary files, as well as set branch info so that
it is exposed in the Phabricator interface.
Differential Revision: https://phab.mercurial-scm.org/D7053
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 18:21:01 +0100] rev 43261
phabricator: change conduit data format to match arcanist
The existing form isn't handled well by Phabricator for certain data types.
Differential Revision: https://phab.mercurial-scm.org/D7052
Ian Moody <moz-ian@perix.co.uk> [Thu, 10 Oct 2019 18:44:58 +0100] rev 43260
phabricator: remove tests and all recordings
The next commit is going to change the format of conduit API requests so none
of the VCR recordings will match and all the tests will fail.
Differential Revision: https://phab.mercurial-scm.org/D7051
Ian Moody <moz-ian@perix.co.uk> [Sat, 12 Oct 2019 14:41:59 +0100] rev 43259
phabricator: convert phabhunk and phabchange keys to bytes when finalising
I thought I had included this code already but I'd missed it out. One of the
disadvantages of attrs is that all the keys are unicode strings on py3, but we
need them to be byte strings.
Differential Revision: https://phab.mercurial-scm.org/D7066
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 17:21:26 +0100] rev 43258
phabricator: add addadded function
This is the most complicated part of the new code, and is responsible for
adding all added files, whether brand new, copied, or moved. This also
includes creating the phabchanges for the original files that have been moved
or copied from, since they might need their awayPaths and type updating if
multiple copies are involved.
Differential Revision: https://phab.mercurial-scm.org/D7050
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 17:53:47 -0400] rev 43257
sidedatacopies: directly fetch copies information from sidedata
When using the sidedata mode, we don't need a complicated and expensive
`context` object. Instead we directly fetch copies information from the
sidedata (through a changelogrevision object). More optimisations coming.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
filelog: ! wall 3.679613 comb 3.680000 user 3.580000 sys 0.100000 (median of 3)
base: ! wall 8.884369 comb 8.880000 user 8.850000 sys 0.030000 (median of 3)
before: ! wall 4.681985 comb 4.680000 user 4.640000 sys 0.040000 (median of 3)
after: ! wall 3.955894 comb 3.950000 user 3.940000 sys 0.010000 (median of 3)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
filelog: ! wall 0.003357 comb 0.010000 user 0.010000 sys 0.000000 (median of 781)
base: ! wall 12.398524 comb 12.400000 user 12.330000 sys 0.070000 (median of 3)
before: ! wall 6.459592 comb 6.470000 user 6.390000 sys 0.080000 (median of 3)
after: ! wall 5.505774 comb 5.500000 user 5.410000 sys 0.090000 (median of 3)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
filelog: ! wall 2.754687 comb 2.760000 user 2.650000 sys 0.110000 (median of 4)
base: ! wall 1.423166 comb 1.420000 user 1.400000 sys 0.020000 (median of 8)
before: ! wall 0.961048 comb 0.960000 user 0.940000 sys 0.020000 (median of 11)
after: ! wall 0.882950 comb 0.880000 user 0.880000 sys 0.000000 (median of 11)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
filelog: ! wall 1.552293 comb 1.550000 user 1.510000 sys 0.040000 (median of 6
base: ! wall 0.022662 comb 0.020000 user 0.020000 sys 0.000000 (median of 128)
before: ! wall 0.021649 comb 0.020000 user 0.020000 sys 0.000000 (median of 135)
after: ! wall 0.020951 comb 0.020000 user 0.020000 sys 0.000000 (median of 141)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
filelog: ! wall 1.500983 comb 1.500000 user 1.420000 sys 0.080000 (median of 7)
base: ! wall 0.006956 comb 0.010000 user 0.010000 sys 0.000000 (median of 392)
before: ! wall 0.004022 comb 0.000000 user 0.000000 sys 0.000000 (median of 735)
after: ! wall 0.003988 comb 0.000000 user 0.000000 sys 0.000000 (median of 736)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
filelog: ! wall 0.011745 comb 0.020000 user 0.020000 sys 0.000000 (median of 250)
base: ! wall 0.000156 comb 0.000000 user 0.000000 sys 0.000000 (median of 17180)
before: ! wall 0.000118 comb 0.000000 user 0.000000 sys 0.000000 (median of 19170)
after: ! wall 0.000097 comb 0.000000 user 0.000000 sys 0.000000 (median of 27276)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
filelog: ! wall 3.228230 comb 3.230000 user 3.110000 sys 0.120000 (median of 4)
base: ! wall 0.997640 comb 1.000000 user 0.980000 sys 0.020000 (median of 10)
before: ! wall 0.679500 comb 0.680000 user 0.680000 sys 0.000000 (median of 15)
after: ! wall 0.596779 comb 0.600000 user 0.600000 sys 0.000000 (median of 17)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
filelog: ! wall 1.052501 comb 1.060000 user 1.040000 sys 0.020000 (median of 10
base: ! wall 0.214519 comb 0.220000 user 0.220000 sys 0.000000 (median of 45)
before: ! wall 0.149675 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
after: ! wall 0.130786 comb 0.130000 user 0.130000 sys 0.000000 (median of 75)
Differential Revision: https://phab.mercurial-scm.org/D7072
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Oct 2019 18:42:03 +0200] rev 43256
copies: use an unfiltered repository for the changeset centric algorithm
Since the algorithm work form heads to ancestors, we don't need to check
filtering for anything but the two entries.
Using an unfiltered version is noticeably more efficient.
Some timing on the pypy repo:
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 2.717861 comb 2.720000 user 2.700000 sys 0.020000 (median of 10)
after: ! wall 2.582204 comb 2.580000 user 2.560000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 4.003146 comb 4.010000 user 3.970000 sys 0.040000 (median of 10)
after: ! wall 3.814613 comb 3.810000 user 3.760000 sys 0.050000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.704204 comb 0.700000 user 0.700000 sys 0.000000 (median of 13)
after: ! wall 0.657387 comb 0.650000 user 0.640000 sys 0.010000 (best of 14)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013493 comb 0.020000 user 0.020000 sys 0.000000 (median of 219)
after: ! wall 0.013523 comb 0.020000 user 0.020000 sys 0.000000 (median of 218)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.003017 comb 0.000000 user 0.000000 sys 0.000000 (median of 985)
after: ! wall 0.002876 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000073 comb 0.000000 user 0.000000 sys 0.000000 (median of 12672)
after: ! wall 0.000082 comb 0.000000 user 0.000000 sys 0.000000 (median of 11456)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.478061 comb 0.470000 user 0.470000 sys 0.000000 (median of 19)
after: ! wall 0.452420 comb 0.450000 user 0.450000 sys 0.000000 (median of 21)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.116015 comb 0.110000 user 0.110000 sys 0.000000 (median of 84)
after: ! wall 0.109153 comb 0.100000 user 0.100000 sys 0.000000 (median of 90)
Differential Revision: https://phab.mercurial-scm.org/D7123
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 17:42:01 -0400] rev 43255
copies: extract data extraction into a `revinfo` function
The function is build once at the beginning of the algorithm and used fetch
appropriate information for each revision.
This abstracts some implementation details from the main algorithm and will help
us to access the data more efficiently in future changesets.
Differential Revision: https://phab.mercurial-scm.org/D7070
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Oct 2019 17:49:30 +0200] rev 43254
copies: avoid instancing more changectx to access parent revisions
We just need to know the revision numbers of the parents, creating full context
is needlessly expensive.
This provide a small, but noticeable performance boost.
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 2.885636 comb 2.900000 user 2.870000 sys 0.030000 (median of 10)
after: ! wall 2.702270 comb 2.710000 user 2.690000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 4.298271 comb 4.290000 user 4.240000 sys 0.050000 (median of 10)
after: ! wall 3.976610 comb 3.970000 user 3.920000 sys 0.050000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.773397 comb 0.770000 user 0.770000 sys 0.000000 (median of 11)
after: ! wall 0.701634 comb 0.700000 user 0.700000 sys 0.000000 (median of 13)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013585 comb 0.010000 user 0.010000 sys 0.000000 (median of 217)
after: ! wall 0.013550 comb 0.010000 user 0.010000 sys 0.000000 (median of 218)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.003202 comb 0.000000 user 0.000000 sys 0.000000 (median of 929)
after: ! wall 0.002993 comb 0.010000 user 0.010000 sys 0.000000 (median of 992)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000077 comb 0.000000 user 0.000000 sys 0.000000 (median of 12060)
after: ! wall 0.000072 comb 0.000000 user 0.000000 sys 0.000000 (median of 12804)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.510614 comb 0.500000 user 0.500000 sys 0.000000 (median of 18)
after: ! wall 0.473681 comb 0.470000 user 0.470000 sys 0.000000 (median of 20)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.126552 comb 0.130000 user 0.130000 sys 0.000000 (median of 77)
after: ! wall 0.115240 comb 0.110000 user 0.110000 sys 0.000000 (median of 85)
Differential Revision: https://phab.mercurial-scm.org/D7122
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Oct 2019 09:52:05 +0200] rev 43253
copies: get copies information directly from _copies
Performance measurement does not show any significant performance movement. This
is not surprising since p1copies() code is `self._copies[0]`.
Differential Revision: https://phab.mercurial-scm.org/D7121
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 13:43:27 -0400] rev 43252
copies: simplify the handling of merges
Instead of stacking copies for both parent on the head, we move copies outside
of the heap into a dedicated dictionary. The two side of merge can we merged
sooner, making the algorithm simpler.
This simplicity reflect in the heap structure and speed up the execution for
copies involving a large amount of merges.
Here are timing for perfpathcopies of multiple revision pairs.
- filelog: timing using filelog (with the introrev condition dropped)
- base: this series base
- before: the parent of this changeset
- after: this changeset
revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
filelog: ! wall 3.679613 comb 3.680000 user 3.580000 sys 0.100000 (median of 3)
base: ! wall 8.884369 comb 8.880000 user 8.850000 sys 0.030000 (median of 3)
before: ! wall 8.443747 comb 8.420000 user 8.410000 sys 0.010000 (median of 3)
after: ! wall 4.697917 comb 4.690000 user 4.660000 sys 0.030000 (median of 3)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
filelog: ! wall 0.003357 comb 0.010000 user 0.010000 sys 0.000000 (median of 781)
base: ! wall 12.398524 comb 12.400000 user 12.330000 sys 0.070000 (median of 3)
before: ! wall 10.852593 comb 10.850000 user 10.800000 sys 0.050000 (median of 3)
after: ! wall 6.750832 comb 6.750000 user 6.640000 sys 0.110000 (median of 3)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
filelog: ! wall 2.754687 comb 2.760000 user 2.650000 sys 0.110000 (median of 4)
base: ! wall 1.423166 comb 1.420000 user 1.400000 sys 0.020000 (median of 8)
before: ! wall 1.068041 comb 1.060000 user 1.050000 sys 0.010000 (median of 10)
after: ! wall 1.045916 comb 1.050000 user 1.040000 sys 0.010000 (median of 10)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
filelog: ! wall 1.552293 comb 1.550000 user 1.510000 sys 0.040000 (median of 6
base: ! wall 0.022662 comb 0.020000 user 0.020000 sys 0.000000 (median of 128)
before: ! wall 0.021111 comb 0.020000 user 0.020000 sys 0.000000 (median of 139)
after: ! wall 0.021577 comb 0.020000 user 0.020000 sys 0.000000 (median of 138)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
filelog: ! wall 1.500983 comb 1.500000 user 1.420000 sys 0.080000 (median of 7)
base: ! wall 0.006956 comb 0.010000 user 0.010000 sys 0.000000 (median of 392)
before: ! wall 0.004356 comb 0.010000 user 0.010000 sys 0.000000 (median of 675)
after: ! wall 0.004329 comb 0.000000 user 0.000000 sys 0.000000 (median of 682)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
filelog: ! wall 0.011745 comb 0.020000 user 0.020000 sys 0.000000 (median of 250)
base: ! wall 0.000156 comb 0.000000 user 0.000000 sys 0.000000 (median of 17180)
before: ! wall 0.000100 comb 0.000000 user 0.000000 sys 0.000000 (median of 26912)
after: ! wall 0.000105 comb 0.000000 user 0.000000 sys 0.000000 (median of 25689)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
filelog: ! wall 3.228230 comb 3.230000 user 3.110000 sys 0.120000 (median of 4)
base: ! wall 0.997640 comb 1.000000 user 0.980000 sys 0.020000 (median of 10)
before: ! wall 0.778291 comb 0.780000 user 0.780000 sys 0.000000 (median of 13)
after: ! wall 0.706594 comb 0.710000 user 0.710000 sys 0.000000 (median of 15)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
filelog: ! wall 1.052501 comb 1.060000 user 1.040000 sys 0.020000 (median of 10
base: ! wall 0.214519 comb 0.220000 user 0.220000 sys 0.000000 (median of 45)
before: ! wall 0.160804 comb 0.160000 user 0.160000 sys 0.000000 (median of 62)
after: ! wall 0.163736 comb 0.160000 user 0.160000 sys 0.000000 (median of 60)
Differential Revision: https://phab.mercurial-scm.org/D7069
Raphaël Gomès <rgomes@octobus.net> [Mon, 07 Oct 2019 23:17:44 +0200] rev 43251
rust-utils: introduce a debug util to print the python stack trace
Differential Revision: https://phab.mercurial-scm.org/D7057
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 17:16:23 +0300] rev 43250
rust-cross-platform: remove `unimplemented!` to get compile-time errors
We should get a compile-time error that there is missing functionality for the
targeted platform instead of the program breaking at runtime.
Differential Revision: https://phab.mercurial-scm.org/D7107
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:18:34 -0700] rev 43249
widening: pass in matchers instead of patterns
This matches how it's done for the non-ellipsis case. The oldmatch is
not used yet, but it should be used eventually when widening no longer
resends manifests and files the client already has.
Differential Revision: https://phab.mercurial-scm.org/D7099
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:41:50 -0700] rev 43248
widening: trust user to give full "known" set
The new narrow_widen wire protocol command is supposed to return data
for exactly the revisions that the client requested (in order for it
to be strip-free). So we should not add ancestors the client didn't
ask for.
Differential Revision: https://phab.mercurial-scm.org/D7098
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:06:27 -0700] rev 43247
widening: remove "depth" argument since it's always None
Differential Revision: https://phab.mercurial-scm.org/D7097
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:03:10 -0700] rev 43246
widening: remove unused "heads" argument
Differential Revision: https://phab.mercurial-scm.org/D7096
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:36:55 -0700] rev 43245
widening: remove pointless code for second changegroup
The caller passes in common==heads, so we will never find any nodes to
visit in this code and there will therefore never be a second
changegroup emitted.
Differential Revision: https://phab.mercurial-scm.org/D7095
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:34:16 -0700] rev 43244
widening: remove always-true condition in widening code
The function is now specifically about widening, so we don't need to
check if we're widening.
Differential Revision: https://phab.mercurial-scm.org/D7094
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 21:51:07 -0700] rev 43243
narrow: drop server support for widening using the getbundle command (BC)
The client still supports widening using the getbundle, which we
(Google) still depend on a for a little while more (we've started the
migration to the new narrow_widen command, but we're not done yet).
Differential Revision: https://phab.mercurial-scm.org/D7093
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:18:35 -0700] rev 43242
widening: duplicate generateellipsesbundle2() for widening
The widening and the non-widening code are quite different. It will be
clearer to have them as sepearate functions. To start with, I've just
copied it exactly, so it's clearer over the next few patches how
they're different.
The new function should gradually become more similar to
bundle2.widen_bundle(), and should perhaps eventually be merged with
that function. However, I've left it in narrowbundle2.py for now since
it still depends on constants like _KILLNODESIGNAL there.
Differential Revision: https://phab.mercurial-scm.org/D7092
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 22:02:44 -0700] rev 43241
tests: show graph log before and after widening in more cases
I'm about to make some changes to which revisions get sent during
widening. Some more tests will make it clearer what changes. I've also
switched to graph log so we easily notice if the graph shape changes.
Differential Revision: https://phab.mercurial-scm.org/D7091
Augie Fackler <augie@google.com> [Wed, 19 Jun 2019 18:22:10 -0400] rev 43240
tracing: add a couple of trace points on obsolete and repoview
I was seeing weird stalls in some fast commands, and these trace
events help explain them.
Differential Revision: https://phab.mercurial-scm.org/D6550
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 11:05:32 -0400] rev 43239
dirstate: add some traces on listdir calls
I'm starting to spend some time trying to make `hg status` do less work, and
knowing what's happening in here and what's expensive is an important start.
Differential Revision: https://phab.mercurial-scm.org/D7109
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 11:06:04 -0400] rev 43238
cleanup: hgdemandimport.tracing accepts strings, not bytes
This does mean that the resulting traces will have some b'' goo in them on
Python 3, but I think that's a worthwile price to pay since it doesn't break
any tests.
Differential Revision: https://phab.mercurial-scm.org/D7108
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:48:27 -0400] rev 43237
tests: fix formatting test after move of black config
Differential Revision: https://phab.mercurial-scm.org/D7102
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 15:27:17 +0200] rev 43236
rust-dependencies: pin the dependencies of hg-core to prevent breakage
A new version of the `rand` crate broke two discovery tests. We should
upgrade our dependencies periodically manually.
Differential Revision: https://phab.mercurial-scm.org/D7115
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 15 Oct 2019 19:13:47 -0700] rev 43235
automation: install Python 3.8.0
Python 3.8 has been released. Let's install it instead of the
RC / development version.
Differential Revision: https://phab.mercurial-scm.org/D7114
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 14 Oct 2019 21:21:58 -0700] rev 43234
automation: schedule an EC2Launch run on next boot
Without this, launching EC2 instances constructed from the AMI
won't go through the normal EC2 instance launch machinery. This
missing machinery does important things like set up network
routes to use the instance metadata service and process any
UserData.
Since EC2Launch now runs on subsequent boots and UserData is
processed, we needed to make setting of UserData conditional
on bootstrapping mode.
Differential Revision: https://phab.mercurial-scm.org/D7113
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:54:05 -0400] rev 43233
dirs: give formatting oversight to clang-format
Differential Revision: https://phab.mercurial-scm.org/D7104
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:52:33 -0400] rev 43232
dirs: tag a struct as not being formattable
Differential Revision: https://phab.mercurial-scm.org/D7103
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:38:34 -0400] rev 43231
sidedatacopies: only store an entry if it has values
This will make for a shorter storage and help use to write faster code in simple
case. This change already provided a speed boost for copy tracing.
For example here is combined time of running copies tracing on a 6989 pairs of
revision on the pypy repos:
before: 771s
after: 631s - 18%
This also has a very positive impact on changelog size. For example here are the
number for the `00changelog.d` file of pypy.
before: 30449712 Bytes
after: 24973718 Bytes - 18%
This give an overall quite acceptable overhead for storing copies into the
changelog:
filelog-only: 23370586
sidedata: 24973718 Bytes + 7%
Differential Revision: https://phab.mercurial-scm.org/D7068
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:16:30 -0400] rev 43230
sidedatacopies: only read from copies when in this mode
If we know we expect data from sidedata, we read them from sidedata and nothing
else.
This avoid looking into extra for revision without sidedata entries. Such
revision will be introduced in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D7067
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 20:06:09 +0900] rev 43229
dirs: remove mutable string optimization at all
As far as I can see, the optimization trick has been dead since 42e89b87ca79
"dirs: speed up by storing number of direct children per dir". After
42e89b87ca79, the key variable is cleared to NULL at each iteration.
Denis Laxalde <denis.laxalde@logilab.fr> [Tue, 15 Oct 2019 12:14:44 +0200] rev 43228
py3: prevent comparison with None in curses histedit (issue6196)
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:30:25 -0700] rev 43227
fix: match patterns relative to root
I was surprised fixer patterns (used to determine which fixers to run)
are applies to the parent directory, not the repo root
directory. Danny Hooper (the author of the extension) seemed to agree
that it's better to apply them to the repo root, so that's what this
patch does.
Differential Revision: https://phab.mercurial-scm.org/D7101
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:30:20 -0700] rev 43226
tests: add test showing that fixer patterns are currently relative to $PWD
Differential Revision: https://phab.mercurial-scm.org/D7100
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:55:40 -0400] rev 43225
black: also ignore grey.py
Differential Revision: https://phab.mercurial-scm.org/D7090
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:54:49 -0400] rev 43224
black: use multiline exclude definition
Our excludes list is pretty complicated, so let's give ourselves some
readability. Black treats multiline exclude definitions as verbse
regular expressions, so we can split this up and make it easier to
consume.
Differential Revision: https://phab.mercurial-scm.org/D7089
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:52:33 -0400] rev 43223
black: move remaining config knobs into toml file
Differential Revision: https://phab.mercurial-scm.org/D7088
Augie Fackler <augie@google.com> [Mon, 14 Oct 2019 12:46:32 -0400] rev 43222
black: rename pyproject.toml to black.toml
Black won't read this automatically (you'll have to specify --config),
but having a pyproject.toml *at all* puts pip in PEP 517/518 mode
which breaks us for obscure reasons I don't understand. Rather than
waste a ton of time fighting with pip, let's just do this.
Differential Revision: https://phab.mercurial-scm.org/D7087
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 15:12:54 -0700] rev 43221
fix: remove a never-true check for unset pattern in Fixer.affects()
We don't create an instance of a Fixer if the pattern is None, so we
don't need to check that in affects().
Differential Revision: https://phab.mercurial-scm.org/D7086
Martin von Zweigbergk <martinvonz@google.com> [Fri, 26 Jul 2019 08:43:57 -0700] rev 43220
fix: warn when a fixer doesn't have a configured command
It seems we currently produce an empty command line and then decide to
not run it, but it seems better to skip that part too.
Differential Revision: https://phab.mercurial-scm.org/D7085
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:00:20 -0700] rev 43219
fix: make Fixer initialization more explicit for clarity
I found it quite confusing that Fixer accessed fields that seemed like
they didn't exist.
Differential Revision: https://phab.mercurial-scm.org/D7084
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:16:19 -0700] rev 43218
fix: don't pass in default value when looking up config
The defaults are now registered correctly so we don't need to pass in
the default value when we look up a config value.
Differential Revision: https://phab.mercurial-scm.org/D7083
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:13:55 -0700] rev 43217
fix: fix registration of config item defaults
Before this patch, because of the "(:<name>)?", all registered
patterns would match and the default value would not be the one we
thought we had registered (maybe it just took the default value for
the first match?). This didn't matter because we didn't care about the
default value; we used our own, intended default value in getfixers()
anyway.
We also have to look up each config item individually in order to not
get developer warnings.
Differential Revision: https://phab.mercurial-scm.org/D7082
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 12 Oct 2019 12:41:53 -0700] rev 43216
automation: capture additional exception when formatting
I started encountering this for some reason.
Differential Revision: https://phab.mercurial-scm.org/D7078
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 12 Oct 2019 11:36:26 -0700] rev 43215
contrib: update to latest Windows package versions
Let's keep modern.
Differential Revision: https://phab.mercurial-scm.org/D7077
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 21:47:05 +0900] rev 43214
rust-cpython: drop direct dependency on python(27|3)_sys
We no longer use it.
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:07:44 +0900] rev 43213
rust-cpython: leverage upstreamed py_capsule_fn!() macro
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:05:09 +0900] rev 43212
rust-cpython: bump cpython crates to 0.3
Unblocks py_capsule_fn!().
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:02:26 +0900] rev 43211
rust-cpython: turn inline comments into non-doc comments
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 17:01:10 +0900] rev 43210
rust-cpython: fix signature of make_dirstate_tuple()
Fortunately, the layout of PyObject {} is compatible with a C pointer, but
we shouldn't rely on that. This also fixes the handling of NULL pointer.
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 16:58:15 +0900] rev 43209
rust-cpython: mark capsule function as unsafe
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 16:55:17 +0900] rev 43208
rust-cpython: add wrapper around decapsule_make_dirstate_tuple()
There are a couple of safety issues. First, the returned function pointer
must be unsafe. Second, its return value must be a raw pointer (i.e.
python27_sys::PyObject), not a cpython::PyObject. The wrapper function will
address these issues.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:10:26 +0200] rev 43207
eol: don't fallback to use .hgeol from tip (BC)
If no .hgeol were found in the current working directory, eol would fallback to
use the one in tip. That could in some cases give very confusing or wrong
behaviour when it applied wrong filters.
It might be convenient to have plain 'clone' immediately apply 'native'
encoding patterns in the cloned repo. But it is wrong to assume that this
revision is tip, and even more wrong to also apply it when not cloning - for
example when updating between history revisions. The encoding should always
match the content of the current .hgeol . It should never use anything else.
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 01:42:24 +0200] rev 43206
eol: tweak test-eol-clone.t with better descriptions and logging
Expose impact of changes coming next ...
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:15:07 +0200] rev 43205
eol: fix update - don't use and apply removed .hgeol patterns
'hg up -C' to revisions with different .hgeol patterns could leave dirty
changes in the working directory. That could make deployment of new .hgeol
filters tricky: they would "occasionally" apply also in branches where they
shouldn't.
Fixed by dropping all old patterns before applying new ones.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:11:33 +0200] rev 43204
eol: cache needs update, also if it has same timestamp as the source
Ignoring same timestamp could (in theory?) cause changes to not be detected.
It might happen quite often that the cache is populated right after .hgeol has
been updated and they thus have the same time stamp second. But we want
correctness, and if it populates the cache so fast, then it can also not be a
big problem to run it again next time when the timestamp has moved on.
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 01:33:18 +0200] rev 43203
eol: update isbinary filter to work without compat wrapper
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 02:05:19 +0200] rev 43202
localrepo: fix variable binding in handling of old filters
The lambda was referencing oldfn in outer scope without binding the current
value. If oldfn function were reassigned before use, wrong filters could be
used.
Fixed by having oldfn as named parameter default value of the lambda.
Mads Kiilerich <mads@kiilerich.com> [Sun, 13 Oct 2019 14:40:00 +0200] rev 43201
localrepo: debug log of filter name when filtering through a function
Mads Kiilerich <mads@kiilerich.com> [Mon, 14 Oct 2019 00:09:25 +0200] rev 43200
eol: test-eol-update.t coverage around update --clean using filters ... badly
This will reveal problems and track their fixes.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 17:18:46 +0200] rev 43199
copies: drop the findlimit logic
We don't use the limit anymore so we should stop computing that limit.
I did not bother measuring the potential performance gain. I am assuming that
not running any code will be faster that doing some computation and not using
the result.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 03:49:33 +0200] rev 43198
pathcopies: give up any optimization based on `introrev`
Between 8a0136f69027 and d98fb3f42f33, we sped up the search for the
introduction revision during path copies. However, further checking show that
finding the introduction revision is still expensive and that we are better off
without it. So we simply drop it and only rely on the linkrev optimisation.
I ran `perfpathcopies` on 6989 pair of revision in the pypy
repository (`hg perfhelper-pathcopies`. The result is massively in favor of
dropping this condition. The result of the copy tracing are unchanged.
Attempt to use a smaller changes preserving linkrev usage were unsuccessful, it
can return wrong result. The following changesets broke test-mv-cp-st-diff.t
- if not f.isintroducedafter(limit):
+ if limit >= 0 and f.linkrev() < limit:
return None
Here are various numbers (before this changeset/after this changesets)
source destination before after saved-time ratio
worth cases e66f24650daf 695dfb0f493b 1.062843 1.246369 -0.183526 1.172675
c979853a3b6a 8d60fe293e79 1.036985 1.196414 -0.159429 1.153743
22349fa2fc33 fbb1c9fd86c0 0.879926 1.038682 -0.158756 1.180420
682b98f3e672 a4878080a536 0.909952 1.063801 -0.153849 1.169074
5adabc9b9848 920958a93997 0.993622 1.147452 -0.153830 1.154817
worse 1% dbfbfcf077e9 aea8f2fd3593 1.016595 1.082999 -0.066404 1.065320
worse 5% c95f1ced15f2 7d29d5e39734 0.453694 0.471156 -0.017462 1.038488
worse 10% 3e144ed1d5b7 2aef0e942480 0.035140 0.037535 -0.002395 1.068156
worse 25% 321fc60db035 801748ba582a 0.009267 0.009325 -0.000058 1.006259
median 2088ce763fc2 e6991321d78b 0.000665 0.000651 0.000014 0.978947
best 25% 915631a97de6 385b31354be6 0.040743 0.040363 0.000380 0.990673
best 10% ad495c36a765 19c10384d3e7 0.431658 0.411490 0.020168 0.953278
best 5% d13ae7d283ae 813c99f810ac 1.141404 1.075346 0.066058 0.942126
best 1% 81593cb4a496 99ae11866969 1.833297 0.063823 1.769474 0.034813
best cases c3b14617fbd7 743a0fcaa4eb 1101.811740 2.735970 1099.075770 0.002483
c3b14617fbd7 9ba6ab77fd29 1116.753953 2.800729 1113.953224 0.002508
058b99d6e81f 57e249b7a3ea 1246.128485 3.042762 1243.085723 0.002442
9a8c361aab49 0354a250d371 1253.111894 3.085796 1250.026098 0.002463
442dbbc53c68 3ec1002a818c 1261.786294 3.138607 1258.647687 0.002487
As one can see, the average case is not really impacted. However, the worth case
we get after this changeset are much better than the one we had before it. We
have 30 pairs where improvements are above 10 minutes.
This reflect in the combined time for all pairs
before: 26256s
after: 1300s (-95%)
If we remove these pathological 30 cases, we still see a significant improvements:
before: 1631s
after: 1245s (-24%)
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 04:28:23 -0400] rev 43197
perf: introduce a `--contains` flag to the `perfdirstate` command
The new flag benchmark a large amount of `filepath in dirstate` call. This will
be useful to compare the Python and Rust implementation of the dirstatemap.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 04:23:04 -0400] rev 43196
perf: introduce a `--iteration` to `perfdirstate`
This flag benchmark an iteration over all the file in the dirstate. This
will be useful to compare the Python and the Rust implementation of the
dirstate.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 13 Oct 2019 00:15:36 -0400] rev 43195
tests: open file in binary mode in test-upgrade-repo.t
Otherwise the file sizes printed in the output diverge from other platforms.
Differential Revision: https://phab.mercurial-scm.org/D7081
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Oct 2019 22:21:51 -0400] rev 43194
hghave: use a native string to invoke the `black` command
Windows can't create a process based on bytes. Apparently only the output test
in `matchoutput` wants bytes.
Differential Revision: https://phab.mercurial-scm.org/D7080
Matt Harbison <matt_harbison@yahoo.com> [Sat, 12 Oct 2019 22:22:17 -0400] rev 43193
py3: convert cwd to native string when running `fix`
This prevents a bunch of failures on Windows.
Differential Revision: https://phab.mercurial-scm.org/D7079
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 18:56:59 -0700] rev 43192
format: format commands.py, which recently regressed
# skip-blame just reformatting
Differential Revision: https://phab.mercurial-scm.org/D7064
Ian Moody <moz-ian@perix.co.uk> [Fri, 04 Oct 2019 19:36:21 +0100] rev 43191
py3: fix test-bad-extension expectations for py35
ModuleNotFoundError was introduced in 3.6, so the tests need to expect to see
ImportError with 3.5.
Differential Revision: https://phab.mercurial-scm.org/D7063
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 16:57:26 +0100] rev 43190
phabricator: add addremoved and addmodified functions
These are relatively simple so add them together. As the name indicates, these
add phabchange objects for removed and modified files to the phabdiff.
Differential Revision: https://phab.mercurial-scm.org/D7049
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:37:13 +0100] rev 43189
phabricator: add makebinary and addoldbinary functions
These populate the phabchange with the data for a binary file, much as maketext
does for text files.
Differential Revision: https://phab.mercurial-scm.org/D7048
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:16:47 +0100] rev 43188
phabricator: add the uploadfile function
This is needed to be able to submit binary files such as images in commits (and
also non-UTF-8 text files). One of the primary disadvantages of the current
use of createrawdiff is that Phabricator simply drops any binary diffs included
in the patch, but not the files, so if such a diff is then committed it
corrupts any binary files in it.
Differential Revision: https://phab.mercurial-scm.org/D7047
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 15:04:54 +0100] rev 43187
phabricator: add the uploadchunks function
Differential Revision: https://phab.mercurial-scm.org/D7046
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:54:46 +0100] rev 43186
phabricator: add the maketext function
This add the diff data for a text file to a phabchange.
Differential Revision: https://phab.mercurial-scm.org/D7045
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:53:03 +0100] rev 43185
phabricator: add the phabdiff data structure
This holds all the data about a commit, and is passed to the
differential.creatediff API.
Differential Revision: https://phab.mercurial-scm.org/D7044
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 14:08:03 +0100] rev 43184
phabricator: add the phabchange data structure
These store data about individual files in a commit.
Differential Revision: https://phab.mercurial-scm.org/D7043
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 13:55:04 +0100] rev 43183
phabricator: add the phabhunk data structure
These store the actual diff data (for UTF-8 text files anyway) and are
equivalent to hunks in a patch file.
Differential Revision: https://phab.mercurial-scm.org/D7042
Ian Moody <moz-ian@perix.co.uk> [Sun, 06 Oct 2019 13:50:32 +0100] rev 43182
phabricator: add the DiffChangeType and DiffFileType constants
These are used in Phabricator change objects.
There are more values but so far as I can tell we don't need them.
Differential Revision: https://phab.mercurial-scm.org/D7041
Rodrigo Damazio <rdamazio@google.com> [Sun, 06 Oct 2019 03:30:00 -0400] rev 43181
help: adding a help category to narrow and remotefilelog commands
Differential Revision: https://phab.mercurial-scm.org/D6998
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 17:02:31 +0900] rev 43180
rust-cpython: drop self.borrow_mut() in favor of PySharedRef wrapper
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:59:50 +0900] rev 43179
rust-cpython: drop self.leak_immutable() in favor of PySharedRef wrapper
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:50:48 +0900] rev 43178
rust-cpython: add safe wrapper representing shared data borrowed from PyObject
PySharedRef is a tempoary wrapper around PySharedRefCell. It provides safe
functions for each shared data. $shared_accessor implements a safe method
to construct PySharedRefCell.
This allows us to add more than once PySharedRefCell to a Python object.
Yuya Nishihara <yuya@tcha.org> [Sun, 08 Sep 2019 20:26:55 +0900] rev 43177
rust-cpython: move $leaked struct out of macro
It wasn't easy to hack the $leaked struct since errors in macro would
generate lots of compile errors. Let's make it a plain struct so we can
easily extend it.
PyLeakedRef keeps a more generic PyObject instead of the $name struct
since it no longer has to call any specific methods implemented by
the $name class. $leaked parameter in py_shared_iterator!() is kept
for future change.
Yuya Nishihara <yuya@tcha.org> [Sun, 15 Sep 2019 16:04:45 +0900] rev 43176
rust-cpython: store leaked reference to PySharedState in $leaked struct
I want to move it out of the macro, and allow multiple sharable objects
per PyObject.
Yuya Nishihara <yuya@tcha.org> [Tue, 17 Sep 2019 07:59:25 +0900] rev 43175
rust-cpython: mark PySharedState as Sync so &'PySharedState can be Send
The goal is to store &'static PySharedState in $leaked struct, which allows
us to move the $leaked struct out of the macro. Currently, it depends on
inner.$data_member(py), which can't be generalized.
PySharedState is Sync because any mutation or read operation is synchronized
by the Python GIL, py: Python<'a>, which should guarantee that &'PySharedState
can be sent to another thread.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Sep 2019 23:17:19 +0900] rev 43174
rust-cpython: move borrow_mut() to PySharedRefCell
PySharedRefCell() will host almost all py_shared public functions. This change
is the first step.
borrow_mut() can be safely implemented since PySharedRefCell knows its inner
object is managed by its own py_shared_state.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Sep 2019 23:01:51 +0900] rev 43173
rust-cpython: move py_shared_state to PySharedRefCell object
The goal of this series is to encapsulate more "py_shared" thingy and
reduce the size of the macro, which is hard to debug.
Since py_shared_state manages the borrowing state of the object owned by
PySharedRefCell, this change makes more sense. If a PyObject has more than
one data to be leaked into Python world, each PySharedState should incref
the parent PyObject, and keep track of the corresponding borrowing state.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:37:12 +0200] rev 43172
py3: decode username and password before SMTP login
smtplib.SMTP.login() requires str on Python 3.
For 'password', we only need to decode when value comes from config as
getpass() returns the correct type already.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:30:44 +0200] rev 43171
py3: use socket.makefile() instead of dropped smtplib.SSLFakeFile
The latter (undocumented internal) class got removed in Python 3.3. Use
socket.makefile() as suggested:
https://docs.python.org/3/whatsnew/3.3.html#porting-python-code
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:05:34 +0200] rev 43170
py3: call SMTP.docmd() with an str
Otherwise, this always returns (502, '5.5.2 Error: command not
recognized').
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 21:00:13 +0200] rev 43169
py3: call SMTP.has_extn() with an str
Passing a bytes on Python 3 always returns False, thus starttls is not
properly detected.
Denis Laxalde <denis@laxalde.org> [Thu, 10 Oct 2019 20:27:34 +0200] rev 43168
py3: fix sorting of obsolete markers when building bundle
Last item of marker tuple (parents) is either None or tuple. Comparison
thus fails on Python 3 with:
TypeError: '<' not supported between instances of 'tuple' and 'NoneType'
Fixing this by coercing None to the empty tuple when sorting markers in
exchange._getbundleobsmarkerpart().
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 04:34:58 +0200] rev 43167
sidedata: rename the configuration option to `exp-use-side-data`
We don't want setup configured to use the final version that would end using the
experimental one while using and older version.
Differential Revision: https://phab.mercurial-scm.org/D7040
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 16:02:47 +0200] rev 43166
py3: drop ui.flush() during interactive patch filtering
With previous changeset fixing line buffering on stdout, this is no
longer needed.
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 16:00:02 +0200] rev 43165
py3: keep stdout as defined by pycompat in procutil
According to https://docs.python.org/3/library/functions.html#open, it's
not possible to use 1 as buffering argument value in binary mode. This
is probably why line buffering does not work well in python3.
On the other hand, sys.stdout.buffer appears to be line-buffered already
on python3. So by not replacing it, there should be no behavior change.
This fixes buffering issue in "hg email" (confirmation prompt shown
before information to be confirmed).
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 21:21:16 -0400] rev 43164
notify: cast hash to bytes
This is needed to avoid a str/bytes mismatch when interpolating a
line or 2 later.
Differential Revision: https://phab.mercurial-scm.org/D7021
Kyle Lippincott <spectral@google.com> [Sat, 05 Oct 2019 13:39:35 -0700] rev 43163
push: support config option to require revs be specified when running push
Differential Revision: https://phab.mercurial-scm.org/D6989
Georges Racinet <georges.racinet@octobus.net> [Thu, 10 Oct 2019 11:33:33 +0200] rev 43162
py3-discovery: using plain str in stats dict
rust-cpython converts automatically from Rust strings to
the appropriate `str` for the target Python version.
Insisting on discovery stats dict keys to be bytes hence breaks
the process (this is spotted by test-setdiscovery.t).
Now that byteify-strings has been run on the entire
codebase, and the import transformer is not there any more,
the simplest fix is to make the keys plain str again.
Another possible fix would be to forcefully convert to bytes in
rust-cpython code, but that feels less natural, and would probably
have to be reverted down the road.
Differential Revision: https://phab.mercurial-scm.org/D7039
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 04:48:31 +0200] rev 43161
perf: fix `perfhelper-mergecopies` report of #changesets
Same as `perfhelper-pathcopies`. The previous computation `<base>::<target>` was
wrong, what we actually need is `::<target> - ::<base>`.
This is now fixed.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 18:39:20 -0400] rev 43160
perf: fix `perfhelper-pathcopies` report of #changesets
The previous computation `<base>::<target>` was wrong, what we actually need is
`::<target> - ::<base>`.
This is now fixed
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 12:22:15 +0200] rev 43159
py3: use integer division in curseschunkselector.printstring()
This fixes a crash when scrolling in curses UI when refresh() is called
when a float value (namely 'self.firstlineofpadtoprint', taking its
value indirectly from 'self.linesprintedtopadsofar').
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 12:20:23 +0200] rev 43158
crecord: drop duplicated set of firstlineofpadtoprint attribute
The attribute is already set a couple of lines above, in
curseschunkselector.__init__().
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 10:53:13 +0200] rev 43157
patchbomb: use mail.Generator alias for py2/py3 compat
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 10:48:57 +0200] rev 43156
py3: use email.generator.BytesGenerator in patch.split()
This fixes test-import.t on python3.
We add Generator alias in mail module to handle python2/python3
compatibility.
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 10 Oct 2019 10:03:01 +0200] rev 43155
py3: only flush before prompting during interactive patch filtering
Follows up on c9093ae8d6c4. It's enough to flush just before each
prompt.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 28 Aug 2019 17:45:18 -0700] rev 43154
py3: add a missing b'' prefix in test extension for chg
Differential Revision: https://phab.mercurial-scm.org/D7038
Augie Fackler <augie@google.com> [Wed, 09 Oct 2019 20:49:58 -0700] rev 43153
fuzz: new target to fuzz jsonescapeu8fast
This code just feels complicated enough we should go ahead and give it
a dedicated fuzzer: we've found bugs in similar things before.
Differential Revision: https://phab.mercurial-scm.org/D7034
Augie Fackler <augie@google.com> [Wed, 09 Oct 2019 20:49:39 -0700] rev 43152
fuzz: new fuzzer for fncache-related functions
Not all of these are strictly fncache-related, but they all have th
same signature and similar-enough behavior that we may as well fuzz
them together. No obvious bugs for once, but these felt like they were
just complicated enough to cover.
Differential Revision: https://phab.mercurial-scm.org/D7033
Augie Fackler <augie@google.com> [Wed, 09 Oct 2019 20:49:23 -0700] rev 43151
fuzz: exercise a little more revlog code
Differential Revision: https://phab.mercurial-scm.org/D7032
Augie Fackler <augie@google.com> [Wed, 09 Oct 2019 20:48:12 -0700] rev 43150
fuzz: new fuzzer for dirs.c
This found a six-year-old bug immediately, and then I put it through a
few CPU-days of time before sending it.
Differential Revision: https://phab.mercurial-scm.org/D7031
Augie Fackler <augie@google.com> [Tue, 08 Oct 2019 16:18:15 -0400] rev 43149
dirs: fix trivial over-read of input data
This code, introduced in 8c0a7eeda06d, was intentionally over-reading
an input string to avoid getting a shared string object for a one-byte
input. Unfortunately with an empty input (like in the case of a fuzzer
getting started) this was a trivial over-read and triggered an
AddressSanitizer failure.
I went out of my way to make sure the code still does the
copy-avoidance tricks. I don't think this change will cost us much
performance since the one-character strings should be cached
aggressively anyway.
Differential Revision: https://phab.mercurial-scm.org/D7030
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:52 -0400] rev 43148
sidedatacopies: deal with upgrading and downgrading to that format
This is quite useful to test this on real life data.
Differential Revision: https://phab.mercurial-scm.org/D6955
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43147
sidedatacopies: move various copies related function to the copies modules
We will need to access these logic form the copies module. So we move them from
their higher level module to the lower level `copies` module. We cannot use them
from their top level module as it would create cycles.
Differential Revision: https://phab.mercurial-scm.org/D6954
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Oct 2019 22:59:38 +0200] rev 43146
sidedatacopies: read rename information from sidedata
Repository using the new format now use changeset centric algorithm and read the
copies information from the changelog sidedata.
Differential Revision: https://phab.mercurial-scm.org/D6953
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 00:06:41 +0200] rev 43145
changelog: make copies related function return None or a valid value
With the previous code, existing but empty value were not "decoded", leading to
the method returning one of `None`, some valid value (`list` or `dict`) or
`b''`.
On a general basis, not explicitly checking for None is a source of bugs.
Having a clean return types will help the side-data copies code in future
changesets.
Differential Revision: https://phab.mercurial-scm.org/D7037
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 10 Oct 2019 00:01:40 +0200] rev 43144
test: fix zstd related output in pure tests
Differential Revision: https://phab.mercurial-scm.org/D7036
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43143
sidedatacopies: get and store sidedata in the changelogrevision object
The object provide a simple way to access changelog entry, we need it to also
bear the sidedata value.
Since the sidedata are retrieved at the same time as the revision, we can do
that without extra cost.
Differential Revision: https://phab.mercurial-scm.org/D6951
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43142
sidedatacopies: write copies information in sidedata when applicable
If the format of the repository indicate it stores copies information into
changeset's sidedata, then we actually write that information into sidedata at
commit time. It will be put to use in later changesets.
Currently, we store all field unconditionally, but that is likely to change in
the future for the sake of efficiency.
Differential Revision: https://phab.mercurial-scm.org/D6950
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43141
sidedatacopies: introduce a sidedata testcase for test-copies-in-changeset.t
For now it is equivalent to the filelog case, but introducing this early helps
make the comings changesets clearer.
Differential Revision: https://phab.mercurial-scm.org/D6949
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43140
sidedatacopies: introduce a sidedata testcase for test-copies-unrelated.t
For now it is equivalent to the filelog case, but introducing this early helps
make the comings changesets clearer.
Differential Revision: https://phab.mercurial-scm.org/D6948
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43139
sidedatacopies: introduce a sidedata testcase for test-copies.t
For now it is equivalent to the filelog case, but introducing this early helps
make the comings changesets clearer.
Differential Revision: https://phab.mercurial-scm.org/D6947
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43138
sidedatacopies: teach upgrade about the new requirement
The `debugformat` and `debugupgraderepo` command now detect the requirement.
(upgrade to and from are not currently possible).
Differential Revision: https://phab.mercurial-scm.org/D6946
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43137
sidedatacopies: add a new requirement for storing copies into sidedata
The end goal is to have changesets centric sidedata information stored into
changelog sidedata. This make it possible to use the changeset based copy
tracing algorithm on any repository without affecting hashes.
The actual implementation is coming. The feature is marked as experimental
(do not use in production) until we stabilise details about the format.
Differential Revision: https://phab.mercurial-scm.org/D6945
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43136
debugsidedata: small doc improvement
The command has a mode to actually show the sidedata, but it wasn't documented.
Differential Revision: https://phab.mercurial-scm.org/D6944
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43135
upgrade: allow for `sidedata` removal
If the side-date feature is removed, we need to drop them.
Differential Revision: https://phab.mercurial-scm.org/D6943
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43134
upgrade: allow upgrade to repository using sidedata
Repository can now be migrated to support sidedata. More requirements and
migration will be needed to actual side-data usage. This is a step in that
direction.
To test the feature, we leverage the test extension. It make sure the `update`
part of the side-data migration actually works.
Differential Revision: https://phab.mercurial-scm.org/D6942
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43133
revlog: add a way to control sidedata changes during revlog.clone
We introduce a new argument to pass a callable dealing with the side data logic.
Since the side data are usually associated to higher level logic, the revlog
code itself is unlikely to know that to do itself. As a result the higher level
upgrade code will be responsible to decide what needs to changes. The lower
level revlog.clone just get simple instructions to apply.
Differential Revision: https://phab.mercurial-scm.org/D6941
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43132
sidedata: use only changegroup3 if sidedata is in use
This is necessary to exchange the flags over the wire (and within bundles).
Differential Revision: https://phab.mercurial-scm.org/D6940
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 23:36:51 -0400] rev 43131
sidedata: apply basic but tight security around exchange
We don't currently have code to deal with exchange between repository using
sidedata and repository not using sidedata. Until we implement such code (eg:
dropping side data when pushing to a non-sidedata repo) we prevent the two kind
of repo to speak to each other. This is somewhere similar to what 'treemanifest'
does.
Note that sidedata exchange is broken unless one use changegroup v3 anyway. See
next changeset for details.
Differential Revision: https://phab.mercurial-scm.org/D6939
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 02:19:04 -0400] rev 43130
perf: use `setup` function in `perfdirstatewrite`
The command seems to pre-date the introduction of the `setup` support in timer.
We move the line that is obviously about benchmark setup in such `setup`
function.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:49:34 -0400] rev 43129
perf: document `perfdirstatewrite`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 02:12:15 -0400] rev 43128
perf: use `setup` function in `perfdirfoldmap`
The command seems to pre-date the introduction of the `setup` support in timer.
We move the line that is obviously about benchmark setup in such `setup`
function.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:49:18 -0400] rev 43127
perf: document `perfdirfoldmap`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 02:16:15 -0400] rev 43126
perf: use `setup` function in `perfdirstatefoldmap`
The command seems to pre-date the introduction of the `setup` support in timer.
We move the line that is obviously about benchmark setup in such `setup`
function.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:48:10 -0400] rev 43125
perf: document `perfdirstatefoldmap`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 02:08:41 -0400] rev 43124
perf: use `setup` function in `perfdirstatedirs`
The command seems to pre-date the introduction of the `setup` support in timer.
We move the line that is obviously about benchmark setup in such `setup`
function.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:47:35 -0400] rev 43123
perf: document `perfdirstatedirs`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:57:10 -0400] rev 43122
perf: use `setup` function in `perfdirstate`
The command seems to pre-date the introduction of the `setup` support in timer.
We move the line that is obviously about benchmark setup in such `setup`
function.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:46:57 -0400] rev 43121
perf: document `perfdirstate`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 08 Oct 2019 01:15:28 -0400] rev 43120
perf: document `perfstatus`
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 09 Oct 2019 15:24:14 +0200] rev 43119
py3: flush ui after each message in interactive patch filtering
Otherwise, actions from ui.write() are buffered and displayed at end of
interactive session.
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 09 Oct 2019 15:28:16 +0200] rev 43118
py3: decode prompt string before calling rawinput
Calling input() (aka pycompat.rawinput() on python3) with a byte string
displays a byte string in the console. E.g. in interactive commit, we
get: b"examine changes to '<file>'?\n(enter ? for help) [Ynesfdaq?]".
Similarly, "hg email" prompts are messed up.
We thus decode the prompt string before running rawinput().
Martin von Zweigbergk <martinvonz@google.com> [Tue, 08 Oct 2019 15:06:18 -0700] rev 43117
cleanup: join string literals that are already on one line
Thanks to Kyle for noticing this and for providing the regular
expression to run on the codebase.
This patch has been reviewed by the test suite and they approved of
it.
# skip-blame: fallout from mass reformatting
Differential Revision: https://phab.mercurial-scm.org/D7028
Martin von Zweigbergk <martinvonz@google.com> [Tue, 08 Oct 2019 15:15:37 -0700] rev 43116
debugcommands: add a few more writenoi18n()
Differential Revision: https://phab.mercurial-scm.org/D7027
Martin von Zweigbergk <martinvonz@google.com> [Sun, 06 Oct 2019 20:17:41 -0700] rev 43115
py3: delete b'' prefix from safehasattr arguments
Differential Revision: https://phab.mercurial-scm.org/D7029
Kyle Lippincott <spectral@google.com> [Tue, 08 Oct 2019 19:35:30 -0700] rev 43114
destutil: provide hint on rebase+merge for how to specify destination/rev
Without a destination specified, rebase and merge attempt to identify a good
candidate; this fails if there's too many heads, the heads have a bookmark, or
other reasons.
Complicating the issue, users may have specified -t, thinking it means target,
and are confused when the error message says that they need to specify an
explicit rev or explicit destination - they feel like they already have.
Differential Revision: https://phab.mercurial-scm.org/D7024
Kyle Lippincott <spectral@google.com> [Tue, 08 Oct 2019 10:40:36 -0700] rev 43113
hghave: document format for version feature checks as <name><vers>, no dots
I had a bit of a difficult time when attempting to fix a use of `py-38` when
trying to figure out what the correct way of specifying the feature was. By
having the strings 'py-3*' and 'py3.8' in the hghave.checkvers docstring, I
would have had a much easier time of it instead of having to trace exactly what
was happening here.
Differential Revision: https://phab.mercurial-scm.org/D7023
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Feb 2018 10:13:42 -0800] rev 43112
check-commit: allow foo_bar naming in functions
nameswithallthewordssmashedtogetherarehardtoread.
especiallyifenglishisnotyourprimarylanguage.
Let's align with the rest of the programming universe and
allow_the_use_of_underscores_in_names.
We took a hand poll at the 5.2 sprint regarding this change and
all but 1 person supported it. The person who didn't expressed
concerns around excessive API breakage if we mass renamed things.
But we're not planning to mass rename things for the sake of
renaming, so all should be well.
Differential Revision: https://phab.mercurial-scm.org/D2010
Augie Fackler <augie@google.com> [Tue, 08 Oct 2019 13:38:02 -0400] rev 43111
infinitepush: mark extension as likely to be deleted
.. bc::
The infinitepush extension is believed to be unused, and will be
deleted at the end of 2020 unless users contact
mercurial-devel@mercurial-scm.org.
Differential Revision: https://phab.mercurial-scm.org/D7022
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 10:34:21 -0400] rev 43110
tests: use range() in generate-churning-module.py
This is a test-only script. Performance on Python 2 for creating a
full list instead of a generator doesn't matter.
With this change, test-check-pyflakes.t passes on Python 3!
Differential Revision: https://phab.mercurial-scm.org/D7019
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 10:31:11 -0400] rev 43109
tests: allow warning about file
The code is protected by a block that makes it Python 2 only, so
use of file is acceptable.
Differential Revision: https://phab.mercurial-scm.org/D7018
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 10:28:32 -0400] rev 43108
lsprof: remove __main__ functionality
I'm pretty sure nobody uses this. I noticed it because Python 3
linting is complaining about execfile.
Differential Revision: https://phab.mercurial-scm.org/D7017
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 10:26:46 -0400] rev 43107
tests: use proper Python 3.8 feature
Fix typo introduced in 830eacef67f8.
Differential Revision: https://phab.mercurial-scm.org/D7016
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 07 Oct 2019 00:04:04 -0400] rev 43106
py3: finish porting iteritems() to pycompat and remove source transformer
This commit finishes porting .iteritems() to pycompat.iteritems()
for the mercurial package.
The translation of .iteritems() to .items() was the last conversion
performed by the source transformer. With the porting to pycompat
complete, we no longer have a need for the source transformer. So
the source transformer has been removed. Good riddance! The code
base is now compatible with Python 2 and Python 3.
For the record, as the person who introduced the source transformer,
it brings me joy to delete it. It accomplished its goal to facilitate
a port to Python 3 without overly burdening people on some painful
low-level differences between Python 2 and 3. It is unfortunate we
still have to wallpaper over many differences with the pycompat
shim. But it is what it is.
Differential Revision: https://phab.mercurial-scm.org/D7015
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 19:25:18 -0400] rev 43105
py3: define and use pycompat.iteritems() for hgext/
.iteritems() -> .items() is the last source transform being performed.
But it is also the most widely used.
This commit adds a pycompat.iteritems symbol and imports it in place
of .iteritems() for usage in hgext/. I chose to stop at just hgext/
because the patch will be large and it is an easy boundary to stop at
since we can disable source transformation on a per-package basis.
There are places where the type does implement items() and we could
call items() directly. However, this would require critical thought
and I thought it would be easier to just blindly change the code. We
know which call sites need to be audited in the future because they
have "pycompat.iteritems."
With this change, we no longer perform source transformation on
hgext!
Differential Revision: https://phab.mercurial-scm.org/D7014
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 17:59:15 -0400] rev 43104
py3: define and use pycompat.itervalues()
.itervalues() only exists on Python 2. Python 3's equivalent is
.values(). But we don't want to blindly use .values() everywhere
because on Python 2, it will create a list, which will have performance
implications.
This commit introduces pycompat.itervalues() which will call the appropriate
method on the passed object. We update all callers of obj.itervalues()
to pycompat.itervalues(obj) instead.
With this commit, the only source tranforming remaining is for
iteritems(). Victory is near...
Differential Revision: https://phab.mercurial-scm.org/D7013
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 17:45:05 -0400] rev 43103
py3: stop normalizing 2nd argument of *attr() to unicode
Now that we don't byteify strings, we can stop normalizing the 2nd
string argument to getattr() and remove explicit overrides we were
using in the code base.
We no longer use some helper functions in the source transformer,
so we remove those as well.
Differential Revision: https://phab.mercurial-scm.org/D7012
Yuya Nishihara <yuya@tcha.org> [Sun, 06 Oct 2019 14:58:41 -0400] rev 43102
log: populate keywords if specified in custom -Tjson(...) or -Tcbor(...)
To make things simple, early return for ui.quiet is disabled if the formatter
is templated and provides some datahint().
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 23:30:09 -0400] rev 43101
formatter: map -Tjson(...) and -Tcbor(...) to templater
Even though custom JSON output could be generated by using
--config template.json="{dict(...)|json}" ..., doing that is tedious because
of the trailing comma handling.
This patch introduces special syntax for JSON/CBOR formats. -Tjson(...) is
translated to template as if function-style template definition were
supported:
[templates]
json(...) = "{dict(...)|json}"
json(...):docheader = "[\n "
json(...):docfooter = "\n]\n"
json(...):separator = ",\n "
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 23:20:35 -0400] rev 43100
formatter: parse name of built-in formatter templates in standard way
This slightly makes it easier to add "-Tjson(...)" handling, which should
be enabled only if the template specifier doesn't look like a literal
template. In other words, it should be handled after "if '{' in tmpl".
This makes "log -Tpickle" and "log -Tdebug" abort, which I think is better
than just printing "picklepicklepickle...".
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 23:04:45 -0400] rev 43099
formatter: pass in template spec to templateformatter as argument
Prepare for the next patch, which will unify handling of the formatter
names and the template names.
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 15:47:38 -0400] rev 43098
templater: add public parseexpr() function to parse "-Tjson(...)"
Extracted _addparseerrorhint() to show nicer hint on error.
Denis Laxalde <denis@laxalde.org> [Sun, 06 Oct 2019 13:06:19 +0200] rev 43097
import: read X-Mercurial-Node email header to determine nodeid
This would be useful to import and obsolete patches sent using 'hg email
--plain', using evolve's --obsolete option of 'hg import'.
If email body contains Mercurial patch header ('# HG changeset patch'
block), nodeid parsed from X-Mercurial-Node header will still be
overridden by respective value found in body.
Yuya Nishihara <yuya@tcha.org> [Wed, 02 Oct 2019 07:35:22 +0900] rev 43096
patchbomb: use procutil.shellquote() instead of shlex to escape email address
This basically backs out 2cc453284d5c, and inserts procutil.shellquote()
instead. I don't care about Windows compatibility here, but shlex.quote()
can't handle byte strings on Python 3.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Oct 2019 11:52:58 -0400] rev 43095
formatting: introduce a `test-check-format-black.t` that enforce formatting
This should prevent use to drift away from the expect format.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Oct 2019 10:58:51 -0400] rev 43094
formatting: run black on all file again
Apparently, since the blackgnarok, we divergence from the expected formatting.
Formatted using::
grey.py -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/** - contrib/grey.py')
# skip-blame mass-reformatting only
# no-check-commit reformats foo_bar functions
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Oct 2019 11:51:34 -0400] rev 43093
formatting: make black --quiet in the example `hg fix` config
I do like cake, but I cannot have so many of them.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Oct 2019 11:48:34 -0400] rev 43092
formatting: remove the data-ogre from the config example
Without he final `-`, black ignore the stdin and just looks at file provided as
command line argument.
Since `hg fix` feeds the file content through stdin and does not pass file
argument, this meant black happily exited successful (all files passed as
argument were formatted) without any output. Fix picked this "no output" as the
new file content, deleting all previous content.
I appreciate the fact this effectively removed all buggy code in any files
processing that way, but this also ate all my data.
The example config is now fixed in that regards.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 17:27:51 -0400] rev 43091
py3: stop normalizing .encode()/.decode() arguments to unicode
Now that we don't byte transform string literals, we no longer need
this transform.
While we're here, we also drop some superfluous u'' prefix in existing
callers.
Differential Revision: https://phab.mercurial-scm.org/D7011
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 16:58:55 -0400] rev 43090
py3: manually import pycompat.delattr where it is needed
And with this change, we no longer need the auto-inserted import statement
in the source transformer, so it has been removed!
Differential Revision: https://phab.mercurial-scm.org/D7010
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 16:55:18 -0400] rev 43089
py3: manually import getattr where it is needed
The march continues.
Differential Revision: https://phab.mercurial-scm.org/D7009
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 16:39:13 -0400] rev 43088
py3: stop injecting pycompat.hasattr into modules
I only found a single user of this pattern, probably because we
use util.hasattr everywhere.
Differential Revision: https://phab.mercurial-scm.org/D7008
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 14:58:42 -0400] rev 43087
py3: manually import pycompat.setattr where it is needed
Continuing to eliminate the implicit import of symbols in the
Python 3 source transformer so we can eliminate it.
Differential Revision: https://phab.mercurial-scm.org/D7007
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 14:51:45 -0400] rev 43086
py3: stop implicitly importing unicode
We should be pycompat.unicode everywhere. It turns out we were doing this
everywhere except for one place in templatefilters!
Differential Revision: https://phab.mercurial-scm.org/D7006
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 13:28:56 -0400] rev 43085
py3: manually import pycompat.open into files that need it
We want to eliminate the source transformer. Currently it inserts
a `from mercurial.pycompat import ...` at the top of files to alias
some builtins.
This commit replaces the implicit import of `open` with an explicit
import on files that need it and changes the source transformer to
no longer import `open`.
As part of this, we needed to store an explicit local for `open` in
the Python 2 code path in `pycompat` so the import works. (Builtins
that are automatically in scope cannot be imported.)
Differential Revision: https://phab.mercurial-scm.org/D7005
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 13:17:19 -0400] rev 43084
import-checker: allow symbol imports from mercurial.pycompat
Currently, the source transformer inserts
`from mercurial.pycompat import delattr, getattr, hasattr, setattr, open, unicode`
to the top of every file. As part of getting rid of the source transformer,
we'll need to have source code call these wrappers directly. Rather than
rewrite all call sites to call pycompat.*, I think it makes sense to import
needed symbols via explicit imports. That requires loosening the import checker
to allow this.
Differential Revision: https://phab.mercurial-scm.org/D7004
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 14:13:03 -0700] rev 43083
contrib: add a fork of black (as "grey") that includes my changes
This is black with https://github.com/psf/black/pull/826 applied as of
today. The current git hash of black master is
d9e71a75ccfefa3d9156a64c03313a0d4ad981e5, and the hash of my commit is
dc1add6e94e212eff37bb3619e1422fb3c6d8dc8. In order to use this, you
need to install `black` (from github master) and `typed-ast` using
pip, preferably into python3, and then you can run `grey.py` with that
Python and you'll have my patched version of black, which is how we've
been formatting the codebase.
Once my PR is merged, I'll follow up by removing this fork and
updating instructions in the example config.
# no-check-commit bad style
Differential Revision: https://phab.mercurial-scm.org/D7002
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Oct 2019 09:58:21 -0400] rev 43082
rust-cpython: change license of ref_sharing.rs to MIT
Since we plan to upstream this feature, it's better to continue further
refactoring under the same license as rust-cpython crate.
According to the file history, copyright holders are:
- Raphaël Gomès <rgomes@octobus.net>
- Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>
- Yuya Nishihara <yuya@tcha.org>
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 10:20:17 -0400] rev 43081
contrib: fix check-code to be able to detect missing _() with bytestrings
Differential Revision: https://phab.mercurial-scm.org/D6997
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 10:51:16 -0400] rev 43080
cleanup: mark some ui.(status|note|warn|write) calls as not needing i18n
These used to be marked with no-op parens, but black removes those now
and this is more explicit.
# skip-blame: fallout from mass reformatting
Differential Revision: https://phab.mercurial-scm.org/D6996
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 06 Oct 2019 10:10:14 -0400] rev 43079
ui: define (write|status|warn|note)noi18n aliases
We currently use `write(('...'))` to suppress check-code warnings
about not using translated strings. However, when we run black, it
will strip the `((...))`. In order to placate black, we'll need to use
a different mechanism to pass untranslatable strings. This commit
introduces a `writenoi18n` alias (and friends) to `write` for that
purpose.
Differential Revision: https://phab.mercurial-scm.org/D6994
Augie Fackler <augie@google.com> [Sat, 04 May 2019 11:25:40 -0400] rev 43078
mangler: stop rewriting string constants to be bytes literals
We've rewritten everything, so we no longer require this step.
Differential Revision: https://phab.mercurial-scm.org/D6973
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 09:48:39 -0400] rev 43077
formatting: byteify all mercurial/ and hgext/ string literals
Done with
python3.7 contrib/byteify-strings.py -i $(hg files 'set:mercurial/**.py - mercurial/thirdparty/** + hgext/**.py - hgext/fsmonitor/pywatchman/** - mercurial/__init__.py')
black -l 80 -t py33 -S $(hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**')
# skip-blame mass-reformatting only
Differential Revision: https://phab.mercurial-scm.org/D6972
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 09:45:02 -0400] rev 43076
formatting: blacken the codebase
This is using my patch to black
(https://github.com/psf/black/pull/826) so we don't un-wrap collection
literals.
Done with:
hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**"' | xargs black -S
# skip-blame mass-reformatting only
# no-check-commit reformats foo_bar functions
Differential Revision: https://phab.mercurial-scm.org/D6971
Augie Fackler <augie@google.com> [Sat, 05 Oct 2019 10:29:34 -0400] rev 43075
style: run a patched black on a subset of mercurial
This applied black to the 20 smallest files in mercurial/:
ls -S1 mercurial/*.py | tail -n20 | xargs black --skip-string-normalization
Note that a few files failed to format, presumably due to a bug in my
patch. The intent is to be able to compare results to D5064 with
https://github.com/python/black/pull/826 applied to black.
I skipped string normalization on this patch for clarity - in reality
I think we'd want one pass without string normalization, followed by
another to normalize strings (which is basically replacing ' with "
globally.)
# skip-blame mass-reformatting only
Differential Revision: https://phab.mercurial-scm.org/D6342
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 04 Oct 2019 15:53:45 -0400] rev 43074
tests: conditionalize output for Python 3
It appears that the random.randint() behavior is different between Python 2
and Python 3. So make the test conditional on that.
This makes the test pass on Python 3.7 (and presumably other Python 3 versions).
Differential Revision: https://phab.mercurial-scm.org/D6964
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Oct 2019 17:01:02 -0400] rev 43073
hg: remove HGUNICODEPEDANTRY and RTUNICODEPEDANTRY
This was added in 73e4a02e6d23 and 89822d7a9d5f as a preliminary
way to better support Python 3. With the Python 3 port nearly done
and better procedures in place, we shouldn't need it.
Differential Revision: https://phab.mercurial-scm.org/D6991
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 04 Oct 2019 16:07:32 -0400] rev 43072
convert: use pycompat.fsencode()
This avoids a
`TypeError: Can't mix strings and bytes in path components`.
Differential Revision: https://phab.mercurial-scm.org/D6965
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Oct 2019 15:28:52 -0400] rev 43071
tests: make tarball output conditional on Python version
Python 3.8 changed the default tar file archive format:
https://docs.python.org/3.8/whatsnew/3.8.html#tarfile.
This commit teaches our tests about the new behavior.
Differential Revision: https://phab.mercurial-scm.org/D6986
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Oct 2019 16:23:00 -0400] rev 43070
tests: conditionalize test-run-tests.t for Python 3.8
It looks like Python 3.8 changed the pretty XML formatting slightly
to strip some whitespace. Let's teach our tests about that.
Differential Revision: https://phab.mercurial-scm.org/D6988
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Oct 2019 17:44:54 -0400] rev 43069
hgweb: use importlib.reload() if available
reload() was nuked in Python 3. We need to use importlib.reload()
instead.
But pyflakes isn't smart enough to detect our conditional usage, so
we allow this error.
Differential Revision: https://phab.mercurial-scm.org/D6992
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 Oct 2019 16:57:45 -0400] rev 43068
tests: allow xrange warning from perf.py
The use of xrange in this file is acceptable.
We need to make the output optional - and not conditional on
the Python version - because `pyflakes` could be executed by any
Python version and pyflakes behaves differently depending on the
Python version.
Differential Revision: https://phab.mercurial-scm.org/D6990
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Oct 2019 09:27:10 -0400] rev 43067
debugsidedata: fix verbose mode on python3
We need to be gently with python3 for it to display this binary data.
Differential Revision: https://phab.mercurial-scm.org/D6995
Denis Laxalde <denis@laxalde.org> [Sun, 06 Oct 2019 12:59:47 +0200] rev 43066
import: add debug messages when parsing data from patch header
Augie Fackler <augie@google.com> [Sun, 06 Oct 2019 06:55:49 -0700] rev 43065
examples: include a sample of how to use black with fix
It's commented out for now since my patch hasn't landed, but we can
uncomment it when that lands.
Differential Revision: https://phab.mercurial-scm.org/D6974