Gregory Szorc <gregory.szorc@gmail.com> [Fri, 08 Mar 2019 10:25:05 -0800] rev 41927
wix: remove sphinx and dependencies
Sphinx was cargo culted into our install environment as part of
emulating TortoiseHG's behavior.
THG seems to install Sphinx in order to generate THG specific
documentation.
We don't appear to need Sphinx or any of its dependencies in
the official WiX installers. So remove it.
This shaves ~1MB off the size of the MSI installers.
.. bc::
The Windows MSI installers no longer include the Python
sphinx package and its various dependencies.
Differential Revision: https://phab.mercurial-scm.org/D6099
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 08 Mar 2019 10:48:22 -0800] rev 41926
wix: functionality to automate building WiX installers
Like we did for Inno Setup, we want to make it easier to
produce WiX installers. This commit does that.
We introduce a new hgpackaging.wix module for performing
all the high-level tasks required to produce WiX installers.
This required miscellaneous enhancements to existing code in
hgpackaging, including support for signing binaries.
A new build.py script for calling into the module APIs has been
created. It behaves very similarly to the Inno Setup build.py
script.
Unlike Inno Setup, we didn't have code in the repo previously
to generate WiX installers. It appears that all existing
automation for building WiX installers lives in the
https://bitbucket.org/tortoisehg/thg-winbuild repository - most
notably in its setup.py file. My strategy for inventing the
code in this commit was to step through the code in that repo's
setup.py and observe what it was doing. Despite the length of
setup.py in that repository, the actual amount of steps required
to produce a WiX installer is actually quite low. It consists
of a basic py2exe build plus invocations of candle.exe and
light.exe to produce the MSI.
One rabbit hole that gave me fits was locating the Visual Studio
9 C Runtime merge modules. These merge modules are only present
on your system if you have a full Visual Studio 2008 installation.
Fortunately, I have a copy of Visual Studio 2008 and was able
to install all the required updates. I then uploaded these merge
modules to a personal repository on GitHub. That is where the
added code references them from. We probably don't need to
ship the merge modules. But that is for another day.
The installs from the MSIs produced with the new automation
differ from the last official MSI in the following ways:
* Our HTML manual pages have UNIX line endings instead of Windows.
* We ship modules in the mercurial.pure package. It appears the
upstream packaging code is not including this package due to
omission (they supply an explicit list of packages that has
drifted out of sync with our setup.py).
* We do not ship various distutils.* modules. This is because
virtualenvs have a custom distutils/__init__.py that automagically
imports distutils from its original location and py2exe gets
confused by this. We don't use distutils in core Mercurial and
don't provide a usable python.exe, so this omission should be
acceptable.
* The version of the enum package is different and we ship
an enum.pyc instead of an enum/__init__.py.
* The version of the docutils package is different and we
ship a different set of files.
* The version of Sphinx is drastically newer and we ship a
number of files the old version did not. (I'm not sure why
we ship Sphinx - I think it is a side-effect of the way the
THG code was installing dependencies.)
* We ship the idna package (dependent of requests which is a
dependency of newer versions of Sphinx).
* The version of imagesize is different and we ship an
imagesize.pyc instead of an imagesize/__init__.pyc.
* The version of the jinja2 package is different and the sets
of files differs.
* We ship the packaging package, which is a dependency for Sphinx.
* The version of the pygments package is different and the sets
of files differs.
* We ship the requests package, which is a dependency for Sphinx.
* We ship the snowballstemmer package, which is a dependency for
Sphinx.
* We ship the urllib3 package, which is a dependency for requests,
which is a dependency for Sphinx.
* We ship a newer version of the futures package, which includes a
handful of extra modules that match Python 3 module names.
# no-check-commit because foo_bar naming
Differential Revision: https://phab.mercurial-scm.org/D6097
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 15:37:42 -0800] rev 41925
wix: move contrib/wix to contrib/packaging/wix
We're trying to consolidate all our packaging code into
contrib/packaging. Let's move the WiX files there.
Differential Revision: https://phab.mercurial-scm.org/D6096
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 08 Mar 2019 10:33:05 -0800] rev 41924
wix: remove hg.cmd
This file is not referenced anywhere AFAICT.
Differential Revision: https://phab.mercurial-scm.org/D6095
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 14:02:02 -0800] rev 41923
setup: include hgext3rd package in py2exe builds
This is a core Mercurial package and we should always ship it.
Differential Revision: https://phab.mercurial-scm.org/D6094
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 13:47:28 -0800] rev 41922
setup: properly install build_hgextindex for py2exe builds
Because the hgbuild class has a private copy of build.sub_commands,
modifying build.sub_commands from this code effectively resulted
in a no-op. Registering the sub-command on hgbuild actually results
in the sub-command running when building Mercurial.
Differential Revision: https://phab.mercurial-scm.org/D6093
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 12:15:32 -0800] rev 41921
setup: configure py2exe config via environment variables
The Inno Setup and WiX installers ship a different set of packages
with py2exe builds. And there are multiple WiX installer variants
(e.g. TortoiseHG).
Since there are multiple variants of py2exe configs and they
can be defined by entities not in our repository, let's
provide a mechanism for setup.py to supplement behavior via
environment variables. This is slighly less hacky than a setup.cfg
file IMO since the caller doesn't need to worry about mutating
global state of the source directory.
Differential Revision: https://phab.mercurial-scm.org/D6092
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 15:43:54 -0800] rev 41920
packaging: extract py2exe functionality to own module
py2exe builds are shared between Inno Setup and WIX. We'll
want the logic for performing py2exe builds to be reusable
across the code for both installers.
This commit extracts the py2exe-specific functionality into
its own module.
There's definitely room to customize things further. This will
be done in future commits, as necessary. (I'm not even sure what
customizations WIX will require yet. Presumably a lot.)
Differential Revision: https://phab.mercurial-scm.org/D6091
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:49:59 -0800] rev 41919
packaging: extract python exe info to own function
This is generic functionality. We'll need it for WIX.
As part of the port, we expose the full version and return
the data as a dict.
Differential Revision: https://phab.mercurial-scm.org/D6090
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:36:20 -0800] rev 41918
packaging: don't use temporary directory
We were no longer doing anything with it after extracting
virtualenv and py2exe to the build directory.
Differential Revision: https://phab.mercurial-scm.org/D6089
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:35:20 -0800] rev 41917
packaging: extract virtualenv and py2exe to build directory
The build directory is essentially a cache. We can extract
the virtualenv and py2exe package sources to this directory.
Differential Revision: https://phab.mercurial-scm.org/D6088
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 15:43:14 -0800] rev 41916
packaging: move Inno Setup core logic into a module
Aspects of building the Inno Setup and WIX installers are shared.
It will make sense for them to share code.
Plus, having code in a reusable library (as opposed to a standalone
script) is just a better approach.
This commit moves the core logic to build the Inno Setup installer
into the hgpackaging package. inno/build.py is now a simple frontend
script that calls into a module to do the bulk of the work.
As part of this change, I also found a typo in build() where it was
referencing "iscc" instead of "iscc_exe." Because "iscc" was in
the global scope via the only caller, things just happened to work
before. Another benefit of always using functions and not putting
global code for __main__ in the same file as library code.
Differential Revision: https://phab.mercurial-scm.org/D6087
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:22:09 -0800] rev 41915
packaging: move find_vc_runtime_files() into hgpackaging.util
In preparation for moving the bulk of the Inno Setup code into
hgpackaging.
Differential Revision: https://phab.mercurial-scm.org/D6086
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:20:37 -0800] rev 41914
packaging: move DOWNLOADS dict to hgpackaging.downloads
We'll want to keep state in sync between multiple packaging tools.
It makes sense to share a central data structure defining downloads.
We also change the function to return the downloads entry so
callers don't have to access the global DOWNLOADS in the new
location.
Differential Revision: https://phab.mercurial-scm.org/D6085
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 15:42:32 -0800] rev 41913
packaging: split downloading code into own module
As we will introduce more code to support packaging, it will be
useful to have download code in its own module.
Differential Revision: https://phab.mercurial-scm.org/D6084
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 07 Mar 2019 10:10:04 -0800] rev 41912
packaging: establish hgpackaging package
Previously, contrib/packaging behaved as a root to a
package directory and we had a "packagingutil" module. As I
work more on packaging code, we'll want to have more code
shared between different packaging tools. I think it makes
sense to have a single package containing multiple modules
than multiple top-level modules.
This commit establishes an "hgpackaging" package by moving
the existing packagingutil code to it.
Differential Revision: https://phab.mercurial-scm.org/D6083
Ian Moody <moz-ian@perix.co.uk> [Sat, 09 Mar 2019 02:07:09 +0000] rev 41911
py3: use % instead of .format() on a bytestring
Differential Revision: https://phab.mercurial-scm.org/D6112
Ian Moody <moz-ian@perix.co.uk> [Fri, 08 Mar 2019 22:26:43 +0000] rev 41910
py3: use r'' for group name arguments to MatchObjects in phabricator.py
MatchObject group names are strings, not bytes. Using bytes in py3 leads to an
IndexError.
Differential Revision: https://phab.mercurial-scm.org/D6111
Ian Moody <moz-ian@perix.co.uk> [Sat, 09 Mar 2019 01:58:51 +0000] rev 41909
py3: use %d instead of %s when formatting an int into a byte string
Differential Revision: https://phab.mercurial-scm.org/D6110
Ian Moody <moz-ian@perix.co.uk> [Sat, 09 Mar 2019 01:53:53 +0000] rev 41908
py3: only pass unicode to json.dumps in writediffproperties
Differential Revision: https://phab.mercurial-scm.org/D6109
Ian Moody <moz-ian@perix.co.uk> [Sat, 09 Mar 2019 01:30:44 +0000] rev 41907
py3: fix a few "dict keys as str instead of bytes" issues in phabricator.py
Differential Revision: https://phab.mercurial-scm.org/D6108
Ian Moody <moz-ian@perix.co.uk> [Sat, 09 Mar 2019 01:00:25 +0000] rev 41906
py3: convert URL to str before passing it to request
Differential Revision: https://phab.mercurial-scm.org/D6106
Ian Moody <moz-ian@perix.co.uk> [Fri, 08 Mar 2019 23:45:12 +0000] rev 41905
py3: convert indexes into bytes when enumerating lists in urlencodenested
Otherwise it'll try to insert them them into a %s slot in a b'' later and fail.
Differential Revision: https://phab.mercurial-scm.org/D6105
Ian Moody <moz-ian@perix.co.uk> [Fri, 08 Mar 2019 23:48:49 +0000] rev 41904
py3: don't try and format a bare dict into a byte string in callconduit
Differential Revision: https://phab.mercurial-scm.org/D6104
Ian Moody <moz-ian@perix.co.uk> [Fri, 08 Mar 2019 17:57:59 +0000] rev 41903
py3: use fsencode for vcr recording paths and strings for custom_patches args
This fixes phabricator.py's vcrcommand under py3
Differential Revision: https://phab.mercurial-scm.org/D6102
Ian Moody <moz-ian@perix.co.uk> [Sat, 02 Mar 2019 18:48:23 +0000] rev 41902
phabricator: convert conduit response JSON unicode to bytes inside callconduit
Previously the byte conversion was happening piecemeal in callers, and in the
case of createdifferentialrevision not at all, leading to UnicodeEncodeErrors
when trying to phabsend a commit with a description containing characters not
representable in ascii. (issue6040)
Remove all the scattered encoding.unitolocal calls and perform it once, inside
callconduit, on the entire response dict recursively before returning it, in
keeping with the strategy of converting at the earliest opportunity. Convert all
keys used on returned object to bytes.
Modify the phabsend tests to test this by adding a € to the commit message of
alpha.
Differential Revision: https://phab.mercurial-scm.org/D6044
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 09 Feb 2019 23:01:30 +0100] rev 41901
transaction: include txnname in the hookargs dictionary
There is no reason to not include the txnname alongside the txnid in all case.
The python hooks already have them, so aligning the the shell hooks seems it
could be useful in the future.
(I don't have a strong opinion about this, we can also decide to never align the
python and shell hooks and this and I'll drop this patch).
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2019 00:00:44 +0100] rev 41900
discovery-helper: reflect argument value in the name of the results
It is common to create multiple pairs of repositories using different argument
values. Recording the argument value in the results names has two main
advantages:
* It is easy to remember the value used to create a pair,
* It get simpler to create multiple pair at the same time from the same source.
Previously, running: `./discovery-helper.sh pypy 50 10` would create a
`pypy-left` and `pypy-right` repository. Now it will create `pypy-50h-10d-left`
and `pypy-50h-10d-right`.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2019 17:21:22 +0100] rev 41899
discovery-helper: echo the stripped revsets early
Having them printed early make it easy for a user to just grab the generated
revset and directly uses them
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2019 17:15:15 +0100] rev 41898
contrib: move the `discovery-helper.sh` script in`perf-utils` directory
The script appeared before the directory. However the directory exists for this
kind of script. We move it there.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 09 Mar 2019 12:55:24 -0500] rev 41897
tests: stabilize test-split.t for Windows
It looks like there will be additional problems beyond this trivial fix, but
this should make the bot green again.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 07 Mar 2019 22:14:22 -0500] rev 41896
tests: stabilize test-share.t on Windows
PYTHON was not getting mangled for MSYS style paths, and remote was spitting out
remote: 'C' is not recognized as an internal or external command,
remote: operable program or batch file.
(once -q was removed). Additionally, this should fix a failure with py3 because
of spaces in the path.
Sushil khanchi <sushilkhanchi97@gmail.com> [Sun, 03 Mar 2019 19:46:59 +0530] rev 41895
split: add tests which demonstrate the issue5864
Differential Revision: https://phab.mercurial-scm.org/D6057
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2019 01:28:24 +0100] rev 41894
discovery: clarify why the caching of children is valid
Yuya Nishihara pointed out that the code looks wrong without this clarification.
(And, unsurprisingly, Yuya is right)
Martin von Zweigbergk <martinvonz@google.com> [Wed, 06 Mar 2019 15:43:52 -0800] rev 41893
tests: clarify test setup test in test-uncommit.t
I assume the "hg uncommit b" is there to prove that the working copy
is dirty before we try "hg uncommit --allow-dirty-working-copy b". It
seems clearer to put that check just before we run the actual test.
Differential Revision: https://phab.mercurial-scm.org/D6078
Martin von Zweigbergk <martinvonz@google.com> [Wed, 06 Mar 2019 15:35:40 -0800] rev 41892
tests: fix a stale reference to experimental.uncommitondirtywdir
These tests no longer test the config option, they test the command
line flag.
Differential Revision: https://phab.mercurial-scm.org/D6077
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 28 Feb 2019 01:49:10 +0100] rev 41891
discovery: explicitly use `undecided` for the children mapping
Recent performance achievements makes the assumption that the `undecided` set
is used for sampling. That assumption is always true in practice. We stop
pretending that taking anything else would make sense and we directly use the
`undecided` set from the object. This provides a more honest API.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 28 Feb 2019 01:48:20 +0100] rev 41890
discovery: cache the children mapping used during each discovery
During discovery, the `undecided` set keep shrinking. Therefore, the map
computed for an iteration N will be valid for iteration N+1. Instead of
computing the same data over and over we cache it the first time.
Our private pathological case speed up from about 7.5 seconds to about 6.3
seconds.
(starting from over 70s at the start of the full series)
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 28 Feb 2019 01:15:45 +0100] rev 41889
discovery: move children computation in its own method
This clarifies the main logic and starts to pave the way to some caching.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 05 Mar 2019 15:39:54 +0100] rev 41888
discovery: simplify the building of the children mapping
Since we only care about the revisions inside the set we are sampling, we can
use simpler code (and probably sightly faster).
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 05 Mar 2019 15:52:14 +0100] rev 41887
discovery: simply walk the undecided revs when building the children mapping
The sampling only care about revisions in the undecided set, so building children
relationship within this set is sufficient.
The set of undecided changesets can be much smaller than the full span from its
smallest item to the tip of the repository. This restriction can significantly
speed up operations in some cases.
For example, on our private pathological case, this speeds things up from about
53 seconds to about 7.5 seconds.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 28 Feb 2019 00:56:27 +0100] rev 41886
discovery: use a lower level but faster way to retrieve parents
We already know that no revision in the undecided set are filtered, so we can
skip multiple checks and directly access lower level data.
In a private pathological case, this improves the timing from about 70 seconds
to about 50 seconds. There are other actions to be taken to improve that case,
however this gives an idea of the general overhead.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 28 Feb 2019 00:12:12 +0100] rev 41885
discovery: avoid computing identical sets of heads twice
The very same set of heads is computed in the previous statement, it seems more
efficient to just copy that result.
Georges Racinet <georges.racinet@octobus.net> [Wed, 27 Feb 2019 23:55:19 +0100] rev 41884
discovery: moved sampling functions inside discovery object
In this patch, we transform the sampling functions into
methods of the `partialdiscovery` class in the Python case.
This will provide multiple benefit. For example we can keep some cache from one
sampling to another. In addition this will help the Oxidation work as all graph
traversal related logic will be contained in a single object.
Georges Racinet <georges.racinet@octobus.net> [Wed, 27 Feb 2019 23:45:06 +0100] rev 41883
discovery: rename `srvheads` to `knownsrvheads`
The `srvheads` variable only contains the known set of remove heads. Renaming
the variable make it clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 14:43:02 +0100] rev 41882
verify: small refactoring and documentation in `_verifymanifest`
Small changes to make this area of code clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 12:39:44 +0100] rev 41881
verify: document the `_verifymanifest` method
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 12:21:58 +0100] rev 41880
verify: document `_verifychangelog`
We document the method input, output and checks.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 14:15:19 +0100] rev 41879
verify: rename the `checklog` to `_checkrevlog`
The method is for internal use only. In addition we make the method name
explicitly contains `revlog` to make it clearer it is checking higher level
revlog properties.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 14:10:23 +0100] rev 41878
verify: document the `checklog` method
Let us add details about what the function is expected to do.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 14:07:27 +0100] rev 41877
revlog: add some documentation to the `checksize` method
I had to look at it, so I figured I would leave some documentation for the next
person seeking information.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 12:20:50 +0100] rev 41876
verify: make `checkentry` a private method
This method is for internal use only.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 12:18:04 +0100] rev 41875
verify: document the `checkentry` method
This method checks various core propertes of a revision. We document inputs, outputs
and the checks performed.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:43:21 +0100] rev 41874
verify: add some inline documentation to the top level `verify` method
The goal is to clarify each section goal.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:38:54 +0100] rev 41873
verify: explicitly return 0 if no error are encountered
Relying on the fact None is treated as 0 by other logics seems smarter than we
should be.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:35:34 +0100] rev 41872
verify: minimal documentation for `verifier.verify`
I expect the docstring to grow over time, so lets start small.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:31:48 +0100] rev 41871
verify: make the `exc` method private
The method is for internal use only.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:31:10 +0100] rev 41870
verify: document the `exc` method
Simple method, get simple documentation.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:27:37 +0100] rev 41869
verify: make `err` a private method
The method is for internal use only.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:25:16 +0100] rev 41868
verify: document the `err` method
Simple method get simple documentation.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:24:40 +0100] rev 41867
verify: make the `warn` method private
The method is for internal use only. Let us make that fact clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 11:23:05 +0100] rev 41866
verify: document the `warn` method
This is the first bit of an effort to document and augment the verify code.
Navaneeth Suresh <navaneeths1998@gmail.com> [Mon, 04 Mar 2019 20:18:13 +0530] rev 41865
uncommit: add flag --allow-dirty-working-copy
This adds a flag `--allow-dirty-working-copy` as an alias for
the experimental config option `experimental.uncommitondirtydir`.
Differential Revision: https://phab.mercurial-scm.org/D6069
Navaneeth Suresh <navaneeths1998@gmail.com> [Tue, 12 Feb 2019 00:17:42 +0530] rev 41864
uncommit: don't allow dirty working copy with PATH (issue5977)
On a dirty PATH, uncommit was working without even setting the config
`experimental.uncommitondirtydir` to `True`. Ideally, it should abort
as it does for a dirty dir. This patch makes uncommit to require the
config option `experimental.uncommitondirtydir` on a dirty PATH.
Original patch to evolve extension authored by Dan Villiom Podlaski Christiansen:
https://bitbucket.org/octobus/evolve-devel/pull-requests/8/bug-5977-uncommit-dirtiness/diff
Differential Revision: https://phab.mercurial-scm.org/D5940
Augie Fackler <augie@google.com> [Tue, 05 Mar 2019 09:51:57 -0500] rev 41863
cleanup: remove two bogus test names from python3 list
I suspect one of these was a typo from the start, the other appears to
have become a .t test at some point.
Differential Revision: https://phab.mercurial-scm.org/D6076
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 04 Mar 2019 15:46:54 +0100] rev 41862
revlog: preserve `_lazydelta` attribute in `revlog.clone`
The attribute was introduce in 688fc33e105d, Yuya Nishihara pointed out that
this preservation was missing. This changeset fixes the preservation and make
sure we set the attribute according the modes.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 04 Mar 2019 16:08:44 +0100] rev 41861
localrepo: explicit `_lock` arguments in `lock`
The argument for `_lock` are non-trivial, having them passed explicitly makes
thing clearer in my opinion. This is a Gratuitous change, I expect it will save
me (and others) time in the future.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 17:22:03 -0800] rev 41860
inno: remove w9xpopen.exe
w9xpopen.exe is a utility program shipped with Python <3.4
(https://bugs.python.org/issue14470 tracked its removal).
The program was used by subprocess to wrap invoked processes
on Windows 95 and 98 or when command.com was used in order to
work around a redirect bug.
The workaround is only used on ancient Windows versions -
versions that we shouldn't see in 2019.
While Python 2.7's subprocess module still references
w9xpopen.exe, not shipping it shouldn't matter unless we're
running an ancient version of Windows. Python will raise
an exception if w9xpopen.exe can't be found.
It's highly unlikely anyone is using current Mercurial releases
on these ancient Windows versions. So remove w9xpopen.exe
from the Inno installer.
.. bc::
The 32-bit Windows Inno installers no longer distribute
w9xpopen.exe. This should only impact people running
Mercurial on Windows 95, 98, or ME.
Differential Revision: https://phab.mercurial-scm.org/D6068
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 15:53:27 -0800] rev 41859
inno: stop shipping pywin32
Ancient versions of Mercurial relied on pywin32 and I suspect
that's why we have this dependency.
We also ship the "keyring" package, which has a dependency
on "pywin32-ctypes" (providing the "win32ctypes" package).
This is a stripped down version of pywin32 that doesn't have
as many dependencies.
Since we don't have a dependency on pywin32 and since pywin32
is a bit annoying to package, let's get rid of it.
With this change, py2exe no longers picks up DLL dependencies
on various UCRT DLLs (because we no longer have a .pyd file
beloning to pywin32 which was pulling them in). So, we were
able to remove code in support of the UCRT DLLs.
.. bc::
The Windows Inno installers no longer ship the pywin32 package.
This package was being bundled for historical reasons. Mercurial
stopped using pywin32 several years ago and the disappearance
of this package should not have any meaningful impact.
Differential Revision: https://phab.mercurial-scm.org/D6067
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 18:19:07 -0800] rev 41858
inno: script to automate building Inno installer
The official Inno installer build process is poorly documented.
And attempting to reproduce behavior of the installer uploaded
to www.mercurial-scm.org has revealed a number of unexpected
behaviors.
This commit attempts to improve the state of reproducibility
of the Inno installer by introducing a Python script to
largely automate the building of the installer.
The new script (which must be run from an environment with the
Visual C++ environment configured) takes care of producing an
Inno installer. When run from a fresh Mercurial source checkout
with all the proper system dependencies (the VC++ toolchain,
Windows 10 SDK, and Inno tools) installed, it "just works."
The script takes care of downloading all the Python
dependencies in a secure manner and manages the build
environment for you. You don't need any additional config
files: just launch the script, pointing it at an existing
Python and ISCC binary and it takes care of the rest.
The produced installer creates a Mercurial installation with
a handful of differences from the existing 4.9 installers
(produced by someone else):
* add_path.exe is missing (this was removed a few changesets ago)
* The set of api-ms-win-core-* DLLs is different (I suspect this
is due to me using a different UCRT / Windows version).
* kernelbase.dll and msasn1.dll are missing.
* There are a different set of .pyc files for dulwich,
keyring, and pygments due to us using the latest versions of
each.
* We include Tcl/Tk DLLs and .pyc files (I'm not sure why these
are missing from the existing installers).
* We include the urllib3 and win32ctypes packages (which are
dependencies of dulwich and pywin32, respectively). I'm not
sure why these aren't present in the existing installers.
* We include a different set of files for the distutils package.
I'm not sure why. But it should be harmless.
* We include the docutils package (it is getting picked up as
a dependency somehow). I think this is fine.
* We include a copy of argparse.pyc. I'm not sure why this was
missing from existing installers.
* We don't have a copy of sqlite3/dump.pyc. I'm not sure why. The
SQLite C extension code only imports this module when
conn.iterdump() is called. It should be safe to omit.
* We include files in the email.test and test packages. The set of
files is small and their presence should be harmless.
The new script and support code is written in Python 3 because
it is brand new and independent code and I don't believe new
Python projects should be using Python 2 in 2019 if they have
a choice about it.
The readme.txt file has been renamed to readme.rst and overhauled
to reflect the existence of build.py.
Differential Revision: https://phab.mercurial-scm.org/D6066
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 14:08:25 -0800] rev 41857
setup: exclude some internal UCRT files
When attempting to build the Inno installer locally, I was getting
several file not found errors when py2exe was crawling DLL
dependencies. The missing DLLs appear to be "internal" DLLs
used by the Universal C Runtime (UCRT). In many cases, the
missing DLLs don't appear to exist on my system at all!
Some of the DLLs have version numbers that appear to be N+1
of what the existing version number is. Maybe the "public" UCRT
DLLs are probing for version N+1 at load time and py2exe is
picking these up? Who knows.
This commit adds the non-public UCRT DLLs as found by
py2exe on my system to the excluded DLLs set. After this
change, I'm able to produce an Inno installer with an
appropriate set of DLLs.
Differential Revision: https://phab.mercurial-scm.org/D6065
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 15:46:26 -0800] rev 41856
setup: include additional packages in py2exe distribution
I'm attempting to reproduce the Inno installers on my local
machine. As part of auditing differences between installer output,
I noticed that the existing Inno installers include various 3rd
party packages.
There is no mention of this in the build instructions nor on
the wiki. This must be something that is done by the installer
producer.
This commit teaches setup.py to include these 3rd party packages
in py2exe's library. After this change, I am able to produce
Inno installers that have a nearly identical set of Python
modules.
It's worth noting that pywin32 is included even though it
probably shouldn't be. But including it is necessary in order
to achieve parity with existing Inno installers.
Differential Revision: https://phab.mercurial-scm.org/D6064
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 10:31:23 -0800] rev 41855
setup: define build_doc command
Currently, various processes for packaging Mercurial state to
manually invoke `make -C doc` in order to generate the documentation.
This Makefile merely invokes `gendoc.py` and `runrst` to produce
man pages and HTML pages.
Not all environments may have the ability to easily run
Makefiles. Windows is notably in this set.
This commit ports the man page and HTML generation logic from
doc/Makefile to setup.py. We introduce a new build_doc command
which generates documentation by calling gendoc.py and runrst.
The documentation can now be built via pure Python by running
`python setup.py build_doc`.
We don't implement dependency tracking because IMO it is more
effort than it is worth.
We could potentially remove the duplicated functionality in
doc/Makefile. But I'm not sure what all is depending on it. So
I plan to keep it around.
# no-check-commit because forced foo_bar function names
Differential Revision: https://phab.mercurial-scm.org/D6063
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 09:16:37 -0800] rev 41854
inno: remove references to pywin32
According to the commit message for 0c35bb01a1195, pywin32 was
removed in Mercurial 1.8!
Differential Revision: https://phab.mercurial-scm.org/D6062
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 09:10:52 -0800] rev 41853
inno: remove cacerts.pem from documentation
The inno distribution no longer includes cacert.pem as of
ca1ad8ef38be22 (April 2015). The docs were never updated.
Differential Revision: https://phab.mercurial-scm.org/D6061
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 28 Feb 2019 12:54:48 -0800] rev 41852
inno: replace add_path.exe with a Pascal script
While attempting to build the Inno installer, I was unable to
find a copy of add_path.exe from the source site previously
listed in the docs.
Some quick Googling revealed that achieving this functionality
in native Pascal scripts seems to be preferred these days.
This commit vendors "Modify Path" (fetched from
https://www.legroom.net/software/modpath) and plugs it into
our Inno config file per its instructions.
The existing Inno installer appears to only modify PATH for
the current user (as opposed to at the system level). I've
maintained this behavior with Modify Path. Although it would
be trivial to change or add checkboxes to control the behavior.
I'll leave this as a follow-up.
Differential Revision: https://phab.mercurial-scm.org/D6060
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 03 Mar 2019 18:22:49 -0800] rev 41851
inno: move inno installer files to contrib/packaging/inno
Let's isolate the inno installer files to their own directory
so the separation between things is clearer.
This required adjusting a few relative paths and references to
the old directory.
Differential Revision: https://phab.mercurial-scm.org/D6059
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 05 Mar 2019 04:34:45 +0530] rev 41850
py3: add a new passing test found by buildbot
Differential Revision: https://phab.mercurial-scm.org/D6073
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 05 Mar 2019 04:32:23 +0530] rev 41849
tests: make test-status.t compatible with test-check-module-imports.t
Otherwise the later fails on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D6072
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 05 Mar 2019 04:11:32 +0530] rev 41848
py3: convert KEY_PRESSED value to bytes in crecord.py
This was a str before.
Differential Revision: https://phab.mercurial-scm.org/D6071
Augie Fackler <augie@google.com> [Mon, 04 Mar 2019 18:11:50 -0500] rev 41847
hghave: skip emacs tests on 24.3 and earlier
Turns out with-eval-after-load is new in 24.4.
Differential Revision: https://phab.mercurial-scm.org/D6074
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 05 Mar 2019 03:33:41 +0530] rev 41846
py3: use bytes instead of str in isinstance()
We use bytes everywhere.
Differential Revision: https://phab.mercurial-scm.org/D6070
Joerg Sonnenberger <joerg@bec.de> [Tue, 12 Feb 2019 19:08:17 +0100] rev 41845
server: allow customizing the default repo filter
hgweb has the (undocument) configuration option web.view that allows
restricting visible revisions to immutable. This is useful for serving
the same storage as publishing and non-publishing repo. Add the new
server.view option to serve the same purpose by changing the default
behavior of `getdispatchrepo`. Drop the hard-coded 'served' filter in the
batch handler of v1 of the wire proto, this is a left-over from the days
before `getdispatchrepo` existed.
Differential Revision: https://phab.mercurial-scm.org/D5946
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 02 Mar 2019 05:24:35 +0530] rev 41844
py3: convert filtername to str if it's None
I have not called pycompat.bytestr() and rather converted the value there
because I am starting to get concerned about the function call overhead of all
this bytes to str or vice versa convert functions.
Differential Revision: https://phab.mercurial-scm.org/D6042
Martin von Zweigbergk <martinvonz@google.com> [Sat, 02 Mar 2019 13:28:17 -0800] rev 41843
walkfilerevs: rename filerevgen() to filerevs() since it's not a generator
Differential Revision: https://phab.mercurial-scm.org/D6053
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 13:15:53 -0800] rev 41842
global: use raw string for setlocale() argument
Otherwise Python 2 will coerce a unicode to str, which fails
on HGUNICODEPEDANTRY=1.
Differential Revision: https://phab.mercurial-scm.org/D6052
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 13:07:58 -0800] rev 41841
encoding: use raw strings for encoding arguments
This prevents the internals of Python from coercing a unicode to str
on Python 2 and makes tests run with HGUNICODEPEDANTRY=1 a lot
happier.
Differential Revision: https://phab.mercurial-scm.org/D6051
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 13:02:39 -0800] rev 41840
revsetlang: use sysbytes() instead of blind encode()
Otherwise we will call str.encode() on Python 2, which is wrong.
sysbytes() does encode('utf-8') on Python 3. But the source is
guaranteed ASCII, so it shouldn't matter.
With this change, `hg` now runs with `HGUNICODEPEDANTRY=1` set.
However, several tests are failing.
Differential Revision: https://phab.mercurial-scm.org/D6050
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 12:57:00 -0800] rev 41839
global: use raw strings for namedtuple()
Otherwise Python 2 will attempt to coerce unicode to str, which
we don't want.
Differential Revision: https://phab.mercurial-scm.org/D6049
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 12:55:29 -0800] rev 41838
attr: don't attempt to .encode() a str on Python 2
Otherwise it coerces automatically.
Differential Revision: https://phab.mercurial-scm.org/D6048
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 12:51:55 -0800] rev 41837
procutil: use a raw string for module name
Otherwise Python 2 will coerce unicode to str.
Differential Revision: https://phab.mercurial-scm.org/D6047
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 02 Mar 2019 12:51:04 -0800] rev 41836
global: use raw strings for __slots__
Otherwise Python 2 will coerce unicode to str at module load time.
We don't like automatic string coercions.
Differential Revision: https://phab.mercurial-scm.org/D6046
Martin von Zweigbergk <martinvonz@google.com> [Thu, 28 Feb 2019 07:45:51 -0800] rev 41835
absorb: run cleanupnodes() within transaction also when not using obsmarkers
scmutil.cleanupnodes() schedules stripping to be done after the
current transaction, so we can safely run it within the
transaction. This also means that the phase will be updated within the
transaction, which I believe means that the initial (possibly
incorrect) phase will not visible.
Differential Revision: https://phab.mercurial-scm.org/D6037
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 02 Mar 2019 05:23:15 +0530] rev 41834
py3: pass a str to getpass.getpass()
Otherwise I see `password: b''` on the prompt.
# skip-blame because just prefix
Differential Revision: https://phab.mercurial-scm.org/D6041
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 02 Mar 2019 05:12:45 +0530] rev 41833
py3: port things from chgserver.py
I have installed mercurial on my personal laptop using Python 3.7. I also have
hg aliased to chg. Before this commit, `hg version` didn't work. After this
patch, things are better and now chg works.
Differential Revision: https://phab.mercurial-scm.org/D6040
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 02 Mar 2019 05:01:00 +0530] rev 41832
py3: make sure return value of posix.groupname() is bytes
Differential Revision: https://phab.mercurial-scm.org/D6039