localrepo: explicit `_lock` arguments in `lock` The argument for `_lock` are non-trivial, having them passed explicitly makes thing clearer in my opinion. This is a Gratuitous change, I expect it will save me (and others) time in the future.

inno: remove w9xpopen.exe w9xpopen.exe is a utility program shipped with Python <3.4 (https://bugs.python.org/issue14470 tracked its removal). The program was used by subprocess to wrap invoked processes on Windows 95 and 98 or when command.com was used in order to work around a redirect bug. The workaround is only used on ancient Windows versions - versions that we shouldn't see in 2019. While Python 2.7's subprocess module still references w9xpopen.exe, not shipping it shouldn't matter unless we're running an ancient version of Windows. Python will raise an exception if w9xpopen.exe can't be found. It's highly unlikely anyone is using current Mercurial releases on these ancient Windows versions. So remove w9xpopen.exe from the Inno installer. .. bc:: The 32-bit Windows Inno installers no longer distribute w9xpopen.exe. This should only impact people running Mercurial on Windows 95, 98, or ME. Differential Revision: https://phab.mercurial-scm.org/D6068

inno: stop shipping pywin32 Ancient versions of Mercurial relied on pywin32 and I suspect that's why we have this dependency. We also ship the "keyring" package, which has a dependency on "pywin32-ctypes" (providing the "win32ctypes" package). This is a stripped down version of pywin32 that doesn't have as many dependencies. Since we don't have a dependency on pywin32 and since pywin32 is a bit annoying to package, let's get rid of it. With this change, py2exe no longers picks up DLL dependencies on various UCRT DLLs (because we no longer have a .pyd file beloning to pywin32 which was pulling them in). So, we were able to remove code in support of the UCRT DLLs. .. bc:: The Windows Inno installers no longer ship the pywin32 package. This package was being bundled for historical reasons. Mercurial stopped using pywin32 several years ago and the disappearance of this package should not have any meaningful impact. Differential Revision: https://phab.mercurial-scm.org/D6067

inno: script to automate building Inno installer The official Inno installer build process is poorly documented. And attempting to reproduce behavior of the installer uploaded to www.mercurial-scm.org has revealed a number of unexpected behaviors. This commit attempts to improve the state of reproducibility of the Inno installer by introducing a Python script to largely automate the building of the installer. The new script (which must be run from an environment with the Visual C++ environment configured) takes care of producing an Inno installer. When run from a fresh Mercurial source checkout with all the proper system dependencies (the VC++ toolchain, Windows 10 SDK, and Inno tools) installed, it "just works." The script takes care of downloading all the Python dependencies in a secure manner and manages the build environment for you. You don't need any additional config files: just launch the script, pointing it at an existing Python and ISCC binary and it takes care of the rest. The produced installer creates a Mercurial installation with a handful of differences from the existing 4.9 installers (produced by someone else): * add_path.exe is missing (this was removed a few changesets ago) * The set of api-ms-win-core-* DLLs is different (I suspect this is due to me using a different UCRT / Windows version). * kernelbase.dll and msasn1.dll are missing. * There are a different set of .pyc files for dulwich, keyring, and pygments due to us using the latest versions of each. * We include Tcl/Tk DLLs and .pyc files (I'm not sure why these are missing from the existing installers). * We include the urllib3 and win32ctypes packages (which are dependencies of dulwich and pywin32, respectively). I'm not sure why these aren't present in the existing installers. * We include a different set of files for the distutils package. I'm not sure why. But it should be harmless. * We include the docutils package (it is getting picked up as a dependency somehow). I think this is fine. * We include a copy of argparse.pyc. I'm not sure why this was missing from existing installers. * We don't have a copy of sqlite3/dump.pyc. I'm not sure why. The SQLite C extension code only imports this module when conn.iterdump() is called. It should be safe to omit. * We include files in the email.test and test packages. The set of files is small and their presence should be harmless. The new script and support code is written in Python 3 because it is brand new and independent code and I don't believe new Python projects should be using Python 2 in 2019 if they have a choice about it. The readme.txt file has been renamed to readme.rst and overhauled to reflect the existence of build.py. Differential Revision: https://phab.mercurial-scm.org/D6066

setup: exclude some internal UCRT files When attempting to build the Inno installer locally, I was getting several file not found errors when py2exe was crawling DLL dependencies. The missing DLLs appear to be "internal" DLLs used by the Universal C Runtime (UCRT). In many cases, the missing DLLs don't appear to exist on my system at all! Some of the DLLs have version numbers that appear to be N+1 of what the existing version number is. Maybe the "public" UCRT DLLs are probing for version N+1 at load time and py2exe is picking these up? Who knows. This commit adds the non-public UCRT DLLs as found by py2exe on my system to the excluded DLLs set. After this change, I'm able to produce an Inno installer with an appropriate set of DLLs. Differential Revision: https://phab.mercurial-scm.org/D6065

setup: include additional packages in py2exe distribution I'm attempting to reproduce the Inno installers on my local machine. As part of auditing differences between installer output, I noticed that the existing Inno installers include various 3rd party packages. There is no mention of this in the build instructions nor on the wiki. This must be something that is done by the installer producer. This commit teaches setup.py to include these 3rd party packages in py2exe's library. After this change, I am able to produce Inno installers that have a nearly identical set of Python modules. It's worth noting that pywin32 is included even though it probably shouldn't be. But including it is necessary in order to achieve parity with existing Inno installers. Differential Revision: https://phab.mercurial-scm.org/D6064

setup: define build_doc command Currently, various processes for packaging Mercurial state to manually invoke `make -C doc` in order to generate the documentation. This Makefile merely invokes `gendoc.py` and `runrst` to produce man pages and HTML pages. Not all environments may have the ability to easily run Makefiles. Windows is notably in this set. This commit ports the man page and HTML generation logic from doc/Makefile to setup.py. We introduce a new build_doc command which generates documentation by calling gendoc.py and runrst. The documentation can now be built via pure Python by running `python setup.py build_doc`. We don't implement dependency tracking because IMO it is more effort than it is worth. We could potentially remove the duplicated functionality in doc/Makefile. But I'm not sure what all is depending on it. So I plan to keep it around. # no-check-commit because forced foo_bar function names Differential Revision: https://phab.mercurial-scm.org/D6063

inno: remove references to pywin32 According to the commit message for 0c35bb01a1195, pywin32 was removed in Mercurial 1.8! Differential Revision: https://phab.mercurial-scm.org/D6062

inno: remove cacerts.pem from documentation The inno distribution no longer includes cacert.pem as of ca1ad8ef38be22 (April 2015). The docs were never updated. Differential Revision: https://phab.mercurial-scm.org/D6061

inno: replace add_path.exe with a Pascal script While attempting to build the Inno installer, I was unable to find a copy of add_path.exe from the source site previously listed in the docs. Some quick Googling revealed that achieving this functionality in native Pascal scripts seems to be preferred these days. This commit vendors "Modify Path" (fetched from https://www.legroom.net/software/modpath) and plugs it into our Inno config file per its instructions. The existing Inno installer appears to only modify PATH for the current user (as opposed to at the system level). I've maintained this behavior with Modify Path. Although it would be trivial to change or add checkboxes to control the behavior. I'll leave this as a follow-up. Differential Revision: https://phab.mercurial-scm.org/D6060

inno: move inno installer files to contrib/packaging/inno Let's isolate the inno installer files to their own directory so the separation between things is clearer. This required adjusting a few relative paths and references to the old directory. Differential Revision: https://phab.mercurial-scm.org/D6059

py3: add a new passing test found by buildbot Differential Revision: https://phab.mercurial-scm.org/D6073

tests: make test-status.t compatible with test-check-module-imports.t Otherwise the later fails on Python 3. Differential Revision: https://phab.mercurial-scm.org/D6072

py3: convert KEY_PRESSED value to bytes in crecord.py This was a str before. Differential Revision: https://phab.mercurial-scm.org/D6071

hghave: skip emacs tests on 24.3 and earlier Turns out with-eval-after-load is new in 24.4. Differential Revision: https://phab.mercurial-scm.org/D6074

py3: use bytes instead of str in isinstance() We use bytes everywhere. Differential Revision: https://phab.mercurial-scm.org/D6070

server: allow customizing the default repo filter hgweb has the (undocument) configuration option web.view that allows restricting visible revisions to immutable. This is useful for serving the same storage as publishing and non-publishing repo. Add the new server.view option to serve the same purpose by changing the default behavior of `getdispatchrepo`. Drop the hard-coded 'served' filter in the batch handler of v1 of the wire proto, this is a left-over from the days before `getdispatchrepo` existed. Differential Revision: https://phab.mercurial-scm.org/D5946

py3: convert filtername to str if it's None I have not called pycompat.bytestr() and rather converted the value there because I am starting to get concerned about the function call overhead of all this bytes to str or vice versa convert functions. Differential Revision: https://phab.mercurial-scm.org/D6042

walkfilerevs: rename filerevgen() to filerevs() since it's not a generator Differential Revision: https://phab.mercurial-scm.org/D6053

global: use raw string for setlocale() argument Otherwise Python 2 will coerce a unicode to str, which fails on HGUNICODEPEDANTRY=1. Differential Revision: https://phab.mercurial-scm.org/D6052

encoding: use raw strings for encoding arguments This prevents the internals of Python from coercing a unicode to str on Python 2 and makes tests run with HGUNICODEPEDANTRY=1 a lot happier. Differential Revision: https://phab.mercurial-scm.org/D6051

revsetlang: use sysbytes() instead of blind encode() Otherwise we will call str.encode() on Python 2, which is wrong. sysbytes() does encode('utf-8') on Python 3. But the source is guaranteed ASCII, so it shouldn't matter. With this change, `hg` now runs with `HGUNICODEPEDANTRY=1` set. However, several tests are failing. Differential Revision: https://phab.mercurial-scm.org/D6050

global: use raw strings for namedtuple() Otherwise Python 2 will attempt to coerce unicode to str, which we don't want. Differential Revision: https://phab.mercurial-scm.org/D6049

attr: don't attempt to .encode() a str on Python 2 Otherwise it coerces automatically. Differential Revision: https://phab.mercurial-scm.org/D6048

procutil: use a raw string for module name Otherwise Python 2 will coerce unicode to str. Differential Revision: https://phab.mercurial-scm.org/D6047

global: use raw strings for __slots__ Otherwise Python 2 will coerce unicode to str at module load time. We don't like automatic string coercions. Differential Revision: https://phab.mercurial-scm.org/D6046

absorb: run cleanupnodes() within transaction also when not using obsmarkers scmutil.cleanupnodes() schedules stripping to be done after the current transaction, so we can safely run it within the transaction. This also means that the phase will be updated within the transaction, which I believe means that the initial (possibly incorrect) phase will not visible. Differential Revision: https://phab.mercurial-scm.org/D6037

py3: pass a str to getpass.getpass() Otherwise I see `password: b''` on the prompt. # skip-blame because just prefix Differential Revision: https://phab.mercurial-scm.org/D6041

py3: port things from chgserver.py I have installed mercurial on my personal laptop using Python 3.7. I also have hg aliased to chg. Before this commit, `hg version` didn't work. After this patch, things are better and now chg works. Differential Revision: https://phab.mercurial-scm.org/D6040

py3: make sure return value of posix.groupname() is bytes Differential Revision: https://phab.mercurial-scm.org/D6039

contrib: make check-code.py check code fragments embedded in test scripts

contrib: add line offset information to file check function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This information will be useful to show correct line number in an actual file for errors detected in code fragments embedded in *.t test scripts.

contrib: change return value of file checking function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. After this patch, caller of _checkfiledata() can count number of errors in each code fragments of an actual file. This will be useful to share --per-file limitation by all embedded code fragments.

contrib: factor out actual error check for file data of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Newly added _checkfiledata() will be useful to apply checks on code fragments embedded in *.t test scripts.

contrib: refactor preparation logic for patterns of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Before this patch, preparation logic in _preparepats() of check-code.py is not reusable. It can handle only module global list "checks". This patch splits preparation logic into small internal functions, and add the loop to invoke them, in order to increase reusability of the logic. "c[-2]" is equivalent to "c[3]" for "checks". This patch uses the former, because it will be more reusable for subsequent patch than the latter.

contrib: split pypats list in check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Ideally, all patterns in "pypats" before this patch should be applied on not only normal *.py files but also code fragments embedded in *.t test scripts. But fixing test scripts for some patterns requires many changes, and has less profit than effort. Therefore, this patch splits pypats list into two below: - commonpypats, which are applied on all (= including code fragments embedded in *.t test scripts) *.py files - pypats, which are applied only on normal *.py

tests: remove useless line wrapping in a code fragment embedded in test script This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This revision avoids error of check-code.py below. Use () to wrap long lines in Python, not \

storage: introduce a `revlog.reuse-external-delta` config This option goes a bit further and provides a way to get the same behavior as the `re-delta-all` optimisation from `hg debugupgraderepo`. The effect of the option is a bit hard to test as we do not have multiple diff algorithm at hand. However, we at least make sure the code path run.

storage: introduce a `revlog.reuse-external-delta-parent` config As pointed in c59987ab96b4, we had no simple way to get a client to not blindly reuse the delta parent from a bundle. Instead one had to rely on a side effect of the deprecated `format.generaldelta` configuration. We introduce an explicit `revlog.reuse-external-delta-parent` configuration option (default to True) to control this behavior. If the option is not set, `format.generaldelta` still control this behavior. To test the new option, we convert a couple of place where `generaldelta` have been used for its side effects.

merge with stable

py3: convert return values of inspect.getabsfile() to bytes It's weird that python docs for inspect does not mention getabsfile(). Differential Revision: https://phab.mercurial-scm.org/D6033

py3: add two new passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D6032

py3: make contrib/debugshell.py work with Python 3 I changed default mercurial installation of my personal laptop to one installed with python 3.7. debugshell is one of the extension which I have enabled and it was failing. This patch makes debugshell works with Python 3. I found that chg does not work with python 3. Differential Revision: https://phab.mercurial-scm.org/D6031

absorb: let scmutil.cleanupnodes() take care of setting phase Differential Revision: https://phab.mercurial-scm.org/D6036

absorb: use scmutil.cleanupnodes() also when obsmarkers are disabled A side-effect of this is the new warning in the test case, but the warning is valid, so I don't think that's a problem. Differential Revision: https://phab.mercurial-scm.org/D6035

absorb: use scmutil.cleanupnodes() so operation gets set The operation is useful for e.g. `hg obslog` output. Differential Revision: https://phab.mercurial-scm.org/D6034

test: follow-up on 1c4d6ab2ecb8, stabilize test-remotefilelog-bgprefetch.t In 1c4d6ab2ecb8, I've added some sleeps after about half of the calls to `hg debugwaitonrepack` but it seems the other calls are also flaky as catched by our CI: https://ci.octobus.net/job/MercurialPy2/299/console. Add sleeps after all calls in order to be safe.

tests: add more wildcards to test-extdiff.t This test has been flakey recently on Windows, but it appears to be the same scenario as 88d4477ac4f6.