rust-format: cleanup ancestors.rs to make rustfmt happy Differential Revision: https://phab.mercurial-scm.org/D8224

heptapod-ci: use strict module policy Without this, test can silently fallback to other compatible policy in some cases. Differential Revision: https://phab.mercurial-scm.org/D8234

histedit: fix formatting after D8150 Differential Revision: https://phab.mercurial-scm.org/D8230

rust-format: make rustfmt happy Differential Revision: https://phab.mercurial-scm.org/D8228

Added signature for changeset 6d121acbb82e

Added tag 5.3.1 for changeset 6d121acbb82e

remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269) This cause traceback on widening using narrow extension when remotefilelog is enabled. Differential Revision: https://phab.mercurial-scm.org/D8134

remotefilelog-test: glob some flaky output line This is similar to ee0959e7d435. The affected line is flaky underload, yet the final result is correct. The command involves background pre-check of output, these are not stable probably because they run in parallel in multiple process. If it become useful to start testing precise internal details of the, they will have to be tested in a more appropriate framework than `.t` tests. Differential Revision: https://phab.mercurial-scm.org/D8179

histedit: py3 fixes for curses mode Differential Revision: https://phab.mercurial-scm.org/D8150

branch: make --force work even when specifying revs The `hg branch` command accepts a `--force` parameter that allows to "set branch name even if it shadows an existing branch". However, before this patch, that didn’t work when specifying revs with `-r`.

setup: exclude the __index__ module from itself when generating This module is generated on Windows to hold all of the extension names and the help summaries, so that they are discoverable inside the py2exe zipfile. The problem is this file is generated by dumping the disabled list, and that list comes from walking the filesystem. So once an install from source into a virtualenv created this module, then next build from source from that virtualenv would also see __index__.py in the filesystem, and include it. Clearly that's wrong because this isn't a real extension, so just filter it from the list when generating it. The Mercurial installer was unaffected by this, but the TortoiseHg package was. In the final package, `hg help -v extensions` and the panel of extensions both showed it. Differential Revision: https://phab.mercurial-scm.org/D8142

tests: stabilize test-amend.t on Windows If $TESTTMP isn't quoted in this context, it ends up like `C:Temphgtests.pikkoxchild1test-amend.t-obsstore-off`. Differential Revision: https://phab.mercurial-scm.org/D8144

py3: fix EOL detection in commandserver.channeledinput This breaks TortoiseHg's email preview which sends b'\n' while readline request is issued and the loop never ends. Spotted by Matt Harbison.

shelve: fix ordering of merge labels Differential Revision: https://phab.mercurial-scm.org/D8140

shelve: add test clearly demonstrating that the conflict labels are backwards Differential Revision: https://phab.mercurial-scm.org/D8139

import: don't ignore `--secret` when `--bypass` is specified Differential Revision: https://phab.mercurial-scm.org/D8126

phabricator: fix a phabsend crash when processing a renamed binary This was a trivial fix, and some more tests are added to cover binary files. Since the old filecontext is passed in, the old name is still available. But I noticed some weirdness around what it marked as binary and not, and what is viewable in Phabricator. Those things have been flagged, and will probably take some digging. Differential Revision: https://phab.mercurial-scm.org/D8133

test: pin the number of CPU for issue4074 tests On machine with an hundreds of CPUs, the "user" CPU time reported can be inflated by the status steps. Since the test especially focus on the diff computation, we restrict the number of CPU to avoid potential issues. Differential Revision: https://phab.mercurial-scm.org/D8112

rust-dirstatemap: add `NonNormalEntries` class This fix introduces the same encapsulation as the `copymap`. There is no easy way of doing this any better for now. `hg up -r null && time HGRCPATH= HGMODULEPOLICY=rust+c hg up tip` on Mozilla Central, (not super recent, but it doesn't matter): Before: 7:44,08 total After: 1:03,23 total Pretty brutal regression! This is a graft on stable of cf1f8660e568 Differential Revision: https://phab.mercurial-scm.org/D8111

rust-dirstatemap: cache non normal and other parent set Performance of `hg update` was significantly worse since the introduction of the Rust `dirstatemap`. This regression was noticed by Valentin Gatien-Baron when working on a large repository, as it goes unnoticed for smaller repositories like Mercurial itself. This fix introduces the same getter/setter mechanism at `hg-core` level as for `set/get_dirs`. While this technique is, as previously discussed, quite suboptimal, it fixes an important enough problem. Refactoring `hg-core` to use the typestate pattern could be a good approach to improving code quality in a future patch. This is a graft of stable of 83b2b829c94e Differential Revision: https://phab.mercurial-scm.org/D8110

chgserver: spawn new process if schemes change The schemes extension updates hg.schemes table. It's technically possible for hg.repository() to look for e.g. ui.schemes instead of depending on module-local table, but I don't think the change would make much sense since [schemes] is usually specified in ~/.hgrc and thus it can be considered static data.

tests: accept new bzr message about switching branches The new version apparently prints "Switched to branch at " instead of "Switched to branch: ". Differential Revision: https://phab.mercurial-scm.org/D8106

tests: add workaround for bzr bug This started failing for me today. I guess my bzr was upgraded. Differential Revision: https://phab.mercurial-scm.org/D8105

test: simplify test-amend.t to avoid race condition Insted on relying on sleep, we could simply have the editor do the file change. This remove the reliance on "sleep" and avoid test failing on heavy load machine. To test this, I reverted the code change in 5558e3437872 and the test started failing again. This is a graft on stable of 141ceec06b55 which should have targeted for stable. Differential Revision: https://phab.mercurial-scm.org/D8103

remotefilelog-test: glob some flaky output line (issue6083) The two following lines are flaky underload, yet the final result is correct. The command involves background pre-check of output, these are not stable probably because they run in parallel in multiple process. I spent a couple of hours trying to understand the pattern and gave up. The documented intend of these tests is safely guaranteed by checking the cache content after the command. If it become useful to start testing precise internal details of the, they will have to be tested in a more appropriate framework than `.t` tests. Differential Revision: https://phab.mercurial-scm.org/D8102

resourceutil: blacken

py3: fully fix bundlepart.__repr__ to return str not bytes My previous fix did not fully fix the issue: it would attempt to use %-formatting to combine two strs into a bytes, which won't work. Let's just switch the entire function to operating in strs. This can cause a small output difference that will likely not be noticed since no one noticed that the method wasn't working at all before: if `id` or `type` are not-None, they'll be shown as `b'val'` instead of `val`. Since this is a debugging aid and these strings shouldn't be shown to the user, slightly rough output is most likely fine and it's likely not worthwhile to add the necessary conditionals to marginally improve it. Differential Revision: https://phab.mercurial-scm.org/D8091

heptapod-ci: add a job to test the rust version of Mercurial The rust version of Mercurial is not currently tested by anything else. So it get quite important that developer runs it. Differential Revision: https://phab.mercurial-scm.org/D8017

heptapod-ci: run the --pure test too These are usually rarely run by individual developper because they are slow. However it is important that they stay happy. Differential Revision: https://phab.mercurial-scm.org/D8016

heptapod-ci: run the normal test suite The usual tests should be run too. We skip the "tests-check*.t" one because their are already covered by another Ci step. Differential Revision: https://phab.mercurial-scm.org/D8015

heptapod-ci: also run the dedicated rust test for the rust code The Rust code has various standard rust test that are fast to run. So let's run them. Differential Revision: https://phab.mercurial-scm.org/D8014

heptapod-ci: run test with python3 too Python3 is the future^W present, it is important to run tests with it too. Differential Revision: https://phab.mercurial-scm.org/D8013

heptapod-ci: colorize output The run result are nicer to read with color. Differential Revision: https://phab.mercurial-scm.org/D8012

heptapod-ci: add a basic file to be able to run tests with heptapod Having this yaml file somewhere in the main mercurial repository makes it trivial for contributors using heptapod to run CI on their in-progress work. There are alot of different combination (python2/python3 pure/cext/rust/pypy) to be tested and making sure all of them are covered manually is cumbersome. Automatic CI runnig on draft really helps in that matters. We start small bu later changesets will add more step testing more of the variants. The series is targetted on stable to make it available to the widest amount of contribution possible. The definition of the docker files used for this are available here: https://dev.heptapod.net/octobus/ci-dockerfiles Differential Revision: https://phab.mercurial-scm.org/D8011

worker: manually buffer reads from pickle stream My previous fix (D8051, cb52e619c99e, which added Python's built-in buffering to the pickle stream) has the problem that the selector will ignore the buffer. When multiple pickled objects are read from the pipe into the buffer at once, only one object will be loaded. This can repeat until the buffer is full and delays the processing of completed items until the worker exits, at which point the pipe is always considered readable and all remaining items are processed. This changeset reverts D8051, removing the buffer again. Instead, on Python 3 only, we use a wrapper to modify the "read" provided to the Unpickler to behave more like a buffered read. We never read more bytes from the pipe than the Unpickler requests, so the selector behaves as expected. Also add a test case for "pickle data was truncated" issue. https://phab.mercurial-scm.org/D8051#119193 Differential Revision: https://phab.mercurial-scm.org/D8076

py3: __repr__ needs to return str, not bytes Differential Revision: https://phab.mercurial-scm.org/D8089

config: also respect HGRCSKIPREPO in the zeroconf extension Differential Revision: https://phab.mercurial-scm.org/D8075

config: also respect HGRCSKIPREPO in hgwebdir_mod Differential Revision: https://phab.mercurial-scm.org/D8074

config: also respect HGRCSKIPREPO in `dispatch._getlocal` For some reason, we are also reading the local config in that function. Differential Revision: https://phab.mercurial-scm.org/D8073

config: add a function in `rcutil` to abstract HGRCSKIPREPO We wil need to respect this environment variable in more place. Differential Revision: https://phab.mercurial-scm.org/D8072

packaging: make the path to Win32 requirements absolute when building WiX Otherwise this broke automation when not launched from `contrib/packaging`. Differential Revision: https://phab.mercurial-scm.org/D8068

Added signature for changeset 7f5410dfc8a6

Added tag 5.3 for changeset 7f5410dfc8a6

rust-dirstatemap: add missing @propertycache While investigating a regression on `hg update` performance introduced by the Rust `dirstatemap`, two missing `@propertycache` were identified when comparing against the Python implementation. This adds back the first one, that has no observable impact on behavior. The second one (`nonnormalset`) is going to be more involved, as the caching has to be done from the Rust side of things. Differential Revision: https://phab.mercurial-scm.org/D8047

worker: Use buffered input from the pickle stream On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError: pickle data was truncated") when it gets a short read, i.e. it receives fewer bytes than it requested. On our build machine, Mercurial seems to frequently hit this problem while updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy to trigger with: #wipe the workdir rm -rf * hg update null chrt -b 0 hg update default I've also written the following program, which demonstrates the core problem: from __future__ import print_function import io import os import pickle import time obj = {"a": 1, "b": 2} obj_data = pickle.dumps(obj) assert len(obj_data) > 10 rfd, wfd = os.pipe() pid = os.fork() if pid == 0: os.close(rfd) for _ in range(4): time.sleep(0.5) print("First write") os.write(wfd, obj_data[:10]) time.sleep(0.5) print("Second write") os.write(wfd, obj_data[10:]) os._exit(0) try: os.close(wfd) rfile = os.fdopen(rfd, "rb", 0) print("Reading") while True: try: obj_copy = pickle.load(rfile) assert obj == obj_copy except EOFError: break print("Success") finally: os.kill(pid, 15) The program reliably fails with Python 3.8 and succeeds with Python 2.7. Providing the unpickler with a buffered reader fixes the issue, so let "os.fdopen" create one. https://bugzilla.mozilla.org/show_bug.cgi?id=1604486 Differential Revision: https://phab.mercurial-scm.org/D8051

packaging: lowercase the `contrib` and `templates` directories with Inno I have no idea why these (and `contrib/vim`) were leading with uppercase with Inno, but not WiX. It probably doesn't matter too much, but might be a problem with `templates` if the user enabled case sensitivity on NTFS. Differential Revision: https://phab.mercurial-scm.org/D8063

packaging: merge the requirements.txt files for WiX and Inno Now that the content is common, there's no need to have separate files. The content still differs from the non-Windows platforms though. Differential Revision: https://phab.mercurial-scm.org/D8066

packaging: bundle dulwich, keyring, and pywin32-ctypes with WiX too TortoiseHg installs these, which is possibly where they originated (though I would have thought it more likely to be in the WiX installer, given its heritage). When I was working on the TortoiseHg app for Mac (which uses the similar `py2app`), it wasn't possible to use the keyring extension (even externally) without bundling this keyring package into the app. Assuming the same principle applies here, these would enable some common extensions. One of the things that the TortoiseHg packager on macOS does now is it adds the user's local `site-packages` directory to `sys.path`. That would allow the user to install these critical modules in cases like this. But that can probably wait for py3 packaging. The only difference in the installed packages that I see now is WiX also bundles distutils for some reason. I suppose that's not harming anything, so I'm not touching it. The only orphans in the install directories when comparing WiX and Inno now is the Copying.txt vs COPYING.rtf, the two uninstaller files for Inno, and a `Mercurial.url` file in Inno. I have no idea what that is, and it has *.ini syntax with a single field pointing to the Mercurial homepage. Differential Revision: https://phab.mercurial-scm.org/D8062