py3: require values in changelog extras to be bytes I don't know what happened here because b436059c1cca (py3: use pycompat.bytestr() on extra values because it can be int, 2019-02-05) came about b44a47214122 (py3: use string for "close" value in commit extras, 2018-02-11). Whatever happened, we shouldn't need to convert the values to bytes now. It's better to not convert because that might cover up bugs where someone sets a unicode value in the extras and that works until the unicode value happens to contain non-ascii (at which point it will fail because `bytestr()` expects its argument to be ascii if it's unicode). Differential Revision: https://phab.mercurial-scm.org/D8332

py3: make setup.py's hgcommand() consistently return bytes Before this patch, it returned unicode when the command failed. That made e.g. `make local PYTHON=python3` fail on an obsolete commit. Differential Revision: https://phab.mercurial-scm.org/D8331

darwin: use vim, not vi, to avoid data-loss inducing posix behavior Apple's version of vim, available at opensource.apple.com/release/macos-1015.html (for Catalina, but this behavior has been there for a while) has several tweaks from the version of vim from vim.org. Most of these tweaks appear to be for "Unix2003" compatibility. One of the tweaks is that if any ex command raises an error, the entire process will (when you exit, possibly minutes/hours later) also exit non-zero. Ex commands are things like `:foo`. Luckily, they only enabled this if vim was executed (via a symlink or copying the binary) as `vi` or `ex`. If you start it as `vim`, it doesn't have this behavior, so let's do that. To see this in action, run the following two commands on macOS: ``` $ vi -c ':unknown' -c ':qa' ; echo $? 1 $ vim -c ':unknown' -c ':qa' ; echo $? 0 ``` We don't want to start ignoring non-zero return types from the editor because that will mean you can't use `:cquit` to intentionally exit 1 (which, shows up as 2 if you combine an ex command error and a cquit, but only a 1 if you just use cquit, so we can't differentiate between the two statuses). Since we can't differentiate, we have to assume that all non-zero exit codes are intentional and an indication of the user's desire to not continue with whatever we're doing. If this was a complicated `hg split` or `hg histedit`, this is especially disastrous :( Differential Revision: https://phab.mercurial-scm.org/D8321

cext: move variable declaration to the top of the block for C89 support Not sure if we still care about C89 in general, but MSVC requires this style too. Differential Revision: https://phab.mercurial-scm.org/D8304

byteify-string: resolve symlink before byteifying Otherwise the script turns symlinks into regular files.

cext-index: propagate inline_scan error in `index_deref` Before this change, revlog index corruption could be silently ignored in some situation. Differential Revision: https://phab.mercurial-scm.org/D8276

heptapod-ci: fix test paths in the listing file Now what we run the test from the root, we need to list test name from the root. Differential Revision: https://phab.mercurial-scm.org/D8275

hg-core: add a compilation error if trying to compile outside of Linux For now, we can only provide support for Linux in `hg-core`, so we have to be explicit about it in case anyone wonders why their Dirstate is suddenly broken, or why the crate does not compile (on Windows for example). Differential Revision: https://phab.mercurial-scm.org/D8246

gzip: use the stdlib version with python 3 (issue6284) It turned out that the stdlib gained the feature we missed in python 3.1. We can now use it directly.

gzip: indent the custom Gzip code We need a new conditional in the next changesets (Adding 3.8 support). We do the large code move beforehand for clarity.

gzip: rename the argument to `mtime` to match upstream python Python gained the feature we missed in 3.1. The argument name is `mtime`. We align our version for consistency.

heptapod-ci: run the test from outside the test directory This will help detecting case where this is broken.

run-tests: fix conditional when tests are run outside of `tests` The logic to find `hghave` was using `__file__` but… non absolute path in `__file__` gets wrong as soon as we change directory (thanks python… you are being helpful). The rest of the `run-tests.py` code already deal with this fine. We simply reuse the `RUNTESTDIR` variable to fix it.

tests: fix isinstance test of wrong variable 3086a8627b2970cd7fbf49bc69413c08c68d5514 changed self._case to be a list, but it was forgotten to adjust this line.

discovery: avoid wrong detection of multiple branch heads (issue6256) This fix the code using obsolescence markers to remove "to be obsoleted" heads during the detection of new head creation from push. The code turned out to not use the branch information at all. This lead changeset from different branch to be detected as new head on unrelated branch. The code fix is actually quite simple. New tests have been added to covers these cases. Differential Revision: https://phab.mercurial-scm.org/D8259

test: cleanly skip test-remotefilelog-datapack.py on policy that breaks it That tests requires the pure module to be available. Differential Revision: https://phab.mercurial-scm.org/D8245

rust-format: cleanup ancestors.rs to make rustfmt happy Differential Revision: https://phab.mercurial-scm.org/D8224

heptapod-ci: use strict module policy Without this, test can silently fallback to other compatible policy in some cases. Differential Revision: https://phab.mercurial-scm.org/D8234

histedit: fix formatting after D8150 Differential Revision: https://phab.mercurial-scm.org/D8230

rust-format: make rustfmt happy Differential Revision: https://phab.mercurial-scm.org/D8228

Added signature for changeset 6d121acbb82e

Added tag 5.3.1 for changeset 6d121acbb82e

remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269) This cause traceback on widening using narrow extension when remotefilelog is enabled. Differential Revision: https://phab.mercurial-scm.org/D8134

remotefilelog-test: glob some flaky output line This is similar to ee0959e7d435. The affected line is flaky underload, yet the final result is correct. The command involves background pre-check of output, these are not stable probably because they run in parallel in multiple process. If it become useful to start testing precise internal details of the, they will have to be tested in a more appropriate framework than `.t` tests. Differential Revision: https://phab.mercurial-scm.org/D8179

histedit: py3 fixes for curses mode Differential Revision: https://phab.mercurial-scm.org/D8150

branch: make --force work even when specifying revs The `hg branch` command accepts a `--force` parameter that allows to "set branch name even if it shadows an existing branch". However, before this patch, that didn’t work when specifying revs with `-r`.

setup: exclude the __index__ module from itself when generating This module is generated on Windows to hold all of the extension names and the help summaries, so that they are discoverable inside the py2exe zipfile. The problem is this file is generated by dumping the disabled list, and that list comes from walking the filesystem. So once an install from source into a virtualenv created this module, then next build from source from that virtualenv would also see __index__.py in the filesystem, and include it. Clearly that's wrong because this isn't a real extension, so just filter it from the list when generating it. The Mercurial installer was unaffected by this, but the TortoiseHg package was. In the final package, `hg help -v extensions` and the panel of extensions both showed it. Differential Revision: https://phab.mercurial-scm.org/D8142

tests: stabilize test-amend.t on Windows If $TESTTMP isn't quoted in this context, it ends up like `C:Temphgtests.pikkoxchild1test-amend.t-obsstore-off`. Differential Revision: https://phab.mercurial-scm.org/D8144

py3: fix EOL detection in commandserver.channeledinput This breaks TortoiseHg's email preview which sends b'\n' while readline request is issued and the loop never ends. Spotted by Matt Harbison.

shelve: fix ordering of merge labels Differential Revision: https://phab.mercurial-scm.org/D8140

shelve: add test clearly demonstrating that the conflict labels are backwards Differential Revision: https://phab.mercurial-scm.org/D8139

import: don't ignore `--secret` when `--bypass` is specified Differential Revision: https://phab.mercurial-scm.org/D8126

phabricator: fix a phabsend crash when processing a renamed binary This was a trivial fix, and some more tests are added to cover binary files. Since the old filecontext is passed in, the old name is still available. But I noticed some weirdness around what it marked as binary and not, and what is viewable in Phabricator. Those things have been flagged, and will probably take some digging. Differential Revision: https://phab.mercurial-scm.org/D8133

test: pin the number of CPU for issue4074 tests On machine with an hundreds of CPUs, the "user" CPU time reported can be inflated by the status steps. Since the test especially focus on the diff computation, we restrict the number of CPU to avoid potential issues. Differential Revision: https://phab.mercurial-scm.org/D8112

rust-dirstatemap: add `NonNormalEntries` class This fix introduces the same encapsulation as the `copymap`. There is no easy way of doing this any better for now. `hg up -r null && time HGRCPATH= HGMODULEPOLICY=rust+c hg up tip` on Mozilla Central, (not super recent, but it doesn't matter): Before: 7:44,08 total After: 1:03,23 total Pretty brutal regression! This is a graft on stable of cf1f8660e568 Differential Revision: https://phab.mercurial-scm.org/D8111

rust-dirstatemap: cache non normal and other parent set Performance of `hg update` was significantly worse since the introduction of the Rust `dirstatemap`. This regression was noticed by Valentin Gatien-Baron when working on a large repository, as it goes unnoticed for smaller repositories like Mercurial itself. This fix introduces the same getter/setter mechanism at `hg-core` level as for `set/get_dirs`. While this technique is, as previously discussed, quite suboptimal, it fixes an important enough problem. Refactoring `hg-core` to use the typestate pattern could be a good approach to improving code quality in a future patch. This is a graft of stable of 83b2b829c94e Differential Revision: https://phab.mercurial-scm.org/D8110

chgserver: spawn new process if schemes change The schemes extension updates hg.schemes table. It's technically possible for hg.repository() to look for e.g. ui.schemes instead of depending on module-local table, but I don't think the change would make much sense since [schemes] is usually specified in ~/.hgrc and thus it can be considered static data.

tests: accept new bzr message about switching branches The new version apparently prints "Switched to branch at " instead of "Switched to branch: ". Differential Revision: https://phab.mercurial-scm.org/D8106

tests: add workaround for bzr bug This started failing for me today. I guess my bzr was upgraded. Differential Revision: https://phab.mercurial-scm.org/D8105

test: simplify test-amend.t to avoid race condition Insted on relying on sleep, we could simply have the editor do the file change. This remove the reliance on "sleep" and avoid test failing on heavy load machine. To test this, I reverted the code change in 5558e3437872 and the test started failing again. This is a graft on stable of 141ceec06b55 which should have targeted for stable. Differential Revision: https://phab.mercurial-scm.org/D8103

remotefilelog-test: glob some flaky output line (issue6083) The two following lines are flaky underload, yet the final result is correct. The command involves background pre-check of output, these are not stable probably because they run in parallel in multiple process. I spent a couple of hours trying to understand the pattern and gave up. The documented intend of these tests is safely guaranteed by checking the cache content after the command. If it become useful to start testing precise internal details of the, they will have to be tested in a more appropriate framework than `.t` tests. Differential Revision: https://phab.mercurial-scm.org/D8102

resourceutil: blacken

py3: fully fix bundlepart.__repr__ to return str not bytes My previous fix did not fully fix the issue: it would attempt to use %-formatting to combine two strs into a bytes, which won't work. Let's just switch the entire function to operating in strs. This can cause a small output difference that will likely not be noticed since no one noticed that the method wasn't working at all before: if `id` or `type` are not-None, they'll be shown as `b'val'` instead of `val`. Since this is a debugging aid and these strings shouldn't be shown to the user, slightly rough output is most likely fine and it's likely not worthwhile to add the necessary conditionals to marginally improve it. Differential Revision: https://phab.mercurial-scm.org/D8091

heptapod-ci: add a job to test the rust version of Mercurial The rust version of Mercurial is not currently tested by anything else. So it get quite important that developer runs it. Differential Revision: https://phab.mercurial-scm.org/D8017

heptapod-ci: run the --pure test too These are usually rarely run by individual developper because they are slow. However it is important that they stay happy. Differential Revision: https://phab.mercurial-scm.org/D8016

heptapod-ci: run the normal test suite The usual tests should be run too. We skip the "tests-check*.t" one because their are already covered by another Ci step. Differential Revision: https://phab.mercurial-scm.org/D8015

heptapod-ci: also run the dedicated rust test for the rust code The Rust code has various standard rust test that are fast to run. So let's run them. Differential Revision: https://phab.mercurial-scm.org/D8014

heptapod-ci: run test with python3 too Python3 is the future^W present, it is important to run tests with it too. Differential Revision: https://phab.mercurial-scm.org/D8013

heptapod-ci: colorize output The run result are nicer to read with color. Differential Revision: https://phab.mercurial-scm.org/D8012

heptapod-ci: add a basic file to be able to run tests with heptapod Having this yaml file somewhere in the main mercurial repository makes it trivial for contributors using heptapod to run CI on their in-progress work. There are alot of different combination (python2/python3 pure/cext/rust/pypy) to be tested and making sure all of them are covered manually is cumbersome. Automatic CI runnig on draft really helps in that matters. We start small bu later changesets will add more step testing more of the variants. The series is targetted on stable to make it available to the widest amount of contribution possible. The definition of the docker files used for this are available here: https://dev.heptapod.net/octobus/ci-dockerfiles Differential Revision: https://phab.mercurial-scm.org/D8011

worker: manually buffer reads from pickle stream My previous fix (D8051, cb52e619c99e, which added Python's built-in buffering to the pickle stream) has the problem that the selector will ignore the buffer. When multiple pickled objects are read from the pipe into the buffer at once, only one object will be loaded. This can repeat until the buffer is full and delays the processing of completed items until the worker exits, at which point the pipe is always considered readable and all remaining items are processed. This changeset reverts D8051, removing the buffer again. Instead, on Python 3 only, we use a wrapper to modify the "read" provided to the Unpickler to behave more like a buffered read. We never read more bytes from the pipe than the Unpickler requests, so the selector behaves as expected. Also add a test case for "pickle data was truncated" issue. https://phab.mercurial-scm.org/D8051#119193 Differential Revision: https://phab.mercurial-scm.org/D8076

py3: __repr__ needs to return str, not bytes Differential Revision: https://phab.mercurial-scm.org/D8089

config: also respect HGRCSKIPREPO in the zeroconf extension Differential Revision: https://phab.mercurial-scm.org/D8075

config: also respect HGRCSKIPREPO in hgwebdir_mod Differential Revision: https://phab.mercurial-scm.org/D8074

config: also respect HGRCSKIPREPO in `dispatch._getlocal` For some reason, we are also reading the local config in that function. Differential Revision: https://phab.mercurial-scm.org/D8073

config: add a function in `rcutil` to abstract HGRCSKIPREPO We wil need to respect this environment variable in more place. Differential Revision: https://phab.mercurial-scm.org/D8072