test-copies: test that copies' sidedata does not get corrupted during push This is an important usecase. Differential Revision: https://phab.mercurial-scm.org/D10348

test-copies: test that copies' sidedata does not get corrupted during pull This is an important usecase. Differential Revision: https://phab.mercurial-scm.org/D10347

test-copies: simplify some conditional output Now that all computation using sidedata give the same result, we can simplify conditional Differential Revision: https://phab.mercurial-scm.org/D10346

sidedata: move documentation about sidedata helpers to sidedata module Differential Revision: https://phab.mercurial-scm.org/D10361

sidedata: move sidedata-related utils to the dedicated module Differential Revision: https://phab.mercurial-scm.org/D10360

sidedata: replace sidedata upgrade mechanism with the new one Note: this is split into a separate change (like some other patches in this series) because it's not easy to have all patches work 100% and this seemed easier for reviewers. When cloning or upgrading a repo, we may need to compute (or remove) sidedata. This is the same mechanism that is used in exchange, so we re-use the new system to simplify the code and fix the remaining issues (correctly dropping flags and handling partial removal, etc.). This also highlighted an issue with `test-copies-in-changeset.t` that kept sidedata categories that are not relevant anymore. They should probably be dropped entirely, but that would be for another patch. Differential Revision: https://phab.mercurial-scm.org/D10359

sidedata: add a way of replacing an existing sidedata computer This will be useful in a future patch to replace a sequential computer with a parallel computer. We only allow for explicit replacement, to force the users to think about overriding computers. Differential Revision: https://phab.mercurial-scm.org/D10358

bundle2: remove restriction around sidedata We are now capable of generating the missing sidedata on-the-fly. Differential Revision: https://phab.mercurial-scm.org/D10345

heptapod-ci: enable pytype checking The trigger is manual for now, until we get a better idea about stability, resource usage, etc. Differential Revision: https://phab.mercurial-scm.org/D10462

hghave: fix the definition of `python3` to work on Windows Both py2 and py3 executables are named `python.exe`, and may or may not be on PATH. So use the dispatcher executable that comes with py3 to fetch the version of the latest py3 executable. This allows at least one relnotes test to run on Windows. Differential Revision: https://phab.mercurial-scm.org/D10694

util: avoid echoing the password to the console on Windows py3 (issue6446) The `getpass.getpass()` implementation on Windows first checks if `sys.stdin` and `sys.__stdin__` are the same object. It's not on py3 because the former is replaced in dispatch.py with something that doesn't normalize '\n' to '\r\n'. When they aren't the same object, it simply calls `sys.stdin.readline()` instead of the mscvrt functions that read the input characters before they are echoed. This simply copies the `getpass.win_getpass()` implementation without the stdin check, and byteifies around the edges. I'm not sure if there's a reasonable replacement for the check that we could implement. When echoing input into the hg command, the `ui.interactive()` check causes `ui.getpass()` to bail before getting here. If the proper config switches are used to bypass that and call this, the process stalls until '\n' is input into the console. So there could be a deadlock here when run by another command if the wrong config settings are applied. Differential Revision: https://phab.mercurial-scm.org/D10708

sidedata: enable sidedata computers to optionally rewrite flags Sidedata computers may want to influence the flags of the revision they touch. For example, the computer for changelog-based copytracing can add a flag to signify that this revision might affect copytracing, inversely removing said flag if the information is no longer applicable. See inline documentation in `storageutil` for more details. Differential Revision: https://phab.mercurial-scm.org/D10344

cg4: introduce protocol flag to signify the presence of sidedata We need a way of signaling whether the current revision has sidedata or not, and re-using the revision flags would waste potential revlog flags and mix two normally independent layers. In this change, we add a single byte at the start of the ch4 delta header to set potential protocol flags. We also reclaim the revlog flag for sidedata, since it is no longer used, in its place now lives the (also experimental) copytracing flag. When generating deltas, apply the `CG_FLAG_SIDEDATA` flag if there is sidedata. When applying the deltas, if said flag is present, the next chunk contains the sidedata. Differential Revision: https://phab.mercurial-scm.org/D10343

changegroup: don't limit cgv4 to revlogv2 repos To help the transition from revlogv1 to revlogv2, we need to be able to enable cgv4 for revlogv1 repos, so that revlogv2 clients can handle adding/removing sidedata over the wire. Differential Revision: https://phab.mercurial-scm.org/D10342

sidedata: gate sidedata functionality to revlogv2 in more places Since revlogv1 is not capable of storing sidedata, we prevent sidedata mechanisms around the revlog layer from doing anything. We however keep the ones that allow a revlogv1 repo to generate sidedata on-the-fly on a push, the pull case simply does not add the sidedata to the revlog. Differential Revision: https://phab.mercurial-scm.org/D10341

sidedata: register copies sidedata computer regardless of the revlog version Repositories should not gate their sidedata computers based on any requirement, only their wanted sidedata. A repository might need to generate sidedata wanted by the peer that it itself does not want. Differential Revision: https://phab.mercurial-scm.org/D10340

revlog: replace the old `revlog_kind` approach with the new `target` one The new `target` attribute supersedes the previous one. Differential Revision: https://phab.mercurial-scm.org/D10353

revlog: introduce an explicit tracking of what the revlog is about Since the dawn of time, people have been forced to rely to lossy introspection of the index filename to determine what the purpose and role of the revlog they encounter is. This is hacky, error prone, inflexible, abstraction-leaky, <insert-your-own-complaints-here>. In f63299ee7e4d Raphaël introduced a new attribute to track this information: `revlog_kind`. However it is initialized in an odd place and various instances end up not having it set. In addition is only tracking some of the information we end up having to introspect in various pieces of code. So we add a new attribute that holds more data and is more strictly enforced. This work is done in collaboration with Raphaël. The `revlog_kind` one will be removed/adapted in the next changeset. We expect to be able to clean up various existing piece of code and to simplify coming work around the newer revlog format. Differential Revision: https://phab.mercurial-scm.org/D10352

config: add --source option to include source of value Showing the source of each config option is quite useful and not something the user should have to reach for the `--debug` flag for. I updates documentation and tests, except for one place in `test-hgrc.t` where I thought the test might have been intended to also test that `--debug` results in `ui.quiet` etc being test. Differential Revision: https://phab.mercurial-scm.org/D10668

rewriteutil: say how many commits would become orphan if commit is rewritten This copies the message from the evolve extension, but modifies it a bit to work with the grammar (in particular with the use of "change branch of" as `action`). I don't know why it doesn't use the same `_formatrevs()` as for public commmits. Differential Revision: https://phab.mercurial-scm.org/D10671

rewriteutil: give examples of public changesets that can't be rewritten This patch copies the feature from the evolve extension. Differential Revision: https://phab.mercurial-scm.org/D10670

convert: update p4 for Python 3 Differential Revision: https://phab.mercurial-scm.org/D10703

tests: change the fixer commands to use the buffer attribute on stdio objects Otherwise `\r` was getting injected into the fixed lines and throwing off the commit hashes on Windows when the fixer is invoked with py3. Differential Revision: https://phab.mercurial-scm.org/D10637

tests: stabilize test-persistent-nodemap.t on Windows Several issues here: - Hooks can't invoke shell scripts on Windows, so use `sh` to launch - `dd` in MSYS only recognizes `status=noxfer` - The `PATH` updating triggered a massive slowdown, but is no longer needed I have no idea why, but removing the `PATH` update substantially increased the speed of the test. It was running finishing at ~4:30 with `--debug` and ~14:50 without it, but now completes in ~2:20 on my Windows laptop. Differential Revision: https://phab.mercurial-scm.org/D10636