abort: removed labels argument from abortmerge() Labels are used to label the code that belongs to `working copy` and `merge rev` in case of a conflicted state. No such labelling is required while aborting merge as conflicted parts are reverted to normal. Differential Revision: https://phab.mercurial-scm.org/D6638

py3: source-transform only call-sites of iteritems(), not definitions branchmap.branchcache, among other classes, defines a iteritems(). That currently gets replaced by items() by the source transformer. That makes it harder for extensions to work with both py2 and py3, since they have to call either items() or iteritems() on branchcache. Let's not replace definitions of iteritems() (and itervalues()) and only replace the call-sites. We need to also add an items() alias to branchcache (etc) so our transformer call-sites will find it. Differential Revision: https://phab.mercurial-scm.org/D6641

py3: fix formatting of branchmap log messages with repo.filtername=None `"%s" % None` does not work on py3. I've extracted a little function for producing a formatted message given the filter name. Differential Revision: https://phab.mercurial-scm.org/D6644

automation: correct the path separator in LIBPATH on Windows I haven't tried building the x86 installer, but happened to notice this when working on the thg installer. Experimenting in PowerShell seems to show that LIBPATH was expanded at the end, but with ':' between, it effectively corrupted `${root}\WinSDK\Lib` and the first path in LIBPATH. Differential Revision: https://phab.mercurial-scm.org/D6642

abort: added support for merge This adds support of `hg merge --abort` to `hg abort` plan. This involves refactoring `hg.merge` into two different functions removing the abort logic of `merge` from `hg.merge` and then creating a seperate `hg.abortmerge` to handle the abort logic so that the abortion of merge can be called independently. `hg.abortmerge` is then registered as `abortfunc` for the state detection API so that `commands.abort` can use it to deal with an unfinished merge operation. Results are shown as tests. Differential Revision: https://phab.mercurial-scm.org/D6588

abort: added support for unshelve This patch adds the support for shelve in `hg abort` plan. For this the logic to load a `shelvedstate` and the error handling for it had been shifted to a seperate function `_loadunshelvedstate()`. This returns a tuple with `state` file and `opts.` `hgabortunshelve()` has been created for independent calls. In case abortion of `unshelve` is called via `hg abort` the `shelvedstate` needs to be loaded seperately. This has been ensured by `_loadunshelvedstate()` `hgabortunshelve()` is then registered as `abortfunc` for state detection API. Results are shown as tests. Differential Revision: https://phab.mercurial-scm.org/D6579

unshelve: changed Corruptedstate error msg from ui.warn to error.Abort This changes the message type of Corruptedstate error in case of `hg unshelve --abort` to error.Abort from warning message. This is done so as to avoid the return statement after the warning. Differential Revision: https://phab.mercurial-scm.org/D6636

mq: fix for merge detection methods Differential Revision: https://phab.mercurial-scm.org/D6548

py3: store _origdoc as str Since __doc__ is str, it seems natural that _origdoc also is. Differential Revision: https://phab.mercurial-scm.org/D6623

copies: follow copies across merge base without source file (issue6163) As in the previous patch, consider these two histories: @ 4 'rename x to y' | o 3 'add x again' | o 2 'remove x' | | o 1 'modify x' |/ o 0 'add x' @ 4 'rename x to y' | o 3 'add x again' | | o 2 'modify x' | | | o 1 'add x' |/ o 0 'base' We trace copies from the 'modify x' commit to commit 4 by going via the merge base (commit 0). When tracing file 'y' (_tracefile()) in the first case, we immediately find the rename from 'x'. We check to see if 'x' exists in the merge base, which it does, so we consider it a valid copy. In the second case, 'x' does not exist in the merge base, so it's not considered a valid copy. As a workaround, this patch makes it so we also attempt the check in mergecopies's base commit (commit 1 in the second case). That feels pretty ugly to me, but I don't have any better ideas. Note that we actually also check not only that the filename matches, but also that the file's nodeid matches. I don't know why we do that, but it was like that already before I rewrote mergecopies(). That means that the rebase will still fail in cases like this (again, it already failed before my rewrite): @ 4 'rename x to y' | o 3 'add x again with content X2' | o 2 'remove x' | | o 1 'modify x to content X2' |/ o 1 'modify x to content X1' | o 0 'add x with content X0' Differential Revision: https://phab.mercurial-scm.org/D6604

copies: filter invalid copies only at end of pathcopies() (issue6163) copies._filter() filters out copies whose source file does not exist in the start commit or whose target file does not exist in the end commit. We do that after chaining copies with dirstate copies or backward renames from another branch. We also do at the end of the changeset-centric copy tracing. The filtering means that we will remove copies to/from files that did not exist in some intermediate commit. That is inconsistent with what we do if a file has been deleted and then re-added (we allow updating across that). Copying the two first examples from issue6163: @ 4 'rename x to y' | o 3 'add x again' | o 2 'remove x' | | o 1 'modify x' |/ o 0 'add x' @ 4 'rename x to y' | o 3 'add x again' | | o 2 'modify x' | | | o 1 'add x' |/ o 0 'base' When doing `hg rebase -r 1 -d 4` in the first case, it succeeds, but `hg rebase -r 2 -d 4` in the second case does not. That's because we chain and filter via commit 0, which does not have file 'x' in the second case. IMO, that's clearly inconsistent. So this patch removes the filtering step so it only happens at the end. If a file was temporarily removed, whether via a merge base or not, it will now still be considered the same file. That fixes issue6163 for the changeset-centric case. Differential Revision: https://phab.mercurial-scm.org/D6603

copies: inline _chainandfilter() to prepare for next patch Differential Revision: https://phab.mercurial-scm.org/D6602

copies: remove most early returns from pathcopies() and _forwardcopies() I want to split up _chainandfilter() more so the call to _filter() consistently happens at the end of pathcopies(). This prepares for that change. Differential Revision: https://phab.mercurial-scm.org/D6601

copies: move short-circuiting of dirstate copies out of _forwardcopies() I'd like to move the filtering of copies we do after chaining to the end of all chaining (in a single place in pathcopies()). One problem that came up when trying that was that we allow things like `hg cp -f <file> <existing file>` so the user can later amend that in. Filtering at the end would mean that we remove those copies. That would break `hg st -C`. This patch therefore moves the short-circuiting of dirstate copies into pathcopies() so we can more easily handle the dirstate-only case differently. I initially thought this might change some behavior when the user does `hg status --rev 'wdir()' --rev .` during an uncommitted merge, since _backwardrenames() would reverse the copies in that case. However, I couldn't come up with a test case where it made a difference. Differential Revision: https://phab.mercurial-scm.org/D6600

tests: add more tests of copy tracing with removed and re-added files We had a test where the destination of a copy was removed and then added back. This patch adds similar cases where the break in history instead happens to the source file. There are three versions of this: 1. The break happens before the rename. 2. The break happens on a branch parallel to the rename (where copy tracing is done via the merge base) 3. The source is added on each side of the merge base. The break in history is thus in the form of a deletion when going backwards to the merge base and the re-add happens on the other branch. I've also added calls to `hg graft` in these cases to show the breakage in issue 6163. Another factor in these cases is matching nodeid (checked in copies._tracefile()). I've made two copies each of the cases to show the impact of that. One of these is the same as a test in test-rename-merge1.t, so I also deleted that test from there. Some of these tests currently fail, where "fail" is based on my current thinking of how things should work. I had initially thought that we should be more strict about not tracing copies across commits where the file did not exist, but issue 6163 made me reconsider. The only test case here that behaved differently in 4.9 is the exact case reported in issue 6163. Differential Revision: https://phab.mercurial-scm.org/D6599

tests: split out tests for unrelated copy source/target into separate file I've realized only recently how many cases there are where a file is treated differently if it's considered "related" to another file (not deleted and re-added). I'll add more tests for some of these cases soon. Differential Revision: https://phab.mercurial-scm.org/D6598

subrepos: make last line of prompts <40 english chars (issue6158) Differential Revision: https://phab.mercurial-scm.org/D6572

largefiles: make last line of prompts <40 english chars (issue6158) Differential Revision: https://phab.mercurial-scm.org/D6571

rust-dirstate: add helper to iterate ancestor paths This is modeled after std::path::Path::ancestors(). find_dirs(b"") yields b"" because Mercurial's util.finddirs() works in that way, and the test case for DirsMultiset expects such behavior.

tests: update test-commit-interactive.t for no-execbit platforms These changes correspond with f802a75da585. Differential Revision: https://phab.mercurial-scm.org/D6624

abort: added support for histedit This patch adds the support for `histedit` in `hg abort` plan. As seperate `hgaborthistedit()` function is created to handle independent calls for abortion of `histedit`. This function is then registered as `abortfunc` for state detection API. hg abort in case of `histedit` also supports ` history-editing-backup` config option. Results are shown as tests. Differential Revision: https://phab.mercurial-scm.org/D6582

abort: added support for rebase This adds support of `rebase` to `hg abort` plan. An independent abort logic for `rebase` is created under `abortrebase()` function. For this a seperate `rebaseruntime` object is created under the function to handle an unfinished `rebasestate` and abort that using abort logic under `_prepareabortorcontinue`. Results of tests are shown. Differential Revision: https://phab.mercurial-scm.org/D6568

abort: added support for graft This adds support of `graft` to `hg abort` plan. The patch creates a seperate function `cmdutil.hgabortgraft` so that abort logic for graft can be called independently. This logic is registered to the statedetection API as `abortfunc`. Results are shown as tests. Differential Revision: https://phab.mercurial-scm.org/D6567

abort: added logic for of hg abort This is part of `GSoC19` project `Implement abort and continue commands`. This patch is part of the `abort plan`. This adds the basic logic for `hg abort`. This command aborts an multistep operation like graft, histedit, rebase, merge and unshelve if they are in an unfinished state. The first part of the logic is determining the unfinished operation from the state detection API under `statemod`. This API is extended to support `hg abort` by adding a method to register the abort logic as a function (here `abortfunc`). Once the unfinished operation is determined the registered logic is used to abort the command. The benefit of this kind of framework is that any new extension developed can support `hg abort` by registering the command and logic under statedetection API. `hg abort` currently supports `--dry-run/-n` flag only. It is used to dry run `hg abort` Further patches sequentially add support for `graft`, `rebase`, `unshelve`, `histedit` and `merge`. Differential Revision: https://phab.mercurial-scm.org/D6566

merge with stable

merge: disallow merge abort in case of an unfinished operation (issue6160) This patch disallows `hg merge --abort` in case an operation of higher precedence i.e unshelve, rebase, histedit are in unfinished states. This is done so as to avoid partial abort of these operations in case merge abort is called at an interrupted step. The patch adds a `cmdutil.getunfinishedstate` function which checks for operations under progress and returns a `statecheck` object for it. Differential Revision: https://phab.mercurial-scm.org/D6607

relnotes: document new range-select mechanism in crecord Differential Revision: https://phab.mercurial-scm.org/D6622

statecheck: updated docstrings related to afterresolvedstates Differential Revision: https://phab.mercurial-scm.org/D6606

extdata: avoid crashing inside subprocess when we get a revset parse error Differential Revision: https://phab.mercurial-scm.org/D6616

extdata: demonstrate bad behavior when a subprocess emits garbage Differential Revision: https://phab.mercurial-scm.org/D6615

py3: don't run source transformer on hgext3rd (extensions) It's unclear why the source transformer runs on hgext3rd. It's been like that since it was introduced in 1c22400db72d (mercurial: implement a source transforming module loader on Python 3, 2016-07-04), and that commit didn't say anything about it (but it says that it doesn't have "support [...] for extensions"). I find that the current handling of hgext3rd just makes it harder to convert extensions to Python 3. It makes you convert a bunch of strings passed to getattr() and kwargs[] to r'' that could otherwise have been left alone. It's also really confusing that the source transformer runs when you import the extension as "extensions.foo=", but not as "extension.foo=/some/path". I suppose there is small number of (very simple) extensions that would have worked without this patch that would now be broken. It seems okay to me to break those. Differential Revision: https://phab.mercurial-scm.org/D6614

crecord: provide 'X' as a range-select mechanism Differential Revision: https://phab.mercurial-scm.org/D6621

crecord: make KEY_ENTER usable in tests (by not updating UI) Differential Revision: https://phab.mercurial-scm.org/D6620

crecord: fix if -> elif when handling key presses This shouldn't actually change any behavior, I only noticed it since I started using KEY_UP in tests, and it was complaining when it got down to the ^L handler that initscr hadn't been called yet. Differential Revision: https://phab.mercurial-scm.org/D6619

crecord: add "x" alias for space, remove test-only "TOGGLE" alias Differential Revision: https://phab.mercurial-scm.org/D6618

crecord: stop using test-only "X" as alternative for "c" Differential Revision: https://phab.mercurial-scm.org/D6617

graft: moved abortgraft and readgraft to cmdutil This patch moves `abortgraft` and `readgraft` to `cmdutil`. Various callers are updated accordingly. This is done because these serve as ulitlity functions for command `graft` and so that new functions regarding graft can be built from them. Differential Revision: https://phab.mercurial-scm.org/D6608

cleanup: use named constants for second arg to .seek() Differential Revision: https://phab.mercurial-scm.org/D6556

patch: use a short, fixed-size message for last line of prompt (issue6158) See issue6158 and the previous commit for examples of what might go wrong if we have some combinations of readline version and terminal and need to wrap the line. Briefly: readline may not display the beginning of the last line of the prompt, or it may print over it with the end of the prompt, making it difficult for users to know what's going on. Differential Revision: https://phab.mercurial-scm.org/D6563

filemerge: make last line of prompts <40 english chars (issue6158) I've chosen <40 as the target so that other languages that may have a 2x blowup in character count can still have a chance to fit into an 80 column screen. Previously, we would show a prompt like: ``` keep (l)ocal [dest], take (o)ther [source], or leave (u)nresolved for some/potentially/really/long/path? ``` On at least some systems, if readline was in use then the last line of the prompt would be wrapped strangely if it couldn't fit entirely on one line. This strange wrapping may be just a carriage return without a line feed, overwriting the beginning of the line; example (100 columns wide, 65 character filename, and yes there's 10 spaces on the end, I assume this is to handle the user inputting longest word we provide as an option, "unresolved"): ``` ng/dir/name/that/does/not/work/well/with/readline/file.txt? ave (u)nresolved for some/lon ``` In some cases it may partially wrap onto the next line, but still be missing earlier parts in the line, such as below (60 columns wide, 65 character filename): ``` rev], or leave (u)nresolved for some/long/dir/name/that/do s/not/work/well/with/readline/file.txt? ``` With this fix, this looks like this on a 60 column screen: ``` tool vim_with_markers (for pattern some/long/dir/name/that/d oes/not/work/well/with/readline/file.txt) can't handle binar y tool meld can't handle binary tool vim_with_markers can't handle binary tool internal:merge3 can't handle binary tool merge can't handle binary no tool found to merge some/long/dir/name/that/does/not/work /well/with/readline/file.txt file 'some/long/dir/name/that/does/not/work/well/with/readli ne/file.txt' needs to be resolved. You can keep (l)ocal [working copy], take (o)ther [merge rev ], or leave (u)nresolved. What do you want to do? ``` Differential Revision: https://phab.mercurial-scm.org/D6562

Added signature for changeset 97ada9b8d51b

Added tag 5.0.2 for changeset 97ada9b8d51b

posix: always seek to EOF when opening a file in append mode Python 3 already does this, so skip it there. Consider the program: #include <stdio.h> int main() { FILE *f = fopen("narf", "w"); fprintf(f, "narf\n"); fclose(f); f = fopen("narf", "a"); printf("%ld\n", ftell(f)); fprintf(f, "troz\n"); printf("%ld\n", ftell(f)); return 0; } on macOS, FreeBSD, and Linux with glibc, this program prints 5 10 but on musl libc (Alpine Linux and probably others) this prints 0 10 By my reading of https://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html this is technically correct, specifically: > Opening a file with append mode (a as the first character in the > mode argument) shall cause all subsequent writes to the file to be > forced to the then current end-of-file, regardless of intervening > calls to fseek(). in other words, the file position doesn't really matter in append-mode files, and we can't depend on it being at all meaningful unless we perform a seek() before tell() after open(..., 'a'). Experimentally after a .write() we can do a .tell() and it'll always be reasonable, but I'm unclear from reading the specification if that's a smart thing to rely on. This matches what we do on Windows and what Python 3 does for free, so let's just be consistent. Thanks to Yuya for the idea.

tweakdefaults: make hg resolve require --re-merge flag to re-merge Pulkit suggested it in https://phab.mercurial-scm.org/D4379, and a discussion with Octobus people reminded me that people still use the error-prone default behavior of `hg resolve`. Differential Revision: https://phab.mercurial-scm.org/D6610

unshelve: rename _dounshelve() to dounshelve() This is a follow-up patch to 3de4f17f4824. Differential Revision: https://phab.mercurial-scm.org/D6605

rust: remove Deref in favor of explicit methods Differential Revision: https://phab.mercurial-scm.org/D6593

rust: simplify overly complicated expression Differential Revision: https://phab.mercurial-scm.org/D6592

rust: run rfmt on all hg-core/hg-cpython code Differential Revision: https://phab.mercurial-scm.org/D6591

move: --force flag forcibly moves, not copies

copy: correct synopsis by making SOURCE a required argument

debugrevlog: fix average size computation for empty data (issue6167) If the file has no full snapshot (eg: was always empty), `hg debugrevlog` would fails when trying to compute their average size.

changelog: fix handling of empty copy entries in changeset Before this patch, when an empty value was found in the changeset, we would get a ValueError, which would result in None being returned for addedfiles/removedfiles and p1copies/p2copies. That made 278dcb24e535 (copies: write empty entries in changeset when also writing to filelog, 2019-04-23) ineffective at helping the read path not look for copies in the filelogs. Differential Revision: https://phab.mercurial-scm.org/D6595

relnotes: document the new --force-close-branch flag Differential Revision: https://phab.mercurial-scm.org/D6590

py3: hack around inconsistency of type of name passed to DNSQuestion I don't like this patch but this is the easiest way I could fix it. There are some callers which pass name which is bytes, some pass name which is str. I just encode() that if that's str. This does makes test-paths.t pass, but I am not confident whether the whole of zeroconf will work on py3 or not. Differential Revision: https://phab.mercurial-scm.org/D6511

py3: add r'' prefixes and do ('%d' % int) instead of str(int) This addresses more failures related to zeroconf on py3. Differential Revision: https://phab.mercurial-scm.org/D6510

zeroconf: port to Python 3 Since we're using the source transformer on Python 3, calls into Zeroconf and return values from it are generally bytes. But various socket functions require str on Python 3. This commit contains enough changes to coerce test-paths.t into passing on Python 3. I suspect there are still a handful of bugs on Python 3. But the tests do pass. Differential Revision: https://phab.mercurial-scm.org/D5805

copies: return only path from _tracefile() since that's all caller needs Differential Revision: https://phab.mercurial-scm.org/D6587

extensions: add shelve to _builtin This is a follow-up patch to 3de4f17f4824. This adds `shelve` to `extensions._builtin` so that the shelve extension is silently ignored. Differential Revision: https://phab.mercurial-scm.org/D6589

merge with stable

bookmarks: backout the attempt to fix the delete race This backs out 044045dce23a because it broke a bunch of tests on Windows. Yuya's theory is that we still rely on in-memory changelog data to be flushed out of the transaction.

automv: access status fields by name, not index Differential Revision: https://phab.mercurial-scm.org/D6586

automv: use public API for getting copies Differential Revision: https://phab.mercurial-scm.org/D6585

commit: add --force-close-branch flag to close a non-head changeset While closing branch from a changeset which is not a branch head current implementation abort this action in every case but, there can be the situations where the changeset is not a local head but could be a remote head. This patch adds the functionality to bypass the "abort: can only close branch heads" by introducing --force-close-branch flag. Test case changes demonstrate the new functionality added. Differential Revision: https://phab.mercurial-scm.org/D6490

shelve: move shelve extension to core Until now, `shelve` was bootstrapped as an extension. This patch adds `shelve` on core. Differential Revision: https://phab.mercurial-scm.org/D6553

shelve: remove rebase.clearstatus() This is a follow-up patch to c829749e7639. After this, shelve will be no longer dependent on rebase. This removes rebase.clearstatus() from shelve. Differential Revision: https://phab.mercurial-scm.org/D6584

shelve: removed redundant merge detection method Differential Revision: https://phab.mercurial-scm.org/D6547

rust-dirstate: call new "dirs" rust implementation from Python This is a simple module attribute replacement, will take precedence over the Python and C implementations. Differential Revision: https://phab.mercurial-scm.org/D6395

rust-dirstate: add "dirs" rust-cpython binding There is an obvious performance and memory issue with those bindings on larger repos as it copies and allocates everything at once, round-trip. Like in the previous patch series, this is only temporary and will only get better once we don't have large data structures going to and from Python. Differential Revision: https://phab.mercurial-scm.org/D6394

rust-dirstate: add "dirs" Rust implementation Following the work done in d1786c1d34fa and working towards the goal of a complete Rust implementation of the dirstate, this rewrites the `dirs` class. There is already a C implementation, which relies heavily on CPython hacks and protocol violations for performance, so I don't expect this to perform as well for now, as this is very straight-forward code. The immediate benefits are new high-level documentation and some unit tests. Differential Revision: https://phab.mercurial-scm.org/D6393

relnotes: added description about statemod._statecheck Differential Revision: https://phab.mercurial-scm.org/D6557

statecheck: shifted defaults to addunfinished() This shifts the definitions and defaults of `_statecheck()` class to `addunfinished()` registration method. Differential Revision: https://phab.mercurial-scm.org/D6583

statecheck: added support for cmdutil.afterresolvedstates This removes `afterresolvedstates` from `cmdutil` and adds support for it in `_statecheck` class. A new flag `continueflag` is added to the class to check whether an operation supports `--continue` option or not. Tests remain unchanged. Differential Revision: https://phab.mercurial-scm.org/D6551

statecheck: added support for STATES This removes `STATES` from `state.py` and adds support to `statecheck` class to handle its features. `getrepostate()` function is modified accordingly. This adds a method 'cmdutil.addunfinished()' for appending to the unfinishedstate list so as to keep 'merge' and 'bisect' at the last. This also makes two separate message formats for `checkunfinished()` and `getrepostate()` as there were previously present. Results of test changed are shown. Differential Revision: https://phab.mercurial-scm.org/D6503

state: moved cmdutil.STATES and utilities to state.py This commit moves `cmdutil.STATES` and adjoining functions to `state.py`. The existing users are updated accordingly. Tests remain unchanged. Differential Revision: https://phab.mercurial-scm.org/D6502

state: created new class statecheck to handle unfinishedstates For the purpose of handling states for various multistep operations like `hg graft`, `hg histedit`, `hg bisect` et al a new class called statecheck is created .This will help in having a unified approach towards these commands and handle them with ease. The class takes in 4 basic arguments which include the name of the command, the name of the state file associated with it , clearable flag , allowcommit flag. This also also adds the support of`checkunfinished()` and `clearunfinished()` to the class. Tests remain unchanged. Differential Revision: https://phab.mercurial-scm.org/D6501

states: moved cmdutil.unfinishedstates to state.py This moves `cmdutil.unfinishedstates`, `checkunfinished()`,`clearunfinished()` to `state.py`. the already existing users of this module are updated accordingly. Test results remain unchanged. Differential Revision: https://phab.mercurial-scm.org/D6484

rebase: fix in-memory rebasing of copy of empty file Classic Python mistake of unintentionally treating None and empty string the same. Differential Revision: https://phab.mercurial-scm.org/D6570

tests: demonstrate broken in-memory rebase of copy to empty file Differential Revision: https://phab.mercurial-scm.org/D6569

zsh: enable completion support for chg as well When verifying this change, you may need to clear/rebuild the completion cache; I did this by deleting the ~/.zcompdump file and then starting a new shell. Differential Revision: https://phab.mercurial-scm.org/D6574

py3: make catapult usable from the test runner in py3 Differential Revision: https://phab.mercurial-scm.org/D6577

py3: use integer division for the value passed to xrange Differential Revision: https://phab.mercurial-scm.org/D6576

pycompat: make fewer assumptions about sys.executable There are many Python "bundlers" which create an archive to run a Python binary from, and they may not set sys.executable at all - handle that case properly, especially to run tests. Differential Revision: https://phab.mercurial-scm.org/D6575

update: fix spurious unclean status bug shown by previous commit The crux of the problem is: - the dirstate is corrupted (the sizes/dates are assigned to the wrong files) - because when worker.worker is used with a return value (batchget in merge.py here), the return value when worker.worker effectively parallelizes is permuted - this is because worker.worker's partition of input and combination of output values are not inverses of one another: it split [1,2,3,4,5,6] into [[1,3,5],[2,4,6]], but combines that into [1,3,5,2,4,6]. Given that worker.worker doesn't call its function argument on contiguous chunks on the input arguments, sticking with lists means we'd need to know the relation between the inputs of worker.worker function argument (for instance, requiring that every input element is mapped to exactly one output element). It seems better to instead switch return values to dicts, which can combined reliably with a straighforward restriction. Differential Revision: https://phab.mercurial-scm.org/D6581

tests: show bug in update introduced in 87a34c767384 As reported by Martin at https://phab.mercurial-scm.org/D6475. Differential Revision: https://phab.mercurial-scm.org/D6580

copies: document how 'copies' dict instances are reused We avoid copying these instances as much as we can, so it's not obvious what's safe to do with them. This patch tries to explain what is safe and what is not. Differential Revision: https://phab.mercurial-scm.org/D6578

copies: simplify merging of copy dicts on merge commits After we removed some filtering in 35d674a3d5db (copies: don't filter out copy targets created on other side of merge commit, 2019-04-18), we will always include all entries from "copies1", so we can simplify the code based on that. Differential Revision: https://phab.mercurial-scm.org/D6561

copies: remove a redundant matcher filtering in _changesetforwardcopies() We filter before pushing items on the queue, so we don't need to filter after popping. Differential Revision: https://phab.mercurial-scm.org/D6560

copies: delete obsolete comment in _changesetforwardcopies() IIRC, the comment applied to the filtering we did before 35d674a3d5db (copies: don't filter out copy targets created on other side of merge commit, 2019-04-18). Differential Revision: https://phab.mercurial-scm.org/D6559

merge with stable

help: add a missing blank line to unhide `revlog-compression` The help was output, but it was elided with "Enabled by default" from the previous item.

copies: avoid reusing the same variable for two different copy dicts "childcopies" is initally the copies the current changeset to one of its children and then we reassign it with the copies from the start of the chain to the child. Let's use different names for these two things. Differential Revision: https://phab.mercurial-scm.org/D6564

drawdag: don't crash when writing copy info to changesets When writing copies to the changeset, localrepo.commitctx() will call ctx.p1copies() and ctx.p2copies(). These crashed on simplecommitctx because they ended up trying to access the manifest. drawdag doesn't support copies at all, so we can simply override the methods to return empty dicts. Differential Revision: https://phab.mercurial-scm.org/D6565

merge with stable

bookmarks: actual fix for race condition deleting bookmark This is a simple but efficient fix to prevent the issue tested in `test-bookmarks-corner-case.t`. It might be worth pursuing a more generic approach where filecache learn to depend on each other, but that would not be suitable for stable. The issue is complicated enough that I documented the race and its current solution as inline comment. See this comment for details on the fix.

localrepo: introduce a `_refreshchangelog` method See next changeset for usage and documentation for details.

bookmarks: actually trigger the race deleting bookmark in the test The previous committed version of the test did not triggered the race, but this was hidden by a strange behavior from the test runner. So we are moving the test to a slightly more complex that actually trigger the issue.

test: add some assert in the bookrace extension This cannot hurt to have a bit more security in the test extension.

test: factor out the "wait" logic in bookrace The test is currently not testing the race it is supposed to test. The synchronisation is still valid, but needs to run at a different point. We start with extracting the synchronisation logic for clarity.

test: remove dead code in the bookrace extension This code is the remain of a previous version of the code. It is never ran, so we can remove it.

run-tests: stop matching line for missing feature Before this change, the following unified test input would silently pass $ echo foo foo (false !) After this change, the "foo" output is properly detected as unexpected. The output of an handful of test had to be updated from broken conditional (that ended up working by chance).

log: pass getcopies() function instead of getrenamed() to displayer (API) This reduces the duplication between the two displayer functions (and between them and scmutil.getcopiesfn()). It's still more code than two patches ago, but there's less duplication. Differential Revision: https://phab.mercurial-scm.org/D6546

copies: create helper for getting all copies for changeset There are a few places where we get all the copies for a changeset (at least the {file_copies} template and in two places in `hg log --copies` code). These places currently call scmutil.getrenamedfn() to get a caching "getrenamed" function. They all use it in a similar way. We will be able to reuse more code by having a function for getting all the copies for a changeset. This patch introduces such a function. It uses it in the {file_copies} template to show that it works. It relies on the existing scmutil.getrenamedfn() for caching in the filelog-centric case. Differential Revision: https://phab.mercurial-scm.org/D6545

logcmdutil: also check for copies in null revision and working copy It's safe (and fast) to look for copies in the null revision, and it's incorrect not to look for them in the working copy, so let's look in both places. Differential Revision: https://phab.mercurial-scm.org/D6544

tests: demonstrate missing copy information in working copy with graphlog Differential Revision: https://phab.mercurial-scm.org/D6543

remotefilelog: handle copies in changesets in getrenamedfn() override E.g. the {file_copies} template keyword didn't work with copies in changesets before this patch because remotefilelog overrides the getrenamedfn() and didn't handle the changeset-centric case. Differential Revision: https://phab.mercurial-scm.org/D6542

remotefilelog: check if RFL is enabled in getrenamedfn() override In 8a0e03f7baf4 (remotefilelog: move most setup from onetimesetup() to uisetup(), 2019-05-01), I said: All the wrappers moved in this patch check if remotefilelog is enabled before they change behavior, so it's safe to always wrap. That was clearly a lie, because getrenamedfn() didn't. That made e.g. `hg log -T {file_copies}` unbearably slow. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D6541

relnotes: document template support for `hg root` Differential Revision: https://phab.mercurial-scm.org/D6540

remotefilelog: tell runbgcommand to not block on child process startup These two invocations will always find a binary because they're re-running hg. As a result, we can skip waiting for the subprocess to start running and save a little bit of wall-time. Differential Revision: https://phab.mercurial-scm.org/D6539

procutil: allow callers of runbgcommand to assume the process starts Experimentally starting the subprocess can take as much as 40ms, and for some of our use cases that's frivolous: we know the binary will start, and if it doesn't we'd only ever ignore it and continue anyway. This lets those use cases be faster. Differential Revision: https://phab.mercurial-scm.org/D6537

shallowrepo: remove backwards compat code that predates in-tree remotefilelog Differential Revision: https://phab.mercurial-scm.org/D6538

commit: make the error message more specific while aborting branch closing Differential Revision: https://phab.mercurial-scm.org/D6493

commit: add a check if it is trying to close an already closed branch head It would check if the revision we are going to close is already a closed branch head and print the error message accordingly. Differential Revision: https://phab.mercurial-scm.org/D6491

strip: move checksubstate() to mq (its only caller) Differential Revision: https://phab.mercurial-scm.org/D6536

strip: use bailifchanged() instead of reimplementing it This also means that we get the standard error messages (see changed test cases). Differential Revision: https://phab.mercurial-scm.org/D6535

strip: remove unused excsuffix argument from checklocalchanges() It was only used by mq, and mq now has its own copy of the function. Differential Revision: https://phab.mercurial-scm.org/D6534

mq: remove dependency on strip's checklocalchanges() Some of the functionality in strip.checklocalchanges() was only used by mq, so let's move it to mq so we can clean up strip. Differential Revision: https://phab.mercurial-scm.org/D6533

copies: avoid calling matcher if matcher.always() When storing copy information in the changesets (experimental.copies.read-from=changeset-only), this patch speeds up hg debugpathcopies FENNEC_58_0_2_BUILD1 FIREFOX_59_0b8_BUILD2 from 5.9s to 4.7s. At the start of this series (b162229e), that command took 18min. Differential Revision: https://phab.mercurial-scm.org/D6422

copies: avoid unnecessary copying of copy dict When storing copy information in the changesets, this patch speeds up hg debugpathcopies FENNEC_58_0_2_BUILD1 FIREFOX_59_0b8_BUILD2 from 11s to 5.9s. That command takes 6.2s when storing copy information in filelogs. Differential Revision: https://phab.mercurial-scm.org/D6421

copies: don't filter out copy targets created on other side of merge commit If file X is copied to Y on one side of merge and the other side creates Y (no copy), we would not mark that as copy. In the changeset-centric pathcopies() version, that was done by checking if the copy target existed on the other branch. Even though merge commits are pretty uncommon, it still turned out to be too expensive to load the manifest of the parents of merge commits. In a repo of mozilla-unified converted to storing copies in changesets, about 2m30s of `hg debugpathcopies FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE` is spent on this check of merge commits. I tried to think of a way of storing more information in the changesets in order to cheaply detect these cases, but I couldn't think of a solution. So this patch simply removes those checks. For reference, these extra copies are reported from the aforementioned command after this patch: browser/base/content/sanitize.js -> browser/modules/Sanitizer.jsm testing/mozbase/mozprocess/tests/process_normal_finish_python.ini -> testing/mozbase/mozprocess/tests/process_normal_finish.ini testing/mozbase/mozprocess/tests/process_waittimeout_python.ini -> testing/mozbase/mozprocess/tests/process_waittimeout.ini testing/mozbase/mozprocess/tests/process_waittimeout_10s_python.ini -> testing/mozbase/mozprocess/tests/process_waittimeout_10s.ini Since these copies were created on one side of some merge, it still seems reasonable to include them, so I'm not even sure it's worse than filelog pathcopies(), just different. Differential Revision: https://phab.mercurial-scm.org/D6420

copies: do full filtering at end of _changesetforwardcopies() As mentioned earlier, pathcopies() is very slow when copies are stored in the changeset. Most of the cost comes from calling _chain() for every changeset, which is slow because it needs to read manifests. It needs to read manifests to be able to filter out copies that are were created in one commit and then deleted. (It also filters out copies that were created from a file that didn't exist in the starting revision, but that's a fixed revision across calls to _chain(), so it's much cheaper.) This patch changes from _chainandfilter() to just _chain() in the main loop in _changesetforwardcopies(). It instead removes copies that have subsequently been removed by using ctx.filesremoved(). We thus rely on that to be fast. It timed this command in mozilla-unified: hg debugpathcopies FIREFOX_59_0b3_BUILD2 FIREFOX_BETA_59_END It took 18s before and 1.1s after. It's still faster when copy information is stored in filelogs: 0.70s. It also still gets slow when there are merge commits involved, because we read manifests there too. We'll deal with that later. Differential Revision: https://phab.mercurial-scm.org/D6419

rust-filepatterns: add comment about Windows path handling As I replied to the Phabricator message, this is wrong. And I even suspect it wouldn't compile because of multiple type mismatches. I think, in Rust where type system is rock solid, we can live with UTF-8 strings except for the bottom storage layer and the top UI/command layer. We'll still have to get around undecodable characters not to be lost, but I think it's okay to drop such filenames from match result if they don't match in UTF-8 world, not in Latin-1 world.

rust-filepatterns: silence warning of non_upper_case_globals

rust: update Cargo.lock to include @generated comment cargo 1.34.0 of Debian sid inserts this comment, and I'm tired of reverting the change every time I do make local. https://github.com/rust-lang/cargo/commit/bd0e4a08471b8bc7957829b4fd294b8985d4fa2d

merge with stable

cborutil: fix streamencode() to handle subtypes Otherwise the template filter 'cbor' could crash because of bytes subclass: ValueError: do not know how to encode <class 'mercurial.encoding.safelocalstr'>

lfs: correct an error in the TODO file

cat: don't prefetch files unless the output requires it It's a waste to cache lfs blobs when cat'ing the raw data at best, but a hassle debugging when the blob is missing. I'm not sure if there are other commands that have '{data}' for output, and if there's a general way to prefetch on that keyword. It's interesting that the verbose output seems to leak into the JSON output, but that seems like an existing bug.

tracing: add support for emitting counters Differential Revision: https://phab.mercurial-scm.org/D6526