byteify-strings: fix misalignment with multi-line parenthesis This improves the current fix to also take into account cases where the last line ended on the opening `(`, `[` or `{` and adds a regression test.

byteify-strings: add test for byteify-strings.py This tests the basic features expected from this script, some cases may not be covered yet. A future patch will demonstrate an issue with multi-line `(`, `[` and `{` alignment and propose a fix.

merge with stable

automation: push changes affecting .hgtags When I went to build the 5.1 tag using the in-repo automation, the automatic version calculation failed to deduce the clean 5.1 version string because we had only pushed the changeset corresponding to the 5.1 tag and not the changeset containing the 5.1 tag. So from the perspective of the remote repo, the 5.1 tag didn't exist yet and automatic version deduction failed. This commit changes the `hg push` to also push all changesets affecting the .hgtags file, ensuring the remote has up-to-date tags information. I tested this by creating a local draft changeset with a dummy tag value on a different DAG head and instructed the automation to build a revision that didn't have this change to .hgtags. The tag was successfully pushed and the built package had a version number incorporating that tag. Sending this to stable so the 5.1.1 automation hopefully "just works."

byteify-strings: add cli argument to handle `attr*()` when they are methods Certain code bases have useful utils that wrap the builtin functions, and are called like `util.setattr`.

bookmarks: actual fix for race condition deleting bookmark This is a simple but efficient fix to prevent the issue tested in `test-bookmarks-corner-case.t`. It might be worth pursuing a more generic approach where filecache learn to depend on each other, but that would not be suitable for stable. The issue is complicated enough that I documented the race and its current solution as inline comment. See this comment for details on the fix.

strip: access bookmark before getting a reference to changelog Bookmark access might invalidate the current changelog (to make sure both are in a reasonable synchronisation state). So we should grab the reference to changelog after we access bookmark. Otherwise we risk using a dead object for the whole strip process. (note: this dead object business probably requires a new layers of checking)

test: use a more verbose output in the test While debugging some test failure, I released the test never checks if the relevant changesets were preserved. So I am updating the test from `hg parents` usage to `hg log -G` with a special template. This increase the area covered by the test and clarify the test failures.

byteify-strings: simplify default value for `--treat-as-kwargs`

byteify-strings: add --treat-as-kwargs argument to handle kwargs-like objects This argument will help extensions move to Python 3 as keyword arguments should not be byte-prefixed. Most of the time, code bases will call this object `kwargs`, but other conventions exist like `opts`, so it should make sense to allow for custom names. This is a best effort solution that does minimal static checking; cases like `options = [o for o in ('a', 'b', 'c') if kwargs.get(o)]` and other just as complicated will not be detected.

byteify-strings: add helpers to check for item access or method call These helpers will be used in a future patch, split for ease of review.

byteify-strings: add support for ignore comments Our simple token analysis is sometimes not clever enough, we need to be able to turn off our script for parts of the code. This change introduces three special comments: - `#no-py3-transform` to tell `byteify-strings` ignore the next line - `#py3-transform: off` to ignore everything until the end of the file - `#py3-transform: on` to stop ignoring The last two can be particularly useful within Python 2/3 compatibility files.

byteify-strings: handle triple quoted strings if they are not docstrings As with anything in this script, this is a best effort approach. Most of the time, when a triple quoted string is assigned to something, it's not a docstring.

byteify-strings: handle multi-line strings in _ensuresysstr The current implementation did not handle calls like `repo.ui.log("first line" "other line")` correctly.

fix: run fixer tools in the repo root as cwd so they can use the working copy This lets fixer tools do things like find configuration files, with the caveat that they'll only see the version of that file in the working copy, regardless of what revisions are being fixed. Differential Revision: https://phab.mercurial-scm.org/D6440

config: add defaultvalue template keyword This patch tries to fix one of the issues mentioned in issue6014. This adds a new `defaultvalue` template keyword to be used with `hg showconfig` to get the default value of the config item. Differential Revision: https://phab.mercurial-scm.org/D6704

merge with stable

module-policy: update rust extension import to use the new module policy Differential Revision: https://phab.mercurial-scm.org/D6677

transaction: leave unfinished without crashing when not properly released I think the transaction.__del__ is there just as a last resort in case we (or an extension) forgot to release the transaction. When that happens, the repo can (or will on Python 3?) get deleted before the transaction. This leads to a crash in test-devel-warnings.t on Python 3 because we tried to access repo.dirstate, where repo was retried from a weak reference. There's not much we can do here, but let's at least avoid the crash. The user will have run `hg recover` afterwards regardless. Differential Revision: https://phab.mercurial-scm.org/D6664

unshelve: add abort on using continue and interactive together `unshelve --continue --interactive` will not work as expected by the user as the mode of in-progress unshelve is preserved and cannot be overwritten. This patch makes `unshelve` to throw an error on using both `--continue` and `--interactive` together with `unshelve`. Differential Revision: https://phab.mercurial-scm.org/D6703

py3: add one more test to list of passing tests Found by buildbot. Differential Revision: https://phab.mercurial-scm.org/D6702

tests: sort imports in test-bookmarks-corner-case.t test-check-module-imports.t breaks on py3 without this change. Differential Revision: https://phab.mercurial-scm.org/D6701

Added signature for changeset e91930d712e8

Added tag 5.1 for changeset e91930d712e8

fix: add some new test cases These cover a couple of behaviors we were testing at Google that weren't covered here before. Differential Revision: https://phab.mercurial-scm.org/D6698

unshelve: store information about interactive mode in shelvedstate This is a follow-up patch to 5162753c4c14. This makes `unshelve` stores the information about interactive mode on conflicts. Differential Revision: https://phab.mercurial-scm.org/D6679

automation: execute powershell when connecting For some reason, the ability to execute PS scripts appears to come online after the ability to execute regular command scripts. This is creating race conditions when connecting to instances resulting in our wait_for_winrm() returning before PS is available leading to an exception being thrown in other code. Let's change the client connection code to execute a minimal PS script so we can try to trap the exception in wait_for_winrm().

automation: allow exit code of 1 for `hg push` `hg push` exits 1 for no-ops. No-op pushes should be fine in the context of automation.

curses: do not setlocale() at import time (issue5261) setlocale() can break date formatting/parsing functions because they are locale dependent. We should avoid doing setlocale() as possible. This patch moves setlocale() just before curses.wrapper(), which function is documented to "initialize curses." I don't know the details about the curses initialization, but I *think* this would work as well. Maybe we can extract a curses setup function later. https://www.mercurial-scm.org/pipermail/mercurial-devel/2019-February/128788.html

unshelve: create a matcher only if required on creating unshelve ctx Differential Revision: https://phab.mercurial-scm.org/D6687

unshelve: changes how date is set on interactive mode On an interactive unshelve, the remaining changes are shelved again for later. This patch modifies the date of remaining shelved change to the time of interactive shelve. Differential Revision: https://phab.mercurial-scm.org/D6685

unshelve: unify logic around creating an unshelve changeset This is a follow-up patch to 5162753c4c14 on addressing reviews on the commit. This unifies the logic around creating an unshelve changeset. Differential Revision: https://phab.mercurial-scm.org/D6683

fix: ignore fixer tool configurations that are missing patterns This is to prevent a crash under the same circumstances. This is also to avoid data loss due to accidental application of a fixer tool to all files, if the matching logic somehow changed to that effect. Affecting all files until otherwise configured would be dangerous, and not very useful. We shouldn't abort because there may be other fixers, and it may still be useful to run them without having to adjust configuration. A user might not feel confident in changing configs, for example. Differential Revision: https://phab.mercurial-scm.org/D6693

fix: add a test case around the effect of cwd on pattern matching This was not covered by previous tests. It is related to a regression encountered at Google due to misconfiguration of [fix]. Differential Revision: https://phab.mercurial-scm.org/D6692

fix: remove support for :fileset sub-config in favor of :pattern Differential Revision: https://phab.mercurial-scm.org/D6691

fsmonitor: add support for extra `hg debuginstall` data This might make some things easier to debug, and for default bug report templates it'll help collect more data from users all at once. I don't actually need fsmonitor in our bug reports (we don't use it), but this demonstrates the utility of the preceding patches without having to add new things to core. Differential Revision: https://phab.mercurial-scm.org/D6682

debugcommands: add support for extensions adding their own debug info We've had a couple of cases where it'd be handy at Google to add data to `hg debuginstall`'s output. We've kludged around that at various times, but it seems reasonable to let extensions add their own data here so extension maintainers can get useful extra data. Differential Revision: https://phab.mercurial-scm.org/D6681

fsmonitor: refactor watchmanclient.client to accept ui and repo path This will make my next patch simpler. Differential Revision: https://phab.mercurial-scm.org/D6680

contrib: install Python 3.8b2 instead of 3.8a2 Let's install the most recent Python 3.8 distribution. Differential Revision: https://phab.mercurial-scm.org/D6674

automation: make Windows base image name configurable Since automation broke in the middle of the 5.0 release cycle, there's a good chance it will break again in the future. While a robust solution might be to search for all available images and choose the newest one, it does seem useful to be able to explicitly choose the name of the image to find and use so users can opt in to using a different image. This commit implements that functionality. Differential Revision: https://phab.mercurial-scm.org/D6673

automation: extract strings to constants Differential Revision: https://phab.mercurial-scm.org/D6672

automation: use newer Windows base image It looks like the old base image disappeared. Let's use a newer image that exists today. Differential Revision: https://phab.mercurial-scm.org/D6671

copies: fix crash on in changeset-centric tracing from commit to itself When we trace copies from a changeset to itself, the "work" queue ends up empty and we hit the "assert False" after it. It was only the last of the three added tests that failed before this patch. That is because the other two cases have fast paths, so _committedforwardcopies() is never reached. Differential Revision: https://phab.mercurial-scm.org/D6675

unshelve: add help text on --interactive in verbose mode This is a follow-up patch to rHG9eace8d6d537. This modifies the help text of unshelve in verbose mode to mention the details about `--interactive` flag. Differential Revision: https://phab.mercurial-scm.org/D6676

amend: stop committing unrequested file reverts (issue6157) Differential Revision: https://phab.mercurial-scm.org/D6667

amend: add a test for a simplified version of issue6157 Differential Revision: https://phab.mercurial-scm.org/D6666

py: error out if a "skip" character was given with non-dict to util.dirs() util.dirs() keeps track of the directories in its input collection. If a "skip" character is given to it, it will assume the input is a dirstate map and it will skip entries that are in the given "skip" state. I think this is used only for skipping removed entries ("r") in the dirtate. The C implementation of util.dirs() errors out if it was given a skip character and a non-dict was passed. The pure implementation simply ignored the request skip state. Let's make it easier to discover bugs here by erroring out in the pure implementation too. Let's also switch to checking for the dict-ness, to make the C implementation (since that's clearly been sufficient for many years). This last change makes test-issue660.t pass on py3 in pure mode, since the old check was for existence of iteritems(), which doesn't exist on py3. Differential Revision: https://phab.mercurial-scm.org/D6669

py3: fix incorrect fix of test-setdiscovery.t in eb27d9eee2cc Both places should have been changed from 185 to 187. Differential Revision: https://phab.mercurial-scm.org/D6668