automation: shore up rebooting behavior There was a race condition in the old code. Use instance.stop()/instance.start() to eliminate it. As part of debugging this, I also found another race condition related to PowerShell permissions after the reboot. Unfortunately, I'm not sure the best way to work around it. I've added a comment for now. Differential Revision: https://phab.mercurial-scm.org/D6288

automation: wait longer for WinRM connection I got a few timeouts waiting for only 120s for the WinRM connection to become available. Increasing to 180s seems to fix. I guess AWS isn't as consistent as I would like :( Differential Revision: https://phab.mercurial-scm.org/D6287

automation: wait for instance profiles and roles Otherwise there is a race condition between creating the resources and us attempting to use them / them becoming available. The role waiter API was recently introduced, so we had to upgrade the boto3 package to get it. Other packages were also updated to latest versions just because. Even with this change, I still run into issues with the IAM instance profile not being available when we attempt to create an EC2 instance using a just-created profile. I'm not sure what's going on. Possibly a bug on Amazon's end. But the new behavior is "more correct." Differential Revision: https://phab.mercurial-scm.org/D6286

automation: don't create resources when deleting things Otherwise running these commands can result in resources being created. In the case of `purge-ec2-resources`, we will create resources only to delete them immediately afterwards! With this change, `purge-ec2-resources` now no-ops if no resources exist. # no-check-commit because foo_bar function name Differential Revision: https://phab.mercurial-scm.org/D6285

automation: detach policies before deleting role You can't delete an IAM role that has attached policies. With this change, the purge-ec2-resources command now works. Differential Revision: https://phab.mercurial-scm.org/D6284

automation: only iterate over our AMIs We can't delete AMIs that we don't own. Iterating over other AMIs won't work and slows down execution. Differential Revision: https://phab.mercurial-scm.org/D6283

remotefilelog: move most setup from onetimesetup() to uisetup() All the wrappers moved in this patch check if remotefilelog is enabled before they change behavior, so it's safe to always wrap. Differential Revision: https://phab.mercurial-scm.org/D6334

remotefilelog: move most functions in onetimeclientsetup() to top level This is how most extensions seem to do it. It makes sure we don't accidentally depend on the captured ui instance. Differential Revision: https://phab.mercurial-scm.org/D6333

tests: avoid the word "dirty" to mean "not a descendant of merge base" The term "dirty" is no longer used in the code since 57203e0210f8 (copies: calculate mergecopies() based on pathcopies(), 2019-04-11). Differential Revision: https://phab.mercurial-scm.org/D6373

releasenotes: add a file in which to record release notes I've just spent a few very boring hours going through the changelog for the 5.0 release (829 commits). We only had 5 commits that used the syntax that the release notes extension expects. This commit adds a file in which we can record important changes. The file should preferably be edited in the patch that makes the important change, but it can also be edited after (I think this is an important benefit compared to the release notes extension). I'm thinking that we can rename the file from "next" to "5.1" or something when it's time, and then we'd create a new "next" file on the default branch. I've used the syntax that we use on the our wiki in the template, but I don't care much that we use any valid syntax at all. The idea is mostly to record important changes when they happen. I expect that some copy editing will be needed at release time anyway. Differential Revision: https://phab.mercurial-scm.org/D6332

record: avoid modifying the matcher passed as a method parameter No problem observed, but I remember the previous pattern causing problems with largefiles and/or subrepos. This special matcher was added in 419ac63fe29c, so directly modifying the `fail` callback was probably an oversight in 44611ad4fbd9. Differential Revision: https://phab.mercurial-scm.org/D6371

sslutil: add support for SSLKEYLOGFILE to wrapsocket I recently learned of a Firefox/Chrome feature that allows wiresharking otherwise-TLS'd network connections. Gloriously, there's a pypi module that enables this same feature on Python, so let's add support for it to Mercurial in case we need to wireshark some HTTPs connections. Differential Revision: https://phab.mercurial-scm.org/D6343

phabricator: add custom vcr matcher to match request bodies Currently when the phabricator extension's conduit output changes the tests don't notice since the default vcr matcher only matches on 'method' and 'uri', not the body. Add a custom matcher that checks the same params are in the body (ignoring ordering). vcr's in-built body matcher can't be used since it fails under py3 with a "UnicodeEncodeError" on the "€ in commit message" tests. The DREV ids have decreased since the recordings were generated against a different phabricator instance to avoid spamming mercurial-devel. Differential Revision: https://phab.mercurial-scm.org/D6347

merge with stable

absorb: be more specific when erroring out on merge commit When you have a merge commit checked out and run `hg absorb`, it would tell you abort: no mutable changeset to change That makes it sound like the problem is public commits when isn't really. Let's be more specific in this case. There was already a test case that test this, so that now prints the new message. I added a new test case that shows the old message (when a public commit is checked out). Differential Revision: https://phab.mercurial-scm.org/D6354

remotefilelog: log when we're about to fetch files I'm debugging a slow client situation and knowing how many files are in the batch request would be a nice thing. Differential Revision: https://phab.mercurial-scm.org/D6353

revset: populate wdir() by its hash or revision number It belongs to the same category as the null hash/revision, and we do handle these virtual identifiers in id()/rev() predicates. Let's do that more consistently.

sslutil: fsencode path returned by certifi (issue6132) By inspection, this is the only codepath that could be returning a string instead of a bytes on Python 3.

revset: extract private constant of {nullrev, wdirrev} set I'll add a few more users of this constant to get around wdir identifiers.

help: suggest merge() revset instead of -m/--only-merges Suggested by Dr Rainer Woitok.

tests: update annotate tests to work around simplemerge bug test-annotate.t and test-fastannotate.hg were failing with --pure since 57203e0210f8 (copies: calculate mergecopies() based on pathcopies(), 2019-04-11). It turned out to be because the pure file merge code behaved differently. I'm guessing it's the mdiff.get_matching_blocks() that behaves differently, but I haven't confirmed that. With this content in the base: a a a And this on the local side: a z a And this on the other side: a a a b4 c b6 It produced this conflict: a z a <<<<<<< working copy: b80e3e32f75a - test: c ||||||| base a ======= a b4 c b5 >>>>>>> merge rev: 64afcdf8e29e - test: mergeb I don't care enough about the pure Python code to fix it, so this patch just updates the tests to manually resolve the conflict. Differential Revision: https://phab.mercurial-scm.org/D6351

copies: delete misplaced comment The comment was added in 78d760aa3607 (duplicatecopies: do not mark items not in the dirstate as copies, 2013-03-28). It became misplaced in 3666331164bb (cmdutil: add copy-filtering support to duplicatecopies, 2014-06-07). Then the relevant code was moved far away in 754b5117622f (context: add workingfilectx.markcopied, 2017-10-15). Differential Revision: https://phab.mercurial-scm.org/D6352

phabricator: include branch in the phabread output Depends on D6301 Differential Revision: https://phab.mercurial-scm.org/D6302

phabricator: fallback to reading metadata from diff for phabread Differential Revision: https://phab.mercurial-scm.org/D6301

phabricator: include commit (node) and parent in the local:commits metadata Differential Revision: https://phab.mercurial-scm.org/D6298

copies: remove redundant filtering of ping-pong renames in _chain() We already handle the ping-pong rename case in the filtering step, so there's very little point in doing it in the chaining loop (ping-pong renames are very rare, so I'm not worried about the cost of adding it and then removing it again). Differential Revision: https://phab.mercurial-scm.org/D6344

repair: reword comments that I noticed while working on source formatting I think this is clearer, and one will also keep us from upsetting check-code when other formatting cleanups happen. Differential Revision: https://phab.mercurial-scm.org/D6339

commit: allow --interactive to work again when naming a directory (issue6131)

gendoc: nest command headers under category headers Differential Revision: https://phab.mercurial-scm.org/D6329

minirst: support subsubsubsubsections (header level 5) with marker '''' Differential Revision: https://phab.mercurial-scm.org/D6328

gendoc: guarantee that all commands were processed The new logic renders the commands belonging to each category in turn. Commands with an unregistered category are at risk of getting skipped because their category is not in the list. By comparing the list of all commands to a log of processed commands, we can detect commands with unregistered categories and fail with an error message. Differential Revision: https://phab.mercurial-scm.org/D6327

gendoc: group commands by category in man page and HTML help Make Mercurial's man page and HTML help group commands by category, and present the categories in a helpful order. `hg help` already does this; this patch uses the same metadata. This patch uses the same header level for command categories and for commands. A subsequent patch will push the command headers down one level. Differential Revision: https://phab.mercurial-scm.org/D6326

gendoc: indent loop to make next patch more legible Differential Revision: https://phab.mercurial-scm.org/D6325

contrib: have byteify-strings explode if run in Python 2 Differential Revision: https://phab.mercurial-scm.org/D6341

repair: reword comment about bookmarks logic Again, this will help auto-formatting shortly. Differential Revision: https://phab.mercurial-scm.org/D6340

monotone: fix a bogus _() wrapper that was caught when formatting code There was a spurious space after `debug`, which hid the _() inside ui.debug() from check-code. Sigh. While here, wrap things more concisely. Differential Revision: https://phab.mercurial-scm.org/D6338

commit: add ability to print file status after each successful invocation When commands.commit.post-status is enabled, `hg commit` will effectively run `hg status -mardu` after committing. It can help catch mistakes like not committing all needed files or not adding unknown files that should've been part of the just created commit.

tests: flatten repo structure in test-commit.t Let's move to parent directory before `hg init` repos, since they don't need to be nested. It makes amend/strip messages that include full path to the backup bundle shorter, for instance.

lfs: add a TODO file This is a cleaned up and reorganized list of items I sent out about a year ago. But tracking this in the repo (like the narrow extension) gives more visibility in case anyone wants to help out.

copies: make "limit" argument to _tracefile() mandatory We always pass a limit. I think the fact that it was optional was also the reason we checked ">=limit" before we used it. So now we can remove that condition too. Differential Revision: https://phab.mercurial-scm.org/D6335

localrepo: don't use defaults arguments that will never be overridden The commithook() callback will be called when the lock is released. lock.release() calls the callback without arguments, so it was quite confusing to me that this function declared extra arguments. We can just close on the variables in the outer scope instead. Differential Revision: https://phab.mercurial-scm.org/D6336

tags: avoid double-reversing a list Differential Revision: https://phab.mercurial-scm.org/D6337

updatecaches: also warm hgtagsfnodescache Now that a full update of this cache run in a reasonable amount of time, we can warm everything when during a full update.

hgtagsfnodescache: inherit fnode from parent when possible If a changeset does not update the content of `.hgtags`, it means it will use the same file-node (for `.hgtags`) as its parents. In this case we can directly reuse the parent's file-node. We use this property when updating the `hgtagsfnodescache` taking a faster path if we already have a cached value for the parents of the node we are looking at. Doing so provides a large performance boost when looking at a lot of fnodes, especially on repository with very large manifest: timing for `tagsmod.fnoderevs(ui, repo, repo.changelog.revs())` mercurial: (41907 revisions, 1923 files) before: 6.9 seconds after: 2.7 seconds (-54%) pypy: (96266 revisions, 5198 files) before: 80 seconds after: 20 seconds (-75%) mozilla-central: (463411 revisions, 272080 files) before: 7166.4 seconds after: 47.8 seconds (-99%, x150 speedup) On a copy of mozilla-try with about 35K heads ans 1.7M changesets, this moves the computation from many hours to a couple of minutes, making it more interesting to do a full warm up of this cache before computing tags (from a cold cache). There seems to be other performance low hanging fruits, like avoiding the use of changectx or a more revision centric logic. However, the new code is fast enough for my needs right now.

hgtagsfnodescache: handle nullid lookup The null revision is empty, so it `.hgtags` content is `nullid` in regards with the `hgtagsfnodescache`. Dealing with `nullid` will help with the next changeset. Before this change, feeding `nullid` to `hgtagsfnodescache.getfnode` would return a wrong result (fnode for tip).

help: register the 'gpg' command category and give it a description help.py expects extensions to register their command category in the CATEGORY_ORDER and CATEGORY_NAMES variables. Once gendoc.py orders commands by category, in the next patch, it'll assume this registration (and raise an exception on encountering any unregistered categories). Luckily, gpg is the only bundled extension with an unregistered custom category, so let's fix it. Differential Revision: https://phab.mercurial-scm.org/D6324

histedit: Speed up scrolling in patch view mode Store patchcontents into the mode state, avoiding the expensive call to ui for computing the patchcontents. Before this change in large repos histedit patch view mode can be very irresponsive.

histedit: Show file names in multiple line format

parser: fix crash by parsing "()" in keyword argument position A tree node can be either None or a tuple because x=("group", None) is reduced to x[1].

repoview: introduce a `experimental.extra-filter-revs` config The option define revisions to additionally filter out of all repository "view". The end goal is to provide and easy to way to serve multiple subset of the same repository using multiple "shares". The simplest use case of this feature is to have one view serving the public changesets and one view also serving the draft. This is currently achievable using the new `server.view` option introduced recently by Joerg Sonnenberger. However, more advanced use cases need more advanced definitions. For example some needs a view dedicated to some release branches, or view that hides security fixes to be released. Joerg Sonnenberger and I discussed this topic at the recent mini-sprint and the both of us have seen real life use cases for this. (This series got written during the same mini-sprint). The feature is fully functional, and use similar cache-fallback mechanism to ensure decent performance. However,there remaining room to ensure each share caches and hooks collaborate with each others. This will come at a later time once users start to actually test this feature on real usecase.

copies: filter out copies from non-existent source later in _chain() _changesetforwardcopies() repeatedly calls _chain(). That is very expensive because _chain() does lookups in the manifest. I hope to split up the function in two parts: 1) simple chaining, not considering end points, and 2) filter out files that don't exist in the end points (and ping-pong copies/renames). This patches gets us closer to that by moving the check for non-existent source later in the function. Now there are no more checks for "src" and "dst" in the first loop; all the filtering of invalid copies is done in the second loop. The code also looks much more consistent now. No measureable impact on `hg debugpathcopies 4.0 4.8`. That shouldn't be surprising since the only case we're doing more checks now is in case of chained copies/renames, which are quire rare in practice. Differential Revision: https://phab.mercurial-scm.org/D6277

copies: clarify mutually exclusive cases in _chain() with a s/if/elif/ If the 'b' dict has a rename from 'x' to 'y', it shouldn't be possible for 'x' to be both (a key) in 'a' and in 'src'. That would mean that 'x' is a file in the source commit and also a rename destination in the intermediate commit. But we currently don't allow renaming files onto existing files, so that shouldn't happen. So let's clarify that by using an "elif" instead of an "if". And if we did allow renaming files onto existing files, we should prefer to use the rename destination in the intermediate commit as source anyway. Differential Revision: https://phab.mercurial-scm.org/D6276

copies: delete a redundant cleanup step in _chain() The check is redundant since d5edb5d3a337 (copies: filter out copies when target is not in destination manifest, 2019-02-14). To test that hypothesis, I made this change in the commit that commit, but all tests still passed. I think the case was necessary before then, we just didn't have tests for it. Differential Revision: https://phab.mercurial-scm.org/D6275

copies: document cases in _chain() Differential Revision: https://phab.mercurial-scm.org/D6274

copies: ignore heuristics copytracing when using changeset-centric algos Differential Revision: https://phab.mercurial-scm.org/D6269

copies: move check for experimental.copytrace==<falsy> earlier I'm going to ignore experimental.copytrace when changeset-centric algorithms are required. This little refactoring makes that easier to add. Differential Revision: https://phab.mercurial-scm.org/D6268