debugdiscovery: allow to select random seed during debugdiscovery run The randomness can lead to large timing difference, controling it is important.

discovery: move cl.hasnode outside of the for-loop IIUC, resolving attributes for changelog can lead to some overhead. So this patch moves that to outside of a for-loop. Differential Revision: https://phab.mercurial-scm.org/D6147

discovery: prevent deleting items from a dictionary Removing elements from Python dictionary is expensive. So let's prevent adding them instead. I added a newline to make code look a bit better. Differential Revision: https://phab.mercurial-scm.org/D6146

discovery: drop some unused sets Differential Revision: https://phab.mercurial-scm.org/D6145

discovery: prevent recomputing info about server and outgoing changesets We already iterate over the outgoing.missing above and lookup repo for them. So let's reuse info calculated at that time instead of recomputing that again. Also we calculate the set of remotebranches by doing set(remotemap), so let's reuse that again. Upcoming patches will clean things a bit more. Differential Revision: https://phab.mercurial-scm.org/D6144

crecord: draw on the whole screen When starting crecord, one can see that it has a small gap on the rightmost column which doesn't get used. This is in contrast to other interactive curses frontends such as chistedit. Disabling the displaying of the cursor allows drawing on the whole availabe area and thus some hacky code in align() could be removed. Differential Revision: https://phab.mercurial-scm.org/D6171

automation: perform tasks on remote machines Sometimes you don't have access to a machine in order to do something. For example, you may not have access to a Windows machine required to build Windows binaries or run tests on that platform. This commit introduces a pile of code intended to help "automate" common tasks, like building release artifacts. In its current form, the automation code provides functionality for performing tasks on Windows EC2 instances. The hgautomation.aws module provides functionality for integrating with AWS. It manages EC2 resources such as IAM roles, EC2 security groups, AMIs, and instances. The hgautomation.windows module provides a higher-level interface for performing tasks on remote Windows machines. The hgautomation.cli module provides a command-line interface to these higher-level primitives. I attempted to structure Windows remote machine interaction around Windows Remoting / PowerShell. This is kinda/sorta like SSH + shell, but for Windows. In theory, most of the functionality is cloud provider agnostic, as we should be able to use any established WinRM connection to interact with a remote. In reality, we're tightly coupled to AWS at the moment because I didn't want to prematurely add abstractions for a 2nd cloud provider. (1 was hard enough to implement.) In the aws module is code for creating an image with a fully functional Mercurial development environment. It contains VC9, VC2017, msys, and other dependencies. The image is fully capable of building all the existing Mercurial release artifacts and running tests. There are a few things that don't work. For example, running Windows tests with Python 3. But building the Windows release artifacts does work. And that was an impetus for this work. (Although we don't yet support code signing.) Getting this functionality to work was extremely time consuming. It took hours debugging permissions failures and other wonky behavior due to PowerShell Remoting. (The permissions model for PowerShell is crazy and you brush up against all kinds of issues because of the user/privileges of the user running the PowerShell and the permissions of the PowerShell session itself.) The functionality around AWS resource management could use some improving. In theory we support shared tenancy via resource name prefixing. In reality, we don't offer a way to configure this. Speaking of AWS resource management, I thought about using a tool like Terraform to manage resources. But at our scale, writing a few dozen lines of code to manage resources seemed acceptable. Maybe we should reconsider this if things grow out of control. Time will tell. Currently, emphasis is placed on Windows. But I only started there because it was likely to be the most difficult to implement. It should be relatively trivial to automate tasks on remote Linux machines. In fact, I have a ~1 year old script to run tests on a remote EC2 instance. I will likely be porting that to this new "framework" in the near future. # no-check-commit because foo_bar functions Differential Revision: https://phab.mercurial-scm.org/D6142

contrib: PowerShell script to install development dependencies Configuring a Windows machine to hack on Mercurial is a bit of work and it isn't documented very well. This commit introduces a PowerShell script to automate going from a fresh Windows install to an environment suitable for building Mercurial, its installers, and running tests. Differential Revision: https://phab.mercurial-scm.org/D6141

chistedit: change in-progress message Saying "running histedit" is an artifact of when chistedit was a separate thing from histedit. I found the message a bit confusing, since wasn't I running histedit from the beginning, just from the curses interface? The whole thing is now histedit, both the curses interface and the underlying procedure to apply a plan, so let's use a message that doesn't make a distinction.

perf: copyedit a few documentation strings Differential Revision: https://phab.mercurial-scm.org/D6170

shelve: add --keep to list of allowables

perf: introduce a `perf.run-limits` options This options make it possible to configure the number of run that the extensions will perform. This is useful for automated benchmark or for performance measurement that need better accuracy.

perf: pass limits as a function argument The function applying the limit has no access to the configuration. Therefore, some higher layer will have to pass it as argument. We do this in an independent change to clarify the next change.

perf: more flexible implementation for checking stop conditions We want to make this logic simpler to configure. The first step is to stop hard-coding every values.

perf: document perfparents

perf: document config options We have configuration, so we better document it.

tests: use "perf" as a the extension name in test-contrib-perf.t This is simpler and seems more "inline" with the name people usually use for this extensions.

shelve: do not update when keeping changes, just move the dirstate This is to leave the working directory unchanged. We reuse the match object as an optimisation.

shelve: refactor _shelvecreatedcommit's match object into calling site We might need to use this match object again to move the dirstate in case the user requested to `--keep` the changes.

shelve: new keep option Does nothing yet. The docstring describes what it will soon be doing.

diff: support diffing explicit files in subrepos Most other commands support implied recursion based on file names already.

fix: make the order of the work queue deterministic This makes any output generated during the parallel phase of execution stable if parallelism is disabled. This helps write tests like that in the future. Differential Revision: https://phab.mercurial-scm.org/D6166

fix: allow fixing untracked files when given as arguments It's more useful to fix the file than to silently avoid it when the user does this: hg fix --working-dir untracked-file We will still do nothing to ignored files, even if they are specified. This may be safer, given that such files can unexpectedly slip into the arguments via a shell glob or fileset. Differential Revision: https://phab.mercurial-scm.org/D6165

branchcache: have a hasnode function to validate nodes Upcoming patches will delay node validation until it's required. So we need to a way in branchcache class to check whether a node exists or node. Other options were making repo or changelog an attribute of branchcache object. But the branchcache depends on filters so I decided to pass a function object. Differential Revision: https://phab.mercurial-scm.org/D6157

branchcache: add attributes to track which nodes are verified Half of the cost of loading branchcache comes from verifiying all the nodes it has. We don't need to verify all the nodes in all the cases. Sometimes we need to verify only a set of nodes for a set of branches. We can ignore nodes of other branches as we are not going to read them. This patch introduces two attributes to branchcache class. _verifiedbranches is a set which will tell the branches for which it's head nodes are verified. _closedverified is a boolean which will tell whether all closednodes are verified or not. Another idea which I had was to keep a set of nodes which are verified, but it will be more ugly and difficult to track things on node level. Differential Revision: https://phab.mercurial-scm.org/D6156

branchcache: make entries a private attribute Differential Revision: https://phab.mercurial-scm.org/D6155

branchcache: introduce hasbranch() This will be used to check whether a branch exists or not. This will optimized in future. Differential Revision: https://phab.mercurial-scm.org/D6154

branchmap: drop branchcache.setdefault() (API) All the callers are updated to call setdefault of branchcache.entries Differential Revision: https://phab.mercurial-scm.org/D6153

branchcache: rename itervalues() to iterheads() The itervalues() exists because branchcache() had a dict interface. Since it no longer has a dict interface, it makes sense to have better function names. If a person does not understand how branchcache stores info, it will be hard for them to guess what itervalues() does. Differential Revision: https://phab.mercurial-scm.org/D6152

branchmap: remove the dict interface from the branchcache class (API) The current branchmap computation involves reading the whole branchmap from disk, validating all the nodes even if they are not required. This leads to a lot of time on repos which have large branchmap or a lot of branches. On large repos, this can validate around 1000's of nodes. On some operations, like finding whether a branch exists or not, we don't need to validate all the nodes. Or updating heads for a single branch. Before this patch, branchcache class was having dict interface and it was hard to keep track of reads. This patch removes the dict interface. Upcoming patches will implement lazy loading and validation of data and implement better API's. Differential Revision: https://phab.mercurial-scm.org/D6151