backout: fix --no-commit option (issue5054)

bundle: exit early when there are no commits to bundle Previously, if you passed a revset that resolved to no nodes, it would get interpreted by the changegroup discovery logic as 'bundle all my heads', which is not what the user asked. Let's exit early when we notice this case. It could be argued that the changeset discovery logic should be smarter and only assume 'all heads' if the incoming heads parameter is None, but that's a much riskier change.

zeroconf: access repo on hgweb_mod properly (issue5036) hgweb_mod.hgweb.repo disappeared in ae33fff17c1e (hg: establish a cache for localrepository instances) and the code for accessing repo instances from hgweb was later refactored to go through a cache-aware context manager. Adapt zeroconf to access the repo instance via the new API.

merge default into stable for 3.7 code freeze

shelve: permit shelves to contain unknown files If an emergency comes in while you're in the middle of an experimental change, it can be useful to shelve not just files hg already tracks but also your unknown files while you handle the emergency. This is especially true if you have hooks intended to prevent you from forgetting to add new code before you push. Teach "hg shelve" to optionally shelve unknown files, not just tracked files. This is functionally similar to --addremove, but with two differences: 1) Deleted files are not removed. 2) Files added during shelve creation are tracked in extra so that they can be forgotten by "hg unshelve". When unshelving, we take care to only forget files if they've been created during the unshelve operation; if you add a file that's being tracked in a shelve as an unknown file, it should not become unknown again when the shelve is unshelved.

localrepo: don't reference transaction from hook closure (issue5043) Before, the hook() closure (which is called as part of locking hooks) would maintain a reference to a transaction instance (which should be finalized by the time lock hooks are called). Because we accumulate hook() instances when there are multiple transactions per lock, this would result in holding references to the transaction instances which would lead to higher memory utilization. Creating a reference to the hook arguments dict minimizes the number of objects that are kept alive until the lock release hook runs, minimizing memory "leaks."

context: don't use util.cachefunc due to cycle creation (issue5043) util.cachefunc stores all arguments as the cache key. For filectxfn functions, the arguments include the memctx instance. This creates a cycle where memctx._filectxfn references self. This causes a memory leak. We break the cycle by implementing our own memoizing function that only uses the path as the cache key. Since each memctx has its own cache instance, there is no concern about invalid cache hits.

largefiles: actions will now always have a file - drop check

largefiles: make prompt order deterministic 42ae1b1f048f introduced iteration of a set. Make it stable.

largefiles: fix commit of missing largefiles 832c98d79587 improved merging of standin files referencing missing largefiles. It did however not test or fix commits of such merges; it would abort. To fix that, change copytostore to skip and warn about missing largefiles with a message similar the one for failing get from remote filestores. (It would perhaps in both cases be better to emit a more helpful warning like "warning: standin file for large1 references 58e24f733a which can't be found in the local store".) To test this, make sure commit doesn't find the "missing" largefile in the global usercache. For further testing, verify that update and status works as expected after this. This will also effectively backout 63116d47cc3f.

diff: don't crash when merged-in addition is copied Similar to what was explained in the previous commit, the diff code expected copy source to be in "ctx1", which is not always the case during a merge. This has been broken since before hg 2.0. Also similar to the previous commit, we fix the problem by fixing up the copy dict.

diff: don't crash when merged-in addition was removed (issue4786) During a merge, if the user removes a file that came from parent 2 and did not exist in parent 1, the file's status will be "removed". This surprises the diff code, which crashes because it expects removed files exist in parent 1. This has been broken since 377124ba6b10 (trydiff: use 'not in addedset' for symmetry with 'not in removedset', 2014-12-23). Fix by fixing up the list of removed file, similar to how we currently fix up the list of modified and added files during a merge.

diff: move status fixup earlier, out of _filepairs() This prepares for future patches, and it also lets us remove the ugly "ctx1" argument to _filepairs() (ugly because of its assymmetry -- there's no "ctx2" argument).

graft: warn when -r is combined with revisions as positional arguments The behaviour in this case is undefined. Instead of silently doing something "random" and surprising, at least issue a warning. (This should perhaps be considered a "deprecation" and turned into an error in a future release.)

graft: clarify in help that `-r` is not just optional Positional parameters are also treated as revisions, but the order of revisions matters and it will often be wrong if the user understands it as `-r` taking multiple revisions as `-r REV1 REV2`. (Alternatively, `-r` could be turned into a no-op flag as the documentation suggests. That would however be less "semantic markup" and I agree with the implementation in 55e7f352b1d3 but not the documentation.)

streamclone: use backgroundfilecloser (issue4889) Closing files that have been appended to is slow on Windows/NTFS. CloseHandle() calls on this platform often take 1-10ms - and that's on my i7-6700K Skylake processor with a modern and fast SSD. Contrast with other I/O operations, such as writing data, which take <100us. This means that creating/appending thousands of files can add significant overhead. For example, cloning mozilla-central creates ~232,000 revlog files. Assuming 1ms per CloseHandle(), that yields 232s (3:52) of wall time waiting for file closes! The impact of this overhead can be measured most directly when applying stream clone bundles. Applying these files is effectively uncompressing a tar archive (read: it's very fast). Using a RAM disk (read: no I/O wait), the difference in wall time for a `hg debugapplystreamclonebundle` for a ~1731 MB mozilla-central bundle between Windows and Linux from the same machine is drastic: Linux: ~12.8s (128MB/s) Windows: ~352.0s (4.7MB/s) Windows is ~27.5x slower. Yikes! After this patch: Linux: ~12.8s (128MB/s) Windows: ~102.1s (16.1MB/s) Windows is now ~3.4x faster. Unfortunately, it is still ~8x slower than Linux. Profiling reveals a few hot code paths that could likely be improved. But those are for other patches. This patch introduces test-clone-uncompressed.t because existing tests of `clone --uncompressed` are scattered about and adding a variation for background thread closing to e.g. test-http.t doesn't feel correct.