test-patchbomb: use mocktime The test was using system time for displaying ETAs, which could be flaky if the sysload is high. This patch extracts mocktime.py from test-progress.t to make sure test-patchbomb.t is unaffected by system time. Differential Revision: https://phab.mercurial-scm.org/D844

test-patchbomb: fix the test With the experimental config `progress.estimate` removed, the progress output in `test-patchbomb.t` has a minor change: it shows ETA since the beginning. (This could be folded into f428c3)

progress: make ETA only consider progress made in the last minute This patch limits the estimate time interval to roughly the last minute (configurable by `estimateinterval`) to be more practical. See the test change for why this is better. .. feature:: Estimated time is more accurate with non-linear progress Differential Revision: https://phab.mercurial-scm.org/D820

progress: remove progress.estimate config It was introduced by 98e4d39 ("progress: add speed format" 2011-5-9) and was intended to hide ETA information for the first few seconds. Later 5d261fd ("progress: add a changedelay to prevent parallel topics from flapping (issue2698)" 2011-6-23) introduced `changedelay` config which hides the entire progress bar for the first few seconds. So `progress.estimate` seems somehow duplicated feature-wise. Since it's experimental and duplicated, let's just remove it. This makes the next patch simpler - it no longer needs to make sure `starttimes` is the real start time. Differential Revision: https://phab.mercurial-scm.org/D828

progress: demonstrate non-linear progress has a bad ETA experience Previously, the ETA and speed assumes the progress is linear. Often, due to network or other issues, it could be fast for the most time, and suddenly slow down: [====================================================> ] \___________________________________________/\______/ very fast suddenly much slower This patch adds a test demonstrating the ETA could be way off in those cases. Differential Revision: https://phab.mercurial-scm.org/D819

copytrace: add a a new config to limit the number of drafts in heuristics The heuristics options tries to the default full copytracing algorithm if both the source and destination branches contains of non-public changesets only. But this can be slow in cases when we have a lot of drafts. This patch adds a new config option experimental.copytrace.sourcecommitlimit which defaults to 100. This value will be the limit of number of drafts from c1 to base. Incase there are more changesets even though they are draft, the heuristics algorithm will be used. Differential Revision: https://phab.mercurial-scm.org/D763

mail: encode long unicode lines in emails properly (issue5687) 3e544c074459 introduced a bug: emails Content-Transfer-Encoding is silently replaced with 'quoted-printable' while any other encoding could be used by underlying code. The problem is revealed when a long unicode line is encoded. The patch implements proper check which works for any text and encoding.

chg: show timestamp with debug messages Like `strace -tr`, this helps finding performance bottlenecks. Differential Revision: https://phab.mercurial-scm.org/D807

keepalive: add more context to bad status line errors As the TODO in the test said, the previous error message was not very helpful. Let's improve things. Differential Revision: https://phab.mercurial-scm.org/D811

tests: add interface checks for bundle, statichttp, and union peers I forgot to add these when I initially wrote the test. They inherit from localrepo.localpeer, so they should be explicitly tested. Differential Revision: https://phab.mercurial-scm.org/D810

alias: make alias command lazily resolved With many aliases, resolving them could have some visible overhead. Below is part of traceprof [1] output of `hg bookmark --hidden`: (time unit: ms) 37 \ addaliases dispatch.py:526 37 | __init__ (60 times) dispatch.py:402 33 | findcmd (108 times) cmdutil.py:721 16 | findpossible (49 times) cmdutil.py:683 It may get better by optimizing `findcmd` to do a bisect, but we don't really need to resolve an alias if it's not used, so let's make those command entries lazy. After this patch, `addalias` takes less than 1ms. .. perf:: improved performance when many aliases are defined [1]: https://bitbucket.org/facebook/hg-experimental/src/9aca0dbdbdfc48457e5d2581ca2d6e662fced2e6/hgext3rd/traceprof.pyx Differential Revision: https://phab.mercurial-scm.org/D805

alias: test duplicated definition earlier This patch moves the old definition checking logic introduced by f4b7be3f8430 earlier. So that the test itself does not depend on `aliasdef`. The check is to avoid wrapping a same alias multiple times. It can be done by checking the config name and value (`definition` in code), without constructing a `cmdalias` instance. This makes the next patch easier to review. Differential Revision: https://phab.mercurial-scm.org/D804

uncommit: add a test for uncommit with uncommitondirtywdir config in merge Differential Revision: https://phab.mercurial-scm.org/D809

largefiles: force an on-disk merge Largefiles isn't a good candidate for in-memory merge (it uses a custom dirstate, matcher, and the files might not fit in memory) so have it always run an old-style merge. Differential Revision: https://phab.mercurial-scm.org/D683

merge: allow a custom working context to be passed to update This will allow anyone to enable the first in-menmory merge milestone by wrapping merge.update in an extension and creating an overlayworkingctx. Differential Revision: https://phab.mercurial-scm.org/D682

patchbomb: add test that shows --to and --cc override matching config item As far as I know this has always been true and is intentional (it's in line with many other behaviors), but it wasn't tested.

blackbox: move _openlogfile to a separate method This removes several `stat` syscalls for accessing `_bbvfs` and makes `ui` object cleaner. Differential Revision: https://phab.mercurial-scm.org/D769

blackbox: do not prevent 'chg init' from working Previously, blackbox always appends to blackbox.log and creates the directory for that file on demand. That could be an issue if: 1. chg starts from `$REPO` directory, so `ui._bbrepo` is set. 2. `rm -rf $REPO`. 3. `chg init $REPO`, blackbox writes something and `init` will fail because `$REPO` directory is non-empty. This patch fixes that by verifying whether vfs exists before re-using it. Differential Revision: https://phab.mercurial-scm.org/D768

tests: add a test for blackbox with nested alias configurations I've observed some weirdness around this, and needed to rule some things out. There aren't any bugs in core around this, but it was nice to have confirmation.

tests: clean up blackbox test around aliases a little bit

revlog: add option to mmap revlog index Following on from Jun Wu's patch last October[1], we have found that using mmap for the revlog index in repos with large revlogs gives a noticable performance improvment (~110ms on each hg invocation), particularly for commands that don't touch the index very much. This changeset adds this as an option, activated by a new experimental config option so that it can be enabled on a per-repo basis. The configuration option specifies an index size threshold at which Mercurial will switch to using mmap to access the index. If the configuration option is not specified, the default remains to load the full file, which seems to be the best option for smaller repos. Some initial performance numbers for average of 5 invocations of `hg log -l 5` for different cache states: | Repo: | HG | FB | |---|---|---| | Index size: | 2.3MB | much bigger | | read (warm): | 237ms | 432ms | | mmap (warm): | 227ms | 321ms | | | (-3%) | (-26%) | | read (cold): | 397ms | 696ms | | mmap (cold): | 410ms | 888ms | | | (+3%) | (+28%) | [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-October/088737.html Test Plan: `hg log --config experimental.mmapindex=true` Differential Revision: https://phab.mercurial-scm.org/D477

util: add an mmapread method This is useful for large files that are only partly touched. Test Plan: Will be used and tested in a later patch. Differential Revision: https://phab.mercurial-scm.org/D476

changegroup: remove dictionary creation from deltachunk Previously delta chunk returned a dictionary. Now that we consume deltachunk within changegroup (instead of outside in revlog) we can just return a tuple and have it be returned directly by deltaiter. Differential Revision: https://phab.mercurial-scm.org/D746

bundlerepo: update to use new deltaiter api Differential Revision: https://phab.mercurial-scm.org/D745

debug: update debugbundle to use new deltaiter api Changegroup now has a deltaiter api for easy iteration over a series of deltas. Let's use that in the debugbundle command. Differential Revision: https://phab.mercurial-scm.org/D744

revlog: add revmap back to revlog.addgroup The recent c8b6ed51386b patch removed the linkmapper argument from addgroup, as part of trying to make addgroup more agnostic from the changegroup format. It turns out that the changegroup can't resolve linkrevs while iterating over the deltas, because applying the deltas might affect the linkrev resolution. For example, when applying a series of changelog entries, the linkmapper just returns len(cl). If we're iterating over the deltas without applying them to the changelog, this results in incorrect linkrevs. This was caught by the hgsql extension, which reads the revisions before applying them. The fix is to return linknodes as part of the delta iterator, and let the consumer choose what to do. Differential Revision: https://phab.mercurial-scm.org/D730

rebase: also include other namespaces in changeset description This makes use of the generic method of listing bookmarks and tags, so other extensions that add other namespaces will get their names added too. This does mean that bookmarks will come before tags, just like we apparently decided to order them in the "hg log" output. It doesn't seem like people would be parsing the rebase output anyway. We also did 79ab5369d55a (rebase: use _ctxdesc in one more place, 2017-08-29) recently, so now seems like a good time. Differential Revision: https://phab.mercurial-scm.org/D741

tersestatus: sort the dictionary before iterating on it There has report of flakiness in test-status-terse.t. In the terse code, we are iterating on a dictionary without sorting and since python dicts are unordered, that can be a reason behind the flakiness. Before we have a better implementation for the terse thing, let's make sure current implementation possess minimum bugs. Differential Revision: https://phab.mercurial-scm.org/D740

copytrace: use the full copytracing method if only drafts are involved This patch adds the functionality to use the full copytracing even if `experimental.copytrace = heuristics` in cases when drafts are involved. This is also a part of copytrace extension in fbext. This also adds tests which are also taken from fbext. .. feature:: The `heuristics` option for `experimental.copytrace` performs full copytracing if both source and destination branches contains non-public changsets only. Differential Revision: https://phab.mercurial-scm.org/D625

templates: introduce a obsfateoperation() function Differential Revision: https://phab.mercurial-scm.org/D723