tests: add a critical flush() to run-tests.py to make output stable on py3 Differential Revision: https://phab.mercurial-scm.org/D5204

merge-tools: when calling external merge tool, describe the resolve inputs It is a common complaint that a user will be running some operation (histedit, rebase, evolve, etc.), get into a merge-conflict situation, and not understand what they are seeing - it is possible that the merge tool is configured to display the hash, but it's difficult for most merge tools to display a good snippet of the description. In the worst case, configuring this template will lead to output that is immediately covered by a terminal application, maybe the user can hit ctrl-z to see it. In the common case, the output will be in a terminal window and a GUI program will start, and it should be possible to view both the terminal and the GUI program at the same time. Differential Revision: https://phab.mercurial-scm.org/D5094

templatekw: deprecate p1rev/p2rev/p1node/p2node in favor of p1/p2

templatekw: add p1/p2 keywords which switches the current ctx This is just an example of mappingdict. I have no idea what should be displayed as {p1|json}. Currently it is an empty dict since {'ctx'} isn't displayable, which is clearly useless and should be changed later.

templater: add wrapper for a single template mapping This can be used to nest template mappings without inserting a sequence-like layer. See the next patch for example.

templatekw: extract internal "{rev}:{node|formatnode}" template to constant This will be used later.

tweakdefaults: remove commands.resolve.mark-check=abort, it is too broken See issue6020 for the current case. I don't want to continue attempting to fix this on the stable branch, so I'm removing from tweakdefaults and will send fixes meant for the default branch and 4.9. Differential Revision: https://phab.mercurial-scm.org/D5225

resolve: when resolve.mark-check=abort, downgrade to warning if pats specified Previously, with --config resolve.mark-check=abort, running `hg resolve -m foo` would abort and emit a message saying to use --all. This command does not work, though: `hg resolve -m foo --all`, and it's really weird for --all to be the "--force" flag. My original goal with the option was to make it so that `hg resolve -m` (no filename arguments) was safer, which is why --all is used; in my mind, `hg resolve -m foo` should always mark it as resolved, and `--all` is how you specify "all the files", so that's why I chose `hg resolve -m --all` as the way out of `hg resolve -m` aborting. This commit makes all of this work the way it was meant to in my head :) Differential Revision: https://phab.mercurial-scm.org/D5218

test: remove changes preparing the drop of `HGMERGE` These changes are no longer necessary so we better drop them.

run-tests: define the default merge tool through configuration Using the `HGMERGE` environment variable generates confusion as it overrides any value set through configuration. By setting the default value through the default HGRC, the usual overriding rules apply and it should reduce confusion.

run-tests: explicitly declare the list of dropped environment variable It will make the list clearer and more maintainable.

test: drop usage of `HGMERGE` in `test-lfconvert.t` Using configuration instead of environment variable has a strange effect on the output, but no impact on the end result.

test: drop usage of `HGMERGE` in `test-merge-types.t` Using configuration instead of environment variable has a strange effect on the output, but no impact on the end result.

test: use the "correct" merge tool in `test-rebase-detach.t` The value of `HGMERGE` overrides the value passed through --config. We are about to drop this `HGMERGE` environment variable so we update the value used in config to match what is currently in use. This is another example of confusion introduced by the `HGMERGE` environment variable.

test: also deactivate `ui.merge` when testing merge tool We are about to introduce a default config for `ui.merge`. Since `test-merge-tools.t` is about testing merge configuration, we need to make sure it won't interfere.

tests: adjust merge tool config in test-largefiles-update.t Previously, the `ui.merge=internal:fail` configuration was shadowed by the the `HGMERGE=internal:merge` environment variable. We would like to remove this usage of `HGMERGE` variable because it is error-prone (eg: this very test). We start by cleaning up test independently to avoid large churn when the change happens.

extensions: include current version in "invalid version" message It is "not so rare" for the mercurial version to be badly detected at build time. In such case, version check for extensions gets confused. To help pinpoint the error, we now include the Mercurial's version number in the error message.

obsolete: prefetch the repo.obsstore used in phasedivergence loop The speedup is probably quite negligible, but it cannot hurt.

obsutil: prefetch method in allpredecessors loop We don't expect a massive speedup from this, but the change was laying around in my repository and it cannot hurt.

merge with stable

Added signature for changeset a91a2837150b

Added tag 4.8 for changeset a91a2837150b

beautifygraph: don't substitute anything for 'X' in rendered graphs It looks like we never actually render graphs this way, although there's an example in a comment next to the code that prevents it from being needed (see graphmod.ascii()). Differential Revision: https://phab.mercurial-scm.org/D5103

py3: do not stringify integers in revlog stats by '%s'

py3: convert revlog stats to a dict of (bytes, int) pairs Py_DECREF(t) is replaced with Py_CLEAR(t) so that t is set to NULL once decrefed. Otherwise, it would be excessively decrefed if a subsequent PyBytes_FromString() failed.

rust: fix signature of rustlazyancestors_init() function Obviously, sizeof(int) != mem::size_of::<usize>() on amd64, though the argument would be passed in 64-bit register anyway.

tests: require SQLite 3.8.3+ as sqlitestore relies on "WITH" clause The test fails on gcc112 because the SQLite is too old. https://sqlite.org/changes.html#version_3_8_3

relnotes: various tweaks for release notes Stop filtering out commits that are expected to be covered by releasenotes extension: now we want two lists, one for WhatsNew and one for ReleaseX.Y. Use `only(stoprev, startrev)` to make `relnotes -h` output be actually true about what revisions are included. More filter rules, mostly obvious. More classifying rules to have less things in "unsorted". Looks like nargs=1 was just making args.startrev and args.stoprev be lists for no reason. BC and API sections are renamed to what we're using on the WhatsNew page, and also just skipped if empty.

delta: skip "empty delta" optimisation for non-general case (issue6006) Non-general delta repository cannot delta against anything than prev. So even if the delta to prev is empty we should use it. This is similar to the change made in bafa1c4bb7a8. Differential Revision: https://phab.mercurial-scm.org/D5201

narrow: fix copies._fullcopytracing() narrowspec filtering in graft case I broke this too in 707c3804e607 (narrow: move copies overrides to core, 2018-09-28). Hopefully I'm done fixing things broken by that commit now. Differential Revision: https://phab.mercurial-scm.org/D5213

tests: demonstrate broken copies._fullcopytracing() Turns out copies._fullcopytracing() was also broken. Differential Revision: https://phab.mercurial-scm.org/D5212

narrow: make copies.pathcopies() filter with narrowspec again I broke this in 707c3804e607 (narrow: move copies overrides to core, 2018-09-28). Differential Revision: https://phab.mercurial-scm.org/D5203

tests: demonstrate broken copies.pathcopies() Differential Revision: https://phab.mercurial-scm.org/D5202

setup: explain to distutils how we write rc versions When we use a rc version number (e.g. 4.8rc0), bdist_msi is using distutils.StrictVersion to parse it into a tuple of numbers. By default, StrictVersion.version_re only recognizes [ab] for alpha/beta, where mercurial may use '-rc' or 'rc'. This change makes StrictVersion parse correctly our version numbers, so that bdist_msi doesn't fail on rc versions.

changegroup: restore default node ordering (issue6001) Changeset db5501d9 changed the default node ordering from "storage" to "linearize". While the new API is more explicit and cleaner, the "linearize" order is problematic on certain repositories like netbeans where it makes bundling slower the more nodes we bundle. Pushing and pulling 100 changesets was ~20% slower and pushing and pulling 1000 changesets was ~600% slower. A very quick analysis of profile traces showed that the pull operation was taking more time creating the delta. Putting back the old default order seems to be the safe option. With more time during the next cycle, we can understand better the impact of sorting with the DAG order by default, the source of the regression and how to mitigate it. /!\ We are still waiting for the full performance impact but with this patch, bundling and pulling locally (not on the performance workstation) 1000 changesets on the netbeans repository is as fast as before the regression. Differential Revision: https://phab.mercurial-scm.org/D5196

changegroup: introduce an explicit linear sorting We still need to linearize the revisions in some cases, introduce an explicit `linear` sorting before changing back the default order. Differential Revision: https://phab.mercurial-scm.org/D5195

py3: roll up threading.Thread constructor args into **kwargs The constructor doesn't have a `verbose` keyword argument in py3.

tests: glob over a single quote vs double quote difference on Windows

py3: port test-log-exthook.t to Python 3 For once an easy one. Differential Revision: https://phab.mercurial-scm.org/D5184

extensions: fix up many many debug logs that use %r This gets us a bunch closer on Python 3, but I'll still have to use a ton of sad globs. A previous version of this patch tried to preserve the %r formatting, but upon review Yuya noted that special characters in extension names is very unlikely, so we can just use %s and not sweat the quoting. Differential Revision: https://phab.mercurial-scm.org/D5002

blackbox: add configitem for format of log timestamps Sometimes blackbox logs are used to report performance problems, but the timestamps are only at second granularity, so often the timings have to stated separately by the reporter. This is inconvenient and error-prone, so I would like to include %f in the date format. This patch makes that possible.

narrow: replace filtering in list comprehension by set operations I didn't think of this while reviewing the patch. Differential Revision: https://phab.mercurial-scm.org/D5188

storage: update sqlitestore to use the new `deltamode` parameter While updating the filelog class, I forget to update the sqlitestore, tests are now passing with this patch. Differential Revision: https://phab.mercurial-scm.org/D5185

narrow: rework logic to check whether we need to widen and narrow This patch reworks logic which calculates whether we need to extend or narrow our working copy or not. We filter the addincludes, removeincludes, addexcludes and removeexcludes passed from user to the actual added and removed includes and excludes. What that means is a user can pass an already included path as addincludes, a path which is not included as removeincludes etc. In such situations the old logic use to think we need to do some work, whereas we don't need to do that work. In old logic, even if we don't have anything new to include but it believes we need to call widen, this adds some good amount of work on large repository. A widen calls involves computing incomming csets, calling the narrow_widen() which in non-ellipses cases goes through all the set of csets which are available which can take ~2-3 mins on large repos. Those 2-3 minutes are spend on doing nothing which a client can prevent by checking is there really anything which needs to be included. The tests changes shows that we don't go to the server anymore in such cases which is nice. Differential Revision: https://phab.mercurial-scm.org/D5183

tests: show that adding an already included path still calls narrow_widen() This patch adds tests demonstrating that we still go to the server in non-ellipses widening when we have that path already on the client and there is nothing new to download. The next patch will try to make client side logic smart and not go to the server if we don't need to download anything. Differential Revision: https://phab.mercurial-scm.org/D5182

graft: introduce --base option for using custom base revision while merging The graft command usually performs an internal merge of the current parent revision with the graft revision, using p1 of the grafted revision as base for the merge. As a trivial extension of this, we introduce the --base option to allow for using another base revision. This can be used as a building block for grafting and collapsing multiple changesets at once, or for grafting the resulting change from a merge as a single simple change. (This is kind of similar to backout --parent ... only different: this graft base must be an ancestor, but is usually *not* a parent.) This is probably an advanced use case, and we do thus not show it in the non-verbose help.

changegroup: add a option to create bundle with full snapshot only This is easy to implement now and can be useful for benchmarking.

changegroup: allow to force delta to be against p1 This new developer option is useful to general more "generic" bundle. Without this option, a bundle generated from the repository use deltas similar to the one stored in the specific repository it was generated from. This makes performance testing a bit tricky. Using deltas similar to the final result means all delta stored in the bundle can be applied to the target repository without any further processing (except for the rare case of a full snapshot). The application of such bundles (almost) never exercises the (slower) path of searching for a new valid delta. This result in unrealistic and too favorable timing and profile. Instead, we introduce an option to make sure all revisions are stored as a delta against p1. It might not be the best generation option, but it guarantees that the content will be "generic", not favoring a specific target.

fix: disable use of thread-based worker getfixes() accesses to repo, changectx, filectx, etc., so I believe there are code paths triggering data race. Mercurial API isn't thread safe in general.

storage: also use `deltamode argument` for ifiledata Now that lower level uses such argument, we can propagate the change to higher layers.

configitems: rename the config to prevent adding an alias in future Right now the config option looks like: [experimental.server] stream-narrow-clones= which does not match how config options are generally defined in core. So let's rename this to: [experimental] server.stream-narrow-clones= before the new release so that we don't have to add an alias in future for this. Differential Revision: https://phab.mercurial-scm.org/D5198

sparse-revlog: only refine delta candidates in the sparse case (issue6006) Starting with 5aef5afa8654, a valid delta parent might be "refined". This allows repository using sparse-revlog to produce better delta chain by using better intermediate snapshot base. However, this refining step was performed in all cases, including for repository not using sparse-revlog. This could produce a strange chain in the general delta case and corrupted repository in the non-general delta case. We now skip this step unless sparse-revlog is in use. In issue 6006, Yuya Nishihara provided a test case using an external repository, so we did not include it. Finding "laboratory" condition to reproduce this case and implementing an efficient test reproducing it is a bit tricky. We do not foresee to have the time to provide one by the release date. Differential Revision: https://phab.mercurial-scm.org/D5197

changegroup: refactor emitrevision to use a `deltamode` argument This new argument gathers the semantic of `sendfulltext` and `deltaprevious` in a single value. We are about to introduce a new type of constraints. Avoiding yet another argument sounds like a plus.

http: work around custom http client classes that refuse extra attrs I have no idea what is going on with our custom http client code at Google, but it chokes on these extra attributes we're tucking on http clients. Since it feels more than a little wrong to just stuff extra data on a client, let's degrade gracefully when the client class refuses the attributes.

branchmap: do not specify changelog as an argument Since (unfiltered)repo.changelog lookup gets as fast as __dict__ lookup, there's no point to pass in changelog instance. $ hg perfbranchmap --clear-revbranch -R mozilla-central ! base (orig) wall 20.593091 comb 20.600000 user 20.520000 sys 0.080000 (best of 3) (this) wall 20.129126 comb 20.130000 user 20.020000 sys 0.110000 (best of 3) This backs out most of the changes in 76d4272bd57b and 47c03042cd1d.

filecache: unimplement __set__() and __delete__() (API) Implementing __set__() implies that the descriptor can't be overridden by obj.__dict__, which means any property access involves slow function call. "Data descriptors with __set__() and __get__() defined always override a redefinition in an instance dictionary. In contrast, non-data descriptors can be overridden by instances." https://docs.python.org/2.7/reference/datamodel.html#invoking-descriptors This patch basically backs out 236bb604dc39, "scmutil: update cached copy when filecached attribute is assigned (issue3263)." The problem described in issue3263 (which is #3264 in Bugzilla) should no longer happen since repo._bookmarkcurrent has been moved to repo._bookmarks.active. We still have a risk of introducing similar bugs, but I think that's the cost we have to pay. $ hg perfrevset 'branch(tip)' -R mercurial (orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66) (prev) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81) (this) wall 0.099038 comb 0.110000 user 0.100000 sys 0.010000 (best of 93)

filecache: use try-except for faster __dict__ lookup Python function call is slow, and the cost could be significant here. $ hg perfrevset 'branch(tip)' -R mercurial (orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66) (this) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81)

crecord: make nextsametype() check that parent item exists (issue6009) Items that represent files in curses interface don't have parents.

help: describe what ui.tweakdefaults changes, concretely Currently, one has to look at the code. A couple things are suboptimal: - probably not translatable - lines don't get wrapped (a couple are a bit too long) but it seems to better this way than without help at all. Differential Revision: https://phab.mercurial-scm.org/D5187

logexchange: convert paths to unix when detecting the active path This fixes the problem in the tests[1] where Windows was showing the whole path as the remotename for local repositories. Somebody with a better understanding of this extension should probably take a deeper look. There may be other cases that need to be converted- specifically the `elif not instance` and the missing `else` cases in activepath(). I also noticed when adding debug prints that the absolute path is stored in the file, probably not normalized. (It's wrapped up in $TESTTMP.) [1] https://buildbot.mercurial-scm.org/builders/Win7%20x86_64%20hg%20tests/builds/1042/steps/run-tests.py%20%28python%202.7.13%29/logs/stdio