run-tests: add missing life-cycle methods on the example custom test result A previous commit introduced `onStart` and `onEnd` methods on test result but the one used in tests lacked those two methods. Fix it and add some output to be sure they are called. Differential Revision: https://phab.mercurial-scm.org/D3899

run-tests: fix test result verbosity A previous refactoring created the test result with a verbosity of 0 hiding some of the outputs in the normal case. Differential Revision: https://phab.mercurial-scm.org/D3898

shelve: use more accurate description in conflict marker We use "shelve" and "working-copy" instead of "source" and "dest". This is a net win. Differential Revision: https://phab.mercurial-scm.org/D3694

shelve: directly handle the initial parent alignment Shelve is currently sub-contracting some of its work to the rebase extension. In order to make shelve more independent and flexible we would like shelve to handle the parent alignment directly. After this change, we no longer need to use rebase in shelve. Differential Revision: https://phab.mercurial-scm.org/D3693

run-tests: extract onStart and onEnd into the test result It would allow custom test result to display custom messages. Differential Revision: https://phab.mercurial-scm.org/D3701

run-tests: add support for external test result The goal is to begin experiment with custom test result. I'm not sure we should offers any backward-compatibility guarantee on that plugin API as it doesn't change often and shouldn't have too much clients. Differential Revision: https://phab.mercurial-scm.org/D3700

encoding: alias cp65001 to utf-8 on Windows As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know how different it is from the UTF-8, but Python 3 appears to have introduced new codec for cp65001, so the alias is enabled only for Python 2. https://bugs.python.org/issue13216 This patch is untested, but hopefully fixes the following issue. https://bitbucket.org/tortoisehg/thg/issues/5127/

remotenames: synchronise remotenames after push also Earlier we use to pull remotenames information from the server in case of pull and clone only. This patch adds logic to push also command to pull remotenames information. This will help us in keeping the remotenames more upto date where there are a lot people changing state of branches and bookmarks at the server. Differential Revision: https://phab.mercurial-scm.org/D2874

fileset: pass in badfn to inner matchers Just for sanity. No idea if this will make a difference, but it should propagate the badfn because the matcher created by mctx.matcher() will be returned by fileset.match() in future patches.

fileset: restrict getfileset() to not return a computed set (API) And rename the functions accordingly. fileset.match() will be changed to not compute the initial subset. test-glog*.t get back to the state before 9f9ffe5f687c "match: compose 'set:' pattern as matcher."

match: add prefixdirmatcher to adapt subrepo matcher back This serves as an inverse function to the subdirmatcher, and will be used to wrap a fileset matcher of subrepositories. One of the root/prefix paths could be deduced from the matcher attributes to be wrapped, but we don't since the callers of this class know the root/prefix paths and can simply pass them in.

fileset: make debugfileset filter repository files This prepares for the structural change of the fileset. A computed fileset will no longer be a set of files, but a boolean function (i.e. matcher) to test if an input file matches the given fileset expression. --all-files option is added because some examples in the test need to scan files across revisions.

bundle2: use ProgrammingError to report bad use of addparam() This allows us to embed error message in bytes.

py3: byte-stringify literals in extension in test-bundle2-format.t # skip-blame just some b''

py3: drop b'' while formatting BundleUnknownFeatureError message

py3: suppress write() result and close file in test-bookmarks-pushpull.t

py3: drop b'' while formatting ResponseError

py3: fix revnums in bookmark discovery to be consumable more than once

py3: byte-stringify literals in extension in test-bundle2-exchange.t # skip-blame just some b''

aggressivemergedeltas: enabled the option by default The option has been around for a while (August 2015) but was never turned on by default. In-depth testing shows large wins for having that on with no significant drawbacks. When enabled, revlog consider delta against both p1 and p2 at the same time when storing a revision. Selecting a delta against "p2" can produce better deltas and chain. This raise large benefit for all repositories, especially if they have a lot of merges. Comparison of `.hg/store/` size: mercurial (6.74% merges): before: 54,225,348 bytes after: 47,279,959 bytes -13% pypy (8.30% merges): before: 459,041,759 bytes after: 346,090,067 bytes -25% netbeans (34.21% merges): before: 2,468,041,333 bytes after: 1,364,077,645 bytes -45% mozilla-central (4.84% merges): before: 2,731,799,546 bytes after: 2,157,718,019 bytes -21% Comparison of `00manifest.d` size: mercurial (6.74% merges): before: 11,682,516 bytes after: 6,143,044 bytes -47% pypy (8.30% merges): before: 156,447,163 bytes after: 52,941,780 bytes -66% netbeans (34.21% merges): before: 1,250,363,851 bytes after: 130,088,982 bytes -90% mozilla-central (4.84% merges): before: 468,202,733 bytes after: 215,096,339 bytes -54% In addition, the better deltas help with the performance of multiple core operations. However, better chains mean longer chains, which can affect performance negatively (mostly manifest revision retrieval time). Chains length is a deeper problem that also affects linear repository too. Overall we think the benefits of using p2 as a diff target are bigger than the downsizes. In addition, we are also working on ways to improve the performance impact of chain length, so theses downsizes get fixed in the future. Below are interesting items from the full benchmark run: bundling 100 revisions from pypy: before: 670ms after: 480ms -28% bundle 10000 revisions from pypy: before: 1.38s after: 1.10s -54% bundle 10000 revisions from pypy: before: 16.1s after: 7.81s -52% bundle 10000 revisions from netbeans: before: 19.3s after: 15.5s -19% unbundle 1000 revisions to pypy: before: 641ms after: 315ms - 51% clone mercurial (http): before: 26.0s after: 22.6s -23% pulling 1000 revisions from pypy (shh): before: 2.07s after: 1.36s -44% pushing 1000 revision through http (pypy repository) before: 2.18s after: 1.35s -48% diff time in mozilla-central: before: 1.420s after: 0.983s -31% status time in mozilla-central: before: 1.260s after: 0.828s -34% Impact in other cases seems minimal (within a couple of percent in worse cases) and can be seen in both direction: Timing for a simple `hg commit`: mozilla-central: before: 3.37s after: 3.22s -4% pypy: before: 194ms after: 197ms +2% Timing for status (from tip to parent of tip): mercurial: before: 52.4ms after: 52.4ms (same) pypy: before: 55.2 after: 56.9 +3% Timing for `hg update` mozilla-central, across 10 revisions: before: 4.82s after: 4.59s -5% mozilla-central, across 10000 revisions: before: 49.1s after: 49.9s +2% pypy, across 10 revisions: before: 213ms after: 216ms +1% pypy, across 10000 revisions: before: 5.31ms after: 5.24ms -1% The negative consequences are related to manifest fetch time: (timing for the tip revision tested by the benchmark) pypy-2018: before: 2.60ms after: 3.88ms +50% mozilla-central-2018: before: 565ms after: 652ms +15% (~+100ms) netbeans-2018: before: 101ms after: 250ms +48% (~+150ms) This shows up as a fixed overhead on some command we benchmarked: no-op push of mozilla-central: before: 945ms after: 1040ms +10% (~+100ms) pushing 10 changeset in netbeabs over ssh: before: 557ms after: 712ms +28% (+155ms) pushing 100 changeset in netbeabs over ssh: before: 592ms after: 771ms +30% (+179ms)

merge with stable

ui: make the large file warning limit fully configurable While add --large can be used to override it selectively, often enough the user simply doesn't care about machines with less than 100MB RAM or so, so make it possible to just specify a larger limit in hgrc. Differential Revision: https://phab.mercurial-scm.org/D3893

fileset: sort debugfileset output Unlike revset, the order of fileset result doesn't matter since it's used as a matcher predicate. This stabilizes debugfileset output for upcoming changes.

fileset: move helper functions to top

py3: fix bundle heads to be consumable more than once

py3: byte-stringify literals in hook script in test-bundle.t # skip-blame just some b''

py3: make 'None in lazyancestors' not crash This looks somewhat weird, but we have callers like 'torev(n) in futurecommon' around where torev(n) is dictlike.get(n). I could fix callers, but that would be unnecessarily verbose.

py3: convert server-string to unicode to make http library happy

py3: fix dumbhttp.py to convert --daemon-postexec arguments back to bytes

py3: use bytes.endswith() instead of bytes[n]

diff: graduate word-diff option from experimental Per 4.6 Sprint notes. I've also made it gated by "formatchanging" since it could change the output if we had an option to highlight words without using colors.

fix: add test case that shows why --whole with --base is useful Differential Revision: https://phab.mercurial-scm.org/D3894

context: raise ProgrammingError on repo['my-tag'] We had an internal extension that I had failed to migrate off of the deprecated API and its "'my-tag' in repo" check just started returning False. It took a while to figure out that that was what was happening. This patch would have helped. Differential Revision: https://phab.mercurial-scm.org/D3895

diffutil: move the module out of utils package mercurial.utils modules inherit the property of the mercurial.util, which is no dependency on ui, repo, ctx, etc. As the diffutil module seems to reside in the scmutil layer, it's probably better to not put it under the utils package.

diffutil: remove diffopts() in favor of diffallopts() patch.diffopts() exists only for backward compatibility. We don't need it in new module.

obsutil: use public interface to access to repo.ui

hgweb: pass ui to diffstatgen() explicitly

synthrepo: simply use the ui passed as a function argument

templatekw: obtain ui directly from the template context

context: remove unneeded alias of diffopts

match: remove ctx argument from code path down to _buildmatch() 'ctx' was there only for filesets.

match: compose 'set:' pattern as matcher Baby step towards porting fileset to matcher composition. We can't use the exactmatcher since it would provide a computed set as exact paths. That's why we use the predicatematcher with fset.__contains__. This will be cleaned up later. The test change in test-glog.t means that the "set:copied()" pattern is no longer be processed as a slow path. That's because the fset is empty. This will also change in future patches.

match: resolve 'set:' patterns first in _buildmatch() This just makes the next patch less complicated. The order of 'set:' and 'subinclude:' expansion doesn't matter.

match: explode if unsupported pattern passed down to _regex() builder

match: add basic wrapper for boolean function This serves as a generic wrapper for fileset predicates. In future patches, a fileset expression will be mapped to a tree of matchers for a better support of match attributes such as visitdir(). For example, $ hg debugwalk -v 'set:contrib/** and binary()' * matcher: <intersectionmatcher m1=<patternmatcher patterns='(?:contrib/.*$)'>, m2=<predicatematcher pred=binary>> ...

stringutil: move _formatsetrepr() from smartset I'll add a matcher subclass wrapping a boolean function, which will use buildrepr() to provide debugging information in a similar way to smartset.filteredset.

pycompat: move rapply() from util I want to use rapply() in utils.* modules, but that would introduce a reference cycle util -> utils.* -> util. Moving rapply() to pycompat should be okay since it mostly serves as a compatibility helper.

strip: improve help text for --no-backup option Help text is modified to clearly define the meaning of --no-backup option. Differential Revision: https://phab.mercurial-scm.org/D3886

test-convert: demonstrate an unstable hash issue for bzr -> hg -> hg It looks like the manifest value changing is the only difference, but I'm not sure why it's happening. I've got a similar divergence in a production repo that was also converted from bzr and has an octopus merge[1]. Unlike here, the manifest values for the destination merge commits reflect the initial merge only, instead of all four merges agreeing like this test. $ hg -R src_repo manifest -r 310 --debug | grep file # octopus fixup merge 2d8775bc2481bd28ac87038ecdf33e1dbddc80e9 644 file1 6adb9353a55bb8be76e71382efc724ec3ccf7ed5 644 file2 $ hg -R src_repo manifest -r 309 --debug | grep file # first merge 362e7cb5163153c4989daad1a834871ae849f205 644 file1 2c65d947191938c3ea616b7ceb7648ff3843261f 644 file2 $ hg -R dst_repo manifest -r 273 --debug | grep file # octopus fixup merge 362e7cb5163153c4989daad1a834871ae849f205 644 file1 2c65d947191938c3ea616b7ceb7648ff3843261f 644 file2 $ hg -R dst_repo manifest -r 272 --debug | grep file # first merge 362e7cb5163153c4989daad1a834871ae849f205 644 file1 2c65d947191938c3ea616b7ceb7648ff3843261f 644 file2 This divergence is espcially annoying because unlike changelog differences, I haven't figured out a way to fix this in code. The only way I found to work around it is to convert up to the point of divergence, `hg bundle` the bad revision in the source, apply it to the destination, add a line to the shamap, and fire off the conversion again. But I suspect that there's more to it than just the octopus merge because I also have a commit in the same repo, done in Mercurial (well after the conversion) that is exhibiting a similar issue (and it's not a merge commit). I'm almost positive that it was created with 4.4 or later. Any ideas? [1] https://www.mercurial-scm.org/pipermail/mercurial/2018-June/050924.html

convert: add a config knob for not saving the bzr revision Now that the timestamp is fixed, the log of the Mercurial repo is still unstable with --debug, because the bzr commit ID keeps changing. Both hg and git implement `convert.*.saverev`, so do the same here.

test-bzr: use a fixed commit time This will help stabilize the hashes when converting to a Mercurial repo.

diff: use `context.diff` to produce diff We want to make sure `context.diff` if full featured. Using it in the main diff code path is a good way to do so. The end goal is to be able to easily compute diff between in-memory context.

context: no longer accept diff options as dictionnary Since we already broke the API earlier in this stack, there are no point to introducing a new deprecation warning.

obsutil: pass a diffopts object to context.diff

synthrepo: pass a diffopts object to context.diff

webutil: pass a diffopts object to context.diff

revset: pass an explicit `diffopts` objet to context.diff

template: directly instantiate diff options for diffstat

tests: update test-context.py to use diffopts as diff argument

diffutil: extract diff options code into a dedicated util-module We want to be able to create and use diffoptions in more places. Currently the official function to instantiate diffoptions live into `mercurial.patch`. A module too high level to be easily imported in some places. So we extract the diff options related function in their own utility module.

packaging: add fedora 28 Differential Revision: https://phab.mercurial-scm.org/D3892

packaging: add missing path segments in cp statements in fedora build Differential Revision: https://phab.mercurial-scm.org/D3891

packaging: fix misplaced ( in docker-fedora rule definition Differential Revision: https://phab.mercurial-scm.org/D3890

packaging: make help output correct for fedora and centos targets Differential Revision: https://phab.mercurial-scm.org/D3889