run-tests: simplify by using dict.pop() with default Differential Revision: https://phab.mercurial-scm.org/D1573

py3: use byteskwargs in sparse.py This removes several dozen failures in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1482

py3: define __next__ in patch.py This needed to appease Python 3's iterator protocol. This is crasher #5 in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1480

run-tests: mechanism to report exceptions during test execution Sometimes when running tests you introduce a ton of exceptions. The most extreme example of this is running Mercurial with Python 3, which currently spews thousands of exceptions when running the test harness. This commit adds an opt-in feature to run-tests.py to aggregate exceptions encountered by `hg` when running tests. When --exceptions is used, the test harness enables the "logexceptions" extension in the test environment. This extension wraps the Mercurial function to handle exceptions and writes information about the exception to a random filename in a directory defined by the test harness via an environment variable. At the end of the test harness, these files are parsed, aggregated, and a list of all unique Mercurial frames triggering exceptions is printed in order of frequency. This feature is intended to aid Python 3 development. I've only really tested it on Python 3. There is no shortage of improvements that could be made. e.g. we could write a separate file containing the exception report - maybe even an HTML report. We also don't capture which tests demonstrate the exceptions, so there's no turnkey way to test whether a code change made an exception disappear. Perfect is the enemy of good. I think the current patch is useful enough to land. Whoever uses it can send patches to imprve its usefulness. Differential Revision: https://phab.mercurial-scm.org/D1477

run-tests: make --extra-config-opt work with Python 3 And add test coverage to ensure it works. Differential Revision: https://phab.mercurial-scm.org/D1476

run-tests: organize options into argument groups And sort arguments so help output is more legible. There are probably a ton of ways to group things. I tried to picture the test harness as a pipeline and attempted to draw boundaries around stages in that pipeline to create the groupings. Differential Revision: https://phab.mercurial-scm.org/D1475

run-tests: convert to argparse optparse has been deprecated since Python 3.2. Best to get on the new boat before the old one sinks. It looks like argparse formats its usage string differently than optparse. Meh. Differential Revision: https://phab.mercurial-scm.org/D1474

merge with stable

server: introduce a 'experimental.single-head-per-branch' option When the option is set, the repository will reject any transaction adding multiple heads to the same named branch. For now we reject all scenario with multiple heads. One could imagine handling closed branches differently. We prefer to keep things simple for now. The feature might get extended later. Branch closing is not the best experience Mercurial has to offer anyway.

scmutil: extra utility to display a reasonable amount of nodes Push have some logic to display a reasonable amount nodes. We extract it to an utility function to make it reusable.

uncommit: unify functions _uncommitdirstate and _unamenddirstate to one The _unamenddirstate() function was inspired by _uncommitdirstate() function as the logic was same but we were unable to use the latter function directly. So previous patch introduced the _unamenddirstate() function and now this patch unifies both the function and we have a _fixdirstate() function. Adding function in previous patch and unifying in a later patch makes the reasoning easier and also leaves the last patch dedicated to what it is meant to be. Differential Revision: https://phab.mercurial-scm.org/D971

unamend: move fb extension unamend to core unamend extension adds an unamend command which undoes the effect of the amend command. This patch moves the unamend command from that extension to uncommit extension and this one does not completely undoes the effect of amend command as it creates a new commit, rather than reviving the old one back. This also adds tests for the same. .. feature:: A new unamend command in uncommit extension which undoes the effect of the amend command by creating a new changeset which was there before amend and moving the changes that were amended to the working directory. Differential Revision: https://phab.mercurial-scm.org/D821

color: respect HGPLAINEXCEPT=color to allow colors while scripting (issue5749) I'd also like --color=always on the command-line to override HGPLAIN=1 et al, but that's more work, and this seems like a better fix. We've got a fair number of programs that actually want to automate hg and get colored output to users, so they should set HGPLAINEXCEPT=alias (what we usually recommend), but this has been breaking them because they then lose color. .. feature:: The ``HGPLAINEXCEPT`` environment variable can now include ``color`` to allow automatic output colorization in otherwise automated environments. Differential Revision: https://phab.mercurial-scm.org/D1532

test-ssh: do not actually look up "brokenrepository" by DNS

largefiles: explicitly set the source and sink types to 'hg' for lfconvert I stumbled into this prior to adding the type indicator on the source and sink, but there's no reason to try to infer the types for this conversion.

lfs: add a repo requirement for this extension when converting to lfs This covers both the vanilla repo -> lfs repo and largefiles -> lfs conversions. The largefiles extension adds the requirement directly, because it has a dedicated command to convert. Using the convert extension is better, because it supports more features. I'd like ideas about how to ensure that converting away from lfs works on all files. (See comments in test-lfs.t)

convert: allow the sink object to be wrapped when the extension isn't loaded The next patch will wrap the conversion code, in order to write out a requirement for 'lfs' when appropriate. Wrapping convcmd.convertsink() in an afterloaded callback works fine when the convert extension is enabled by the user. The problem here is that lfconvert uses the convert extension, whether or not it was formally enabled by the user. My first attempt was to have lfs install an afterloaded callback that would wrap the convert sink if convert was loaded, or wrap lfconvert if it wasn't. Then the lfconvert override could install an afterloaded callback to try wrapping the convert sink again, before calling the original lfconvert. But that breaks down if largefiles can't load the convert extension on the fly. [1] Further, some tests were failing with an error indicating that the size of the afterloaded list changed while iterating it. Yuya mentioned that maybe some bits of convert could be moved into core, but I'm not sure where to draw that line. The convertsink() method depends on the list of sinks, which in turn depends on the sink classes. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/108038.html

convert: save an indicator of the repo type for sources and sinks This seems like basic info to have, and will be used shortly when deciding whether or not to wrap the class for lfs conversions. The other option is to just add a function to each class. But this seems better in that the strings aren't duplicated, and the constructor for most of these will run even if the VCS isn't installed, so it's easier to catch errors.

lfs: add a repo requirement for this extension once an lfs file is committed Largefiles does the same thing (also delayed until the first largefile commit), to prevent access to the repo without the extension. In the case of this extension, not having the extension loaded while accessing an lfs file results in cryptic errors about "missing processor for flag '0x2000'". If enabled locally but not remotely, the cryptic error message is about no common changegroup version. (It wants '03', which is currently experimental.) The largefiles extension looks for any tracked file that starts with '.hglf/'. Unfortunately, that doesn't work here. I didn't see any way to get the files that were just committed, without doing a full status. But since there's no secondary check on adding an lfs file once the extension is loaded and a threshold set, the best practice is to only enable this locally on a repo that needs it. That should minimize the unnecessary overhead for repos without an lfs file.

match: remove doc about undefined behavior of visitdir() This was added by 8545bd381504, but core matchers support visitdir() of arbitrary locations since 2773540c3650, and verifier._verifymanifest() doesn't seem to strictly obey the restriction. I have no idea how important this API contract is for third-party extensions. That's why this patch is RFC.

merge with stable

hgweb: add .jshintrc with some basic rules This file is picked up automatically by jshint, so no extra changes required in test-check-jshint.t.

hgweb: look up "URLSearchParams" in "window" to work around jshint issues Unfortunately, current version of jshint (2.9.5) doesn't know such a global variable and complains that it's undefined. Since this line tries to look up URLSearchParams in a global scope (i.e. window), let's simply preface it with "window." to work around jshint.

hgweb: define locally used variables as actually local in mercurial.js Variables that are used or assigned without any declaration using var (or let, or const) are considered global. In many cases this is inadvertent and actually causes a variable leaking to a broader scope, such as a temporary variable used inside a loop suddenly being accessible in global scope. (This corresponds to "undef" option of jshint). So this patch limits the scope of variables that don't need to be global. There are a lot of helper variables in Graph.render() used in a loop, I've declared them all on one line to reduce patch size. "radius" is special because it wasn't passed to graph.vertex, but was used there (it worked because this variable leaked to global scope). "window.graph" is created by an inline script in graph.tmpl so that it can be used in ajaxScrollInit() function, this patch makes this fact explicit by assigning window.graph to a local variable.

hgweb: rename an instance of XMLHttpRequest to xhr in mercurial.js "xhr" is a really widespread name for this kind of things, no idea where did "xfr" come from (the original 2228bd109706 doesn't explain that). Let's just change one letter so the name makes more sense.

hgweb: properly iterate over arrays and objects in mercurial.js In JavaScript, using for-in loops to access every property of an object can have unexpected results when inheritance is involved. For example, if some piece of code adds a property (it may be a method too) to Object.prototype, then all for-in loops that iterate over keys of any object (also anything that inherits Object) will get that property on one of the iterations. To filter out such unexpected properties for-in loops have to use Object.hasOwnProperty() method. (This corresponds to "forin" option of jshint). In the two first cases "data" and "edges" are arrays, to it's simpler to just switch to using a regular for-with-a-counter loop.

hgweb: use strict equals in mercurial.js This patch changes "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Some of the changes in this patch are straightforward, e.g. when comparing results of typeof (they could only be strings). The same goes for 'none' and similar strings that can't be sensibly coerced to some other type. Two changes that compare values to "1" and "0" can be clarified: getAttribute() returns either a string or null, but comparing null to a string is always false, so no logic is lost.

hgweb: use strict equals, remove non-breaking space in followlines.js The first hunk had a non-breaking space character just before "{", it's not an error or anything, but let's fix it while we're at it. (This corresponds to "nonbsp" option of jshint). Hunks 2 and 3 change "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Most of this file already uses strict equals operator, and in the code affected type coercion is not needed, because tagName and selectableTag are both strings and endId and startId are both numbers.

run-tests: make "| foo (re)" not match everything We make "foo (re)" match the entire line by adding a \Z to the regular expression before matching. However, that doesn't help when the regular expression is something like "| foo", because that gets translated to "| foo\Z", where the "|" has lower precedence and it thus matches the empty string, which is of course a prefix of every string. Fix by wrapping expression in a group before adding the \Z to the end. Differential Revision: https://phab.mercurial-scm.org/D1546

tests: fix regex in test-subrepo-git.t to match entire string Due to a bug in the test runner (fixed by the next commit), the regex used for matching lines like " foobar | 2 +-" stoppped at the "|" and the test passed even though the rest of the line did not match. The test seems to have been supposed to match "|" and "+" literally on those lines, so this changes the regex to escape those characters. It also changes a "\s*" to "\s+" since I think we'll always include a space after the "|" in the diffstat output. Differential Revision: https://phab.mercurial-scm.org/D1545