py3: use bytes in place of basestring All strings in mercurial must be bytes so we can easily replace basestring with bytes. Differential Revision: https://phab.mercurial-scm.org/D1555

amend: make a copy of "extra" to avoid mutating an input I don't know of any problems this has caused, it just seems less surprising. Differential Revision: https://phab.mercurial-scm.org/D1575

tests: removes bashism from test-unamend.t FreeBSD builders fail because of bashism. Differential Revision: https://phab.mercurial-scm.org/D1578

run-tests: avoid calculating _testdir again Differential Revision: https://phab.mercurial-scm.org/D1574

run-tests: simplify by using dict.pop() with default Differential Revision: https://phab.mercurial-scm.org/D1573

py3: use byteskwargs in sparse.py This removes several dozen failures in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1482

py3: define __next__ in patch.py This needed to appease Python 3's iterator protocol. This is crasher #5 in Python 3. Differential Revision: https://phab.mercurial-scm.org/D1480

run-tests: mechanism to report exceptions during test execution Sometimes when running tests you introduce a ton of exceptions. The most extreme example of this is running Mercurial with Python 3, which currently spews thousands of exceptions when running the test harness. This commit adds an opt-in feature to run-tests.py to aggregate exceptions encountered by `hg` when running tests. When --exceptions is used, the test harness enables the "logexceptions" extension in the test environment. This extension wraps the Mercurial function to handle exceptions and writes information about the exception to a random filename in a directory defined by the test harness via an environment variable. At the end of the test harness, these files are parsed, aggregated, and a list of all unique Mercurial frames triggering exceptions is printed in order of frequency. This feature is intended to aid Python 3 development. I've only really tested it on Python 3. There is no shortage of improvements that could be made. e.g. we could write a separate file containing the exception report - maybe even an HTML report. We also don't capture which tests demonstrate the exceptions, so there's no turnkey way to test whether a code change made an exception disappear. Perfect is the enemy of good. I think the current patch is useful enough to land. Whoever uses it can send patches to imprve its usefulness. Differential Revision: https://phab.mercurial-scm.org/D1477

run-tests: make --extra-config-opt work with Python 3 And add test coverage to ensure it works. Differential Revision: https://phab.mercurial-scm.org/D1476

run-tests: organize options into argument groups And sort arguments so help output is more legible. There are probably a ton of ways to group things. I tried to picture the test harness as a pipeline and attempted to draw boundaries around stages in that pipeline to create the groupings. Differential Revision: https://phab.mercurial-scm.org/D1475

run-tests: convert to argparse optparse has been deprecated since Python 3.2. Best to get on the new boat before the old one sinks. It looks like argparse formats its usage string differently than optparse. Meh. Differential Revision: https://phab.mercurial-scm.org/D1474

merge with stable

server: introduce a 'experimental.single-head-per-branch' option When the option is set, the repository will reject any transaction adding multiple heads to the same named branch. For now we reject all scenario with multiple heads. One could imagine handling closed branches differently. We prefer to keep things simple for now. The feature might get extended later. Branch closing is not the best experience Mercurial has to offer anyway.

scmutil: extra utility to display a reasonable amount of nodes Push have some logic to display a reasonable amount nodes. We extract it to an utility function to make it reusable.

uncommit: unify functions _uncommitdirstate and _unamenddirstate to one The _unamenddirstate() function was inspired by _uncommitdirstate() function as the logic was same but we were unable to use the latter function directly. So previous patch introduced the _unamenddirstate() function and now this patch unifies both the function and we have a _fixdirstate() function. Adding function in previous patch and unifying in a later patch makes the reasoning easier and also leaves the last patch dedicated to what it is meant to be. Differential Revision: https://phab.mercurial-scm.org/D971

unamend: move fb extension unamend to core unamend extension adds an unamend command which undoes the effect of the amend command. This patch moves the unamend command from that extension to uncommit extension and this one does not completely undoes the effect of amend command as it creates a new commit, rather than reviving the old one back. This also adds tests for the same. .. feature:: A new unamend command in uncommit extension which undoes the effect of the amend command by creating a new changeset which was there before amend and moving the changes that were amended to the working directory. Differential Revision: https://phab.mercurial-scm.org/D821

color: respect HGPLAINEXCEPT=color to allow colors while scripting (issue5749) I'd also like --color=always on the command-line to override HGPLAIN=1 et al, but that's more work, and this seems like a better fix. We've got a fair number of programs that actually want to automate hg and get colored output to users, so they should set HGPLAINEXCEPT=alias (what we usually recommend), but this has been breaking them because they then lose color. .. feature:: The ``HGPLAINEXCEPT`` environment variable can now include ``color`` to allow automatic output colorization in otherwise automated environments. Differential Revision: https://phab.mercurial-scm.org/D1532

test-ssh: do not actually look up "brokenrepository" by DNS

largefiles: explicitly set the source and sink types to 'hg' for lfconvert I stumbled into this prior to adding the type indicator on the source and sink, but there's no reason to try to infer the types for this conversion.

lfs: add a repo requirement for this extension when converting to lfs This covers both the vanilla repo -> lfs repo and largefiles -> lfs conversions. The largefiles extension adds the requirement directly, because it has a dedicated command to convert. Using the convert extension is better, because it supports more features. I'd like ideas about how to ensure that converting away from lfs works on all files. (See comments in test-lfs.t)

convert: allow the sink object to be wrapped when the extension isn't loaded The next patch will wrap the conversion code, in order to write out a requirement for 'lfs' when appropriate. Wrapping convcmd.convertsink() in an afterloaded callback works fine when the convert extension is enabled by the user. The problem here is that lfconvert uses the convert extension, whether or not it was formally enabled by the user. My first attempt was to have lfs install an afterloaded callback that would wrap the convert sink if convert was loaded, or wrap lfconvert if it wasn't. Then the lfconvert override could install an afterloaded callback to try wrapping the convert sink again, before calling the original lfconvert. But that breaks down if largefiles can't load the convert extension on the fly. [1] Further, some tests were failing with an error indicating that the size of the afterloaded list changed while iterating it. Yuya mentioned that maybe some bits of convert could be moved into core, but I'm not sure where to draw that line. The convertsink() method depends on the list of sinks, which in turn depends on the sink classes. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/108038.html

convert: save an indicator of the repo type for sources and sinks This seems like basic info to have, and will be used shortly when deciding whether or not to wrap the class for lfs conversions. The other option is to just add a function to each class. But this seems better in that the strings aren't duplicated, and the constructor for most of these will run even if the VCS isn't installed, so it's easier to catch errors.

lfs: add a repo requirement for this extension once an lfs file is committed Largefiles does the same thing (also delayed until the first largefile commit), to prevent access to the repo without the extension. In the case of this extension, not having the extension loaded while accessing an lfs file results in cryptic errors about "missing processor for flag '0x2000'". If enabled locally but not remotely, the cryptic error message is about no common changegroup version. (It wants '03', which is currently experimental.) The largefiles extension looks for any tracked file that starts with '.hglf/'. Unfortunately, that doesn't work here. I didn't see any way to get the files that were just committed, without doing a full status. But since there's no secondary check on adding an lfs file once the extension is loaded and a threshold set, the best practice is to only enable this locally on a repo that needs it. That should minimize the unnecessary overhead for repos without an lfs file.

match: remove doc about undefined behavior of visitdir() This was added by 8545bd381504, but core matchers support visitdir() of arbitrary locations since 2773540c3650, and verifier._verifymanifest() doesn't seem to strictly obey the restriction. I have no idea how important this API contract is for third-party extensions. That's why this patch is RFC.

merge with stable

hgweb: add .jshintrc with some basic rules This file is picked up automatically by jshint, so no extra changes required in test-check-jshint.t.

hgweb: look up "URLSearchParams" in "window" to work around jshint issues Unfortunately, current version of jshint (2.9.5) doesn't know such a global variable and complains that it's undefined. Since this line tries to look up URLSearchParams in a global scope (i.e. window), let's simply preface it with "window." to work around jshint.

hgweb: define locally used variables as actually local in mercurial.js Variables that are used or assigned without any declaration using var (or let, or const) are considered global. In many cases this is inadvertent and actually causes a variable leaking to a broader scope, such as a temporary variable used inside a loop suddenly being accessible in global scope. (This corresponds to "undef" option of jshint). So this patch limits the scope of variables that don't need to be global. There are a lot of helper variables in Graph.render() used in a loop, I've declared them all on one line to reduce patch size. "radius" is special because it wasn't passed to graph.vertex, but was used there (it worked because this variable leaked to global scope). "window.graph" is created by an inline script in graph.tmpl so that it can be used in ajaxScrollInit() function, this patch makes this fact explicit by assigning window.graph to a local variable.

hgweb: rename an instance of XMLHttpRequest to xhr in mercurial.js "xhr" is a really widespread name for this kind of things, no idea where did "xfr" come from (the original 2228bd109706 doesn't explain that). Let's just change one letter so the name makes more sense.

hgweb: properly iterate over arrays and objects in mercurial.js In JavaScript, using for-in loops to access every property of an object can have unexpected results when inheritance is involved. For example, if some piece of code adds a property (it may be a method too) to Object.prototype, then all for-in loops that iterate over keys of any object (also anything that inherits Object) will get that property on one of the iterations. To filter out such unexpected properties for-in loops have to use Object.hasOwnProperty() method. (This corresponds to "forin" option of jshint). In the two first cases "data" and "edges" are arrays, to it's simpler to just switch to using a regular for-with-a-counter loop.

hgweb: use strict equals in mercurial.js This patch changes "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Some of the changes in this patch are straightforward, e.g. when comparing results of typeof (they could only be strings). The same goes for 'none' and similar strings that can't be sensibly coerced to some other type. Two changes that compare values to "1" and "0" can be clarified: getAttribute() returns either a string or null, but comparing null to a string is always false, so no logic is lost.

hgweb: use strict equals, remove non-breaking space in followlines.js The first hunk had a non-breaking space character just before "{", it's not an error or anything, but let's fix it while we're at it. (This corresponds to "nonbsp" option of jshint). Hunks 2 and 3 change "==" (equals operator) to "===" (strict equals operator). The difference between them is that the latter doesn't do any type coercions. It's handy to compare string '1' to number 1 sometimes, but most of the time using "==" is inadvertent and can be replaced by an explicit type conversion. (This corresponds to "eqeqeq" option of jshint). Most of this file already uses strict equals operator, and in the code affected type coercion is not needed, because tagName and selectableTag are both strings and endId and startId are both numbers.

run-tests: make "| foo (re)" not match everything We make "foo (re)" match the entire line by adding a \Z to the regular expression before matching. However, that doesn't help when the regular expression is something like "| foo", because that gets translated to "| foo\Z", where the "|" has lower precedence and it thus matches the empty string, which is of course a prefix of every string. Fix by wrapping expression in a group before adding the \Z to the end. Differential Revision: https://phab.mercurial-scm.org/D1546

tests: fix regex in test-subrepo-git.t to match entire string Due to a bug in the test runner (fixed by the next commit), the regex used for matching lines like " foobar | 2 +-" stoppped at the "|" and the test passed even though the rest of the line did not match. The test seems to have been supposed to match "|" and "+" literally on those lines, so this changes the regex to escape those characters. It also changes a "\s*" to "\s+" since I think we'll always include a space after the "|" in the diffstat output. Differential Revision: https://phab.mercurial-scm.org/D1545

contrib: improve check-code ban on $LOCALIP in output without (glob) Differential Revision: https://phab.mercurial-scm.org/D1553

tests: re-add (glob) for $LOCALIP matches This should fix most of the failing tests on the FreeBSD builder, since it has no 127/8 series IP as a side effect of being trapped in a jail. Differential Revision: https://phab.mercurial-scm.org/D1552

py3: make sure the first argument of time.strftime() is str time.strftime() does not accepts bytes as its first argument on py3. Differential Revision: https://phab.mercurial-scm.org/D1559

py3: alias xrange to range in tests/seq.py Differential Revision: https://phab.mercurial-scm.org/D1560

py3: use pycompat.maplist() instead of map() Differential Revision: https://phab.mercurial-scm.org/D1558

tests: move JSON escape test to test-hgweb-json.t The original tests (kanji and null) in test-hgweb-commands.t come from aff419e260f9 and 823a7d79ef82, but they check json escape filter by using JavaScript variable on /graph page, which is awkward, and I'm planning to remove commit description from this variable soon. Let's move the parts that check json template filter to a more appropriate file and use normal json-* templates.

py3: fix handling of keyword arguments in revert Differential Revision: https://phab.mercurial-scm.org/D1554

fsmonitor: issue debug messages when we fall back to core status Having more information about when and why fsmonitor bails out help when looking into status performance.

py3: add b'' to regular expressions which are raw strings Differential Revision: https://phab.mercurial-scm.org/D1538

py3: use '%d' for integers rather than '%s' obsolete._readmarkers() returns an integer version number. Differential Revision: https://phab.mercurial-scm.org/D1537

py3: fix args handling for obsfate template Differential Revision: https://phab.mercurial-scm.org/D1536

py3: remove test-terse-status.t from python3 whitelist as it was renamed The renamed file exists in the whitelist. Differential Revision: https://phab.mercurial-scm.org/D1540

test-lfs: allow the test server to be killed on Windows Apparently '$!' doesn't return a Win32 PID, so the process was never killed, and the next run was screwed up. Oddly, without the explicit killdaemons.py at the end, the test seems to hang. This spawning is just sad, so I limited it to Windows.

test-lfs: perform the `chmod +x` command in a manner compatible with Windows

hghave: add a check for lfs-test-server This is consistent with how the other tests require a feature.

hgweb: show changeset age in more places (gitweb and monoblue) mercurial.js has a process_dates() function that calculates relative age for a given date, it works for all elements with "age" css class. If those elements also have "date" css class, the original text is preserved and age is added at the end. This patch adds these two css classes in some pages in gitweb and monoblue that weren't already using this feature.

obsolete: drop usage of changectx in '_computecontentdivergentset' Changectx are expensive and not needed there. The use of `repo.set` denote old code that predate the introduction of `repo.revs` that we now use. On my mercurial repository 495 draft: before: 0.054239 second after: 0.046935 second On a mercurial repository with 115973 draft: before: 0.564548 second after: 0.130534 second

obsolete: drop usage of changectx in '_computephasedivergentset' Changectx are expensive and not needed there. The use of `repo.set` denote old code that predate the introduction of `repo.revs` that we now use. On my mercurial repository 495 draft: before: 0.010275 second after: 0.008832 second On a mercurial repository with 115973 draft: before: 0.899255 second after: 0.397131 second

hgweb: remove unused Graph() properties Both of these were introduced in 0dba955c2636, but were already unused.

gitweb: remove unused css classes Looks like they were unused since the very introduction of gitweb theme in 385b8872b8e3.

monoblue: also highlight target line on annotate and comparison pages Clicking on a line link on pages that show any kind of file contents (including diffs) should highlight that line, and in monoblue it works when there's a <pre> element (e.g. diff), but pages that use <table> element (annotate and compare) need this css class. It matches and highlights linked (":target") table rows. This line is pretty much copied from gitweb theme.

paper: remove css hack that made .branchname look like .branchhead There's a visual difference in hgweb between one changeset that is the tip of its branch and another that simply belongs to that branch. But paper theme ignored this difference on changeset page and used to always use "branchname" css class, be that changeset the tip of its branch or not. That has been recently fixed, so this piece of css is not needed anymore.

hgweb: rename the main attribute of instabilities Let's make "instabilities" list contain items with "instability" key as opposed to "name" key. This way it's more explicit and more consistent with the log command-line template.

hbisect: use a defaultdict to avoid large allocations for a large changelogs We can avoid a SPACE(len(changelog)) allocation by using a defaultdict. Test Plan: python run-tests.py test-bisect* Differential Revision: https://phab.mercurial-scm.org/D1499

hbisect: use a revset for ancestor calculation Since we have revsets we can be more concise in doing the ancestor calulcation. Significant commits are all descendent of the topmost good commits. Test Plan: python run-tests.py test-bisect* Differential Revision: https://phab.mercurial-scm.org/D1498

hbisect: pass repo into hbisect.bisect Pass repo into the bisect function to get more flexibility in what we can call. This will allow us to use revsets to rewrite parts of the ancestor and children calculation in later patches. Test Plan: python run-tests.py test-bisect* Differential Revision: https://phab.mercurial-scm.org/D1497