merge with stable

statprof: remove a line of deadcode Differential Revision: https://phab.mercurial-scm.org/D1689

extdata: abort if external command exits with non-zero status (BC) Per the last discussion, this is more reliable and consistent way than suppressing an error. For grep, erroring out might be inconvenient, but for curl, non-zero exit status should be detected. The latter wouldn't be possible if non-zero status is ignored. https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-October/105727.html

hgweb: update graph function docstring

rebase: fix for hgsubversion 5c25fe7fb1e broke something in the hgsubversion test path, causing it raise an abort (Abort: nothing to merge) during a perfectly good rebase. I tracked it down to this change. It's probably not hgsubversion related. I suspect that using the same `wctx` from before the initial update causes problems with the wctx's cached manifest property. I noticed we also sometimes stick random gunk on the wctx object in other places (like in `copies.py`) so it's probably best to reset it for now. The line I added before was actually useless since we don't pass wctx to the initial `merge.update`, so it defaults to `repo[None]`. So I just removed it. Differential Revision: https://phab.mercurial-scm.org/D1679

hgweb: render next pages on /graph incrementally Previously, when user scrolled down to see the next page on /graph, all hgweb did was re-render everything that would be visible (by simply incrementing revcount). It was not efficient at all, and this patch makes /graph page behave similarly to the regular /log: every new page only consists of new changesets, no duplication, and only jsdata is based on the full set of changesets required to build accurate graph. This is achieved by adding "?graphtop=<node>" to the next page URL template, effectively remembering where the graph started, and using that value to create the new `tree` that covers the whole visible graph. That variable is then used to produce jsdata for redrawing graph client-side. nextentry is used for the same purpose as on /log page (to format the next page URL), but it's not a part of the graph.

hgweb: split graphdata() into jsdata() and nodes() nodes keyword passed to the template can be any iterator, but jsdata needs to be a list because it gets JSONified.

hgweb: calculate <canvas> width and height client-side hgweb determines and passes to templates some variables related to graph appearance, like bg_height, canvaswidth and canvasheight. bg_height was and still is used for graph.scale() call in graph.tmpl, and the two latter variables were used in <canvas> element as width and height properties, and they were set before JS code got to run. Setting these properties server-side doesn't make a lot of sense, because a graph that has been scaled should calculate things like width and height on its own when being rendered. Let's move (re)sizing <canvas> to JavaScript (to Graph.render function) and stop parsing HTML with regular expressions just to know new width and height. That extra loop that only counts cols is required because <canvas> can't be resized after or in the process of rendering (or it gets cleared). Incidentally, SVG doesn't have this problem and I'm hoping to switch graph to using it in future. There also was truecanvasheight, but according to hg grep --all it was never used, see d490edc71146.

hgweb: filter graphmod.colored() output before iterating over it Consumers in this function use output of graphmod.colored(), but only want items with type == CHANGESET, so let's filter it early. This is primarily just a refactoring, but it also fixes a potential small bug with `rows = len(tree)` (this variable is used for "Rows shown" line in raw-graph) if there are items of other types.

py3: introduce pycompat.ziplist as zip is a generator on Python 3 Similar to map, the return value of zip was also turned into a generator on Python 3. We have multiple occurences in our codebase where we retrieve the values of zip multiple times, so let's introduce pycompat.ziplist similar to pycompat.maplist. Differential Revision: https://phab.mercurial-scm.org/D1675

py3: pass bytes from extension to ui.write() in test-revlog-mmapindex.t This makes the test pass on Python 3. Differential Revision: https://phab.mercurial-scm.org/D1674

py3: handle keyword arguments correctly in hgext/record.py Differential Revision: https://phab.mercurial-scm.org/D1673

py3: handle keyword arguments correctly in bundlerepo.py Differential Revision: https://phab.mercurial-scm.org/D1672

py3: handle keyword arguments correctly in debugcommands.py Differential Revision: https://phab.mercurial-scm.org/D1671

memfilectx: make changectx argument mandatory in constructor (API) committablefilectx has three subclasses: workingfilectx, memfilectx, and overlayfilectx. committablefilectx takes an optional (change) ctx instance to its constructor. If it's provided, it's set on the instance as self._changectx. If not, that property is supposed to be defined by the class. However, only workingfilectx does that. The other two will have the property undefined if it's not passed in the constructor. That seems bad to me. This patch makes the changectx argument to the memfilectx constructor mandatory because that fixes the failure I ran into. It seems like we should also fix the overlayfilectx case. Differential Revision: https://phab.mercurial-scm.org/D1658

debugbuilddag: create filectx instance in 'filectxfn' callback Same motivation is previous patch. Differential Revision: https://phab.mercurial-scm.org/D1670

synthrepo: create filectx instance in 'filectxfn' callback I would like to pass the memctx to the memfilectx constructor, but it's not available where we currently create the memfilectx. It is available in the 'filectxfn' callback, so let's create the memfilectx there instead. A later patch will start actually passing the memctx. Differential Revision: https://phab.mercurial-scm.org/D1669

lfs: use 'ui' provided to `upgrade` for output, instead of stealing srcrepo's Also spotted by Yuya.

lfs: correct the directory list value returned by lfsvfs.walk() Spotted by Yuya.

debuglocks: allow setting a lock

debuglocks: add tests (and fix typo in early return)

tests: remove (glob) annotations that were only for '\' matches # skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)

check-code: drop the rules for adding (glob) for Windows paths I'll probably resurrect at least some of these as replacement patterns to switch '\' to '/' when creating tests at some point. But since this subset of globs isn't needed anymore after dfae14354660, and the rules are blocking the glob removal, just drop them for now. It probably isn't worth turning them into checks for unnecessary globs being present.

run-tests: stop automatically adding a (glob) for bundle backup lines This is the first step to dropping the existing globs for '\' matches, now that it is handled automatically. Instead of just dropping it, this pattern is now used to convert to '/' paths, to reduce the amount of manual cleanup required when creating tests on Windows.

histedit: preserve active branch while histediting The branch information was properly preserved in the changeset, but the "active" branch of the working copy could be lost (the branch of the base being used). Histedit used to behave properly in this regard but the case was not tested and regressed 4 years ago in ab2362e1672e.

hgweb: implement json-graph It's essentially a copy of json-log with graph-related things added (col, row, color, edges).

rebase: replace --inmemory flag with rebase.experimental.inmemory config Differential Revision: https://phab.mercurial-scm.org/D1666

tests: add some commentary and diagnostics to test-run-tests.t Hopefully this will give us a shot at fixing it on the Windows builder. Differential Revision: https://phab.mercurial-scm.org/D1653

templater: fix "one arguments"

tests: add commit hashes to log commands in rebase tests Commit hashes are a useful way to ensure the content of commits made in the tests are not changing, even if we don't query every aspect of every commit. (And some properties, like extras, are rarely printed at all.) Many of the rebase log -G calls didn't show hashes; by adding hashes to places that weren't showing them we can help protect those tests from unwanted changes. Differential Revision: https://phab.mercurial-scm.org/D1650

tests: add a simple test for in-memory rebase This is just a very simple start, but verifies some of the basic cases of an in-memory rebase. Differential Revision: https://phab.mercurial-scm.org/D1652

patch: do not break up multibyte character when highlighting word This changes {\W} to {\W - any 8bit characters} so that multibyte sequences are taken as words. Since we don't know the encoding of user content, this is the most sensible definition of a non-word.

run-tests: accept '\' vs '/' path differences without '(glob)' Having to constantly adjust these is a hassle. It's easy for this to slip by when not testing on Windows, and then when it happens on stable, the tests fail for the next 3 months if we follow the rules for stable. This works the same way the EOL differences are ignored, namely to adjust on the fly and recheck on Windows. I can't think of any situation where there would be a '\' on Windows, a '/' elsewhere, and the '/' should be considered a failure on Windows. This fixes the obvious output problems where (glob) is missing. Without this, test-alias.t, test-remotenames.t and test-largefiles-misc.t are failing. The flip side (not handled by this) is the case where an unnecessary glob is present. There seems to be two separate behaviors. cf300c1ad7bf is an example of where the test has been autocorrecting (with output differences), and d4ec69ff652a is an example where the test fails and reports 'no result code from test'. Hopefully those cases will become even more rare if people don't need to guess at when a glob is needed for a Windows path. It's probably unreasonable to submit a single patch that wipes out all of the (glob) instances that were only used to hide path differences, given the churn from other contributors. Since their presence isn't harming the tests, these can be removed through attrition.

run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too We already do this for lines ending in '\n', such that the test only needs to be run with --interactive and the changes accepted at the end. But that wasn't working with list-tree.py output for example, and required manual fixup.

tests: stabilize the sorted output of list-tree.py on Windows The test-largefiles-misc.t test was moving 'dir2\' before 'dir\' because while '/' precedes most of the printable ASCII characters, '\' comes after numbers and capital letters, among other symbols.

upgrade: simplify workaround for repo.ui.copy() Copied from commandserver.py.

debugformat: embed raw values in JSON and template output

debugformat: flush formatter output per item

tests: use Python to write binary data in lfs test instead of shell The shell construct here appears to be unevenly supported: it works in /bin/sh on FreeBSD, but it doesn't seem to work when /bin/sh is dash. Using a Python inline directive works fine, so let's just do that instead. Differential Revision: https://phab.mercurial-scm.org/D1636

py3: add 32 new tests passing to the whitelist Yay, we have crossed 100 in number of tests passing on Python 3. There are 662 tests in our test suite, so there is a lot more which is need to be done. Differential Revision: https://phab.mercurial-scm.org/D1648

py3: handle keyword arguments correctly in wireproto.py Differential Revision: https://phab.mercurial-scm.org/D1647

py3: handle keyword arguments correctly in ui.py Differential Revision: https://phab.mercurial-scm.org/D1646

py3: handle keyword arguments correctly in templater.py Differential Revision: https://phab.mercurial-scm.org/D1645

py3: handle keyword arguments correctly in subrepo.py Differential Revision: https://phab.mercurial-scm.org/D1644

py3: handle keyword arguments correctly in statprof.py Differential Revision: https://phab.mercurial-scm.org/D1643

py3: handle keyword arguments correctly in sslutil.py Differential Revision: https://phab.mercurial-scm.org/D1642

py3: handle keyword arguments correctly in simplemerge.py Differential Revision: https://phab.mercurial-scm.org/D1641

py3: handle keyword arguments correctly in revset.py Differential Revision: https://phab.mercurial-scm.org/D1640

py3: handle keyword arguments correctly in patch.py Differential Revision: https://phab.mercurial-scm.org/D1639

py3: handle keyword arguments correctly in obsolete.py Differential Revision: https://phab.mercurial-scm.org/D1638

py3: handle keyword arguments correctly in keepalive.py Differential Revision: https://phab.mercurial-scm.org/D1637

lfs: restore the local blob store after a repo upgrade This also ends up testing the local extension wrapping for dstrepo during upgrade, which was fixed in 06987c6971be.

lfs: override walk() in lfsvfs In order to fix the missing lfs store after an upgrade, I attempted to walk the store vfs to hardlink to the upgraded repo's store. But the custom join() clashes with the default walk() implementation. First, 'path=None' blew up in the regex matcher, because it wanted a string. But even if that is fixed, the join to walk the root of the vfs wouldn't match the required xx/xx...xx pattern. The first cut of this was a copy/paste/tweak of the base implementation, but this version of walk() hides the internal directories, and treats the vfs as a flat store. I think this makes sense because most vfs methods call join() on input paths, which wants the simple oid format. It also relieves the caller from having to deal with bogus files/directories in the store.

tests: add coverage for preserving 'lfs' requirement on repo upgrade The test also shows that the local blob store is erroneously lost.

test-upgrade-repo: glob away timing values

py3: handle keyword arguments correctly in httppeer.py Differential Revision: https://phab.mercurial-scm.org/D1635

py3: handle keyword arguments correctly in httpconnection.py Differential Revision: https://phab.mercurial-scm.org/D1634

py3: handle keyword arguments correctly in hook.py Differential Revision: https://phab.mercurial-scm.org/D1633

py3: handle keyword arguments correctly in help.py Differential Revision: https://phab.mercurial-scm.org/D1632

py3: handle keyword arguments correctly in exchange.py Differential Revision: https://phab.mercurial-scm.org/D1631