memfilectx: make changectx argument mandatory in constructor (API) committablefilectx has three subclasses: workingfilectx, memfilectx, and overlayfilectx. committablefilectx takes an optional (change) ctx instance to its constructor. If it's provided, it's set on the instance as self._changectx. If not, that property is supposed to be defined by the class. However, only workingfilectx does that. The other two will have the property undefined if it's not passed in the constructor. That seems bad to me. This patch makes the changectx argument to the memfilectx constructor mandatory because that fixes the failure I ran into. It seems like we should also fix the overlayfilectx case. Differential Revision: https://phab.mercurial-scm.org/D1658

debugbuilddag: create filectx instance in 'filectxfn' callback Same motivation is previous patch. Differential Revision: https://phab.mercurial-scm.org/D1670

synthrepo: create filectx instance in 'filectxfn' callback I would like to pass the memctx to the memfilectx constructor, but it's not available where we currently create the memfilectx. It is available in the 'filectxfn' callback, so let's create the memfilectx there instead. A later patch will start actually passing the memctx. Differential Revision: https://phab.mercurial-scm.org/D1669

lfs: use 'ui' provided to `upgrade` for output, instead of stealing srcrepo's Also spotted by Yuya.

lfs: correct the directory list value returned by lfsvfs.walk() Spotted by Yuya.

debuglocks: allow setting a lock

debuglocks: add tests (and fix typo in early return)

tests: remove (glob) annotations that were only for '\' matches # skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)

check-code: drop the rules for adding (glob) for Windows paths I'll probably resurrect at least some of these as replacement patterns to switch '\' to '/' when creating tests at some point. But since this subset of globs isn't needed anymore after dfae14354660, and the rules are blocking the glob removal, just drop them for now. It probably isn't worth turning them into checks for unnecessary globs being present.

run-tests: stop automatically adding a (glob) for bundle backup lines This is the first step to dropping the existing globs for '\' matches, now that it is handled automatically. Instead of just dropping it, this pattern is now used to convert to '/' paths, to reduce the amount of manual cleanup required when creating tests on Windows.

hgweb: implement json-graph It's essentially a copy of json-log with graph-related things added (col, row, color, edges).

rebase: replace --inmemory flag with rebase.experimental.inmemory config Differential Revision: https://phab.mercurial-scm.org/D1666

tests: add some commentary and diagnostics to test-run-tests.t Hopefully this will give us a shot at fixing it on the Windows builder. Differential Revision: https://phab.mercurial-scm.org/D1653

tests: add commit hashes to log commands in rebase tests Commit hashes are a useful way to ensure the content of commits made in the tests are not changing, even if we don't query every aspect of every commit. (And some properties, like extras, are rarely printed at all.) Many of the rebase log -G calls didn't show hashes; by adding hashes to places that weren't showing them we can help protect those tests from unwanted changes. Differential Revision: https://phab.mercurial-scm.org/D1650

tests: add a simple test for in-memory rebase This is just a very simple start, but verifies some of the basic cases of an in-memory rebase. Differential Revision: https://phab.mercurial-scm.org/D1652

patch: do not break up multibyte character when highlighting word This changes {\W} to {\W - any 8bit characters} so that multibyte sequences are taken as words. Since we don't know the encoding of user content, this is the most sensible definition of a non-word.

run-tests: accept '\' vs '/' path differences without '(glob)' Having to constantly adjust these is a hassle. It's easy for this to slip by when not testing on Windows, and then when it happens on stable, the tests fail for the next 3 months if we follow the rules for stable. This works the same way the EOL differences are ignored, namely to adjust on the fly and recheck on Windows. I can't think of any situation where there would be a '\' on Windows, a '/' elsewhere, and the '/' should be considered a failure on Windows. This fixes the obvious output problems where (glob) is missing. Without this, test-alias.t, test-remotenames.t and test-largefiles-misc.t are failing. The flip side (not handled by this) is the case where an unnecessary glob is present. There seems to be two separate behaviors. cf300c1ad7bf is an example of where the test has been autocorrecting (with output differences), and d4ec69ff652a is an example where the test fails and reports 'no result code from test'. Hopefully those cases will become even more rare if people don't need to guess at when a glob is needed for a Windows path. It's probably unreasonable to submit a single patch that wipes out all of the (glob) instances that were only used to hide path differences, given the churn from other contributors. Since their presence isn't harming the tests, these can be removed through attrition.

run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too We already do this for lines ending in '\n', such that the test only needs to be run with --interactive and the changes accepted at the end. But that wasn't working with list-tree.py output for example, and required manual fixup.

tests: stabilize the sorted output of list-tree.py on Windows The test-largefiles-misc.t test was moving 'dir2\' before 'dir\' because while '/' precedes most of the printable ASCII characters, '\' comes after numbers and capital letters, among other symbols.

upgrade: simplify workaround for repo.ui.copy() Copied from commandserver.py.

debugformat: embed raw values in JSON and template output

debugformat: flush formatter output per item

tests: use Python to write binary data in lfs test instead of shell The shell construct here appears to be unevenly supported: it works in /bin/sh on FreeBSD, but it doesn't seem to work when /bin/sh is dash. Using a Python inline directive works fine, so let's just do that instead. Differential Revision: https://phab.mercurial-scm.org/D1636

py3: add 32 new tests passing to the whitelist Yay, we have crossed 100 in number of tests passing on Python 3. There are 662 tests in our test suite, so there is a lot more which is need to be done. Differential Revision: https://phab.mercurial-scm.org/D1648

py3: handle keyword arguments correctly in wireproto.py Differential Revision: https://phab.mercurial-scm.org/D1647

py3: handle keyword arguments correctly in ui.py Differential Revision: https://phab.mercurial-scm.org/D1646

py3: handle keyword arguments correctly in templater.py Differential Revision: https://phab.mercurial-scm.org/D1645

py3: handle keyword arguments correctly in subrepo.py Differential Revision: https://phab.mercurial-scm.org/D1644

py3: handle keyword arguments correctly in statprof.py Differential Revision: https://phab.mercurial-scm.org/D1643

py3: handle keyword arguments correctly in sslutil.py Differential Revision: https://phab.mercurial-scm.org/D1642

py3: handle keyword arguments correctly in simplemerge.py Differential Revision: https://phab.mercurial-scm.org/D1641

py3: handle keyword arguments correctly in revset.py Differential Revision: https://phab.mercurial-scm.org/D1640

py3: handle keyword arguments correctly in patch.py Differential Revision: https://phab.mercurial-scm.org/D1639

py3: handle keyword arguments correctly in obsolete.py Differential Revision: https://phab.mercurial-scm.org/D1638

py3: handle keyword arguments correctly in keepalive.py Differential Revision: https://phab.mercurial-scm.org/D1637

lfs: restore the local blob store after a repo upgrade This also ends up testing the local extension wrapping for dstrepo during upgrade, which was fixed in 06987c6971be.

lfs: override walk() in lfsvfs In order to fix the missing lfs store after an upgrade, I attempted to walk the store vfs to hardlink to the upgraded repo's store. But the custom join() clashes with the default walk() implementation. First, 'path=None' blew up in the regex matcher, because it wanted a string. But even if that is fixed, the join to walk the root of the vfs wouldn't match the required xx/xx...xx pattern. The first cut of this was a copy/paste/tweak of the base implementation, but this version of walk() hides the internal directories, and treats the vfs as a flat store. I think this makes sense because most vfs methods call join() on input paths, which wants the simple oid format. It also relieves the caller from having to deal with bogus files/directories in the store.

tests: add coverage for preserving 'lfs' requirement on repo upgrade The test also shows that the local blob store is erroneously lost.

test-upgrade-repo: glob away timing values

py3: handle keyword arguments correctly in httppeer.py Differential Revision: https://phab.mercurial-scm.org/D1635

py3: handle keyword arguments correctly in httpconnection.py Differential Revision: https://phab.mercurial-scm.org/D1634

py3: handle keyword arguments correctly in hook.py Differential Revision: https://phab.mercurial-scm.org/D1633

py3: handle keyword arguments correctly in help.py Differential Revision: https://phab.mercurial-scm.org/D1632

py3: handle keyword arguments correctly in exchange.py Differential Revision: https://phab.mercurial-scm.org/D1631

py3: handle keyword arguments correctly in dispatch.py Differential Revision: https://phab.mercurial-scm.org/D1630

py3: handle keyword arguments correctly in context.py Differential Revision: https://phab.mercurial-scm.org/D1629

py3: handle keyword arguments correctly in commands.py Differential Revision: https://phab.mercurial-scm.org/D1628

py3: handle keyword arguments correctly in color.py Differential Revision: https://phab.mercurial-scm.org/D1627

py3: handle keyword arguments correctly in cmdutil.py Differential Revision: https://phab.mercurial-scm.org/D1626

py3: handle keyword arguments correctly in archival.py Differential Revision: https://phab.mercurial-scm.org/D1625

py3: handle keyword arguments correctly in hgext/largefiles/ Keys of keyword arguments must be str(unicode) on Python 3. The transformer which we use on Python 3, appends b'' in front of each string literal, so this may lead in KeyError or None return even when the key is present by we are using bytes value and it's stored in unicodes. This patch and all the similar patches handle this by either converting the keys of kwargs to bytes using 'pycompat.byteskwargs()' or adding r'' so that the transformer won't append b''. This next 23 patches follows the above mentioned way to handle keyword arguments. Differential Revision: https://phab.mercurial-scm.org/D1624

remotenames: rename related file and storage dir to logexchange This patch renames remotenames.py to logexchange.py, test-remotenames.t to test-logexchange.t. Also this patch renames the directory in which the data is stored from remotenames to logexchange. After this patch, data about bookmarks and branches from a server we pull is stored in `.hg/logexchange/(bookmarks|branches)` files. Thanks to smf for the suggestion. Differential Revision: https://phab.mercurial-scm.org/D1607

lfs: allow to run 'debugupgraderepo' on repo with largefiles The extensions wrap the necessary function to ensure the 'lfs' requirements won't be dropped. It is now possible to run `hg debugupgraderepo` on a repository with lfs.

upgrade: add a 'redeltafullall' mode We add a new mode for delta recomputation, when selected, each full text will go through the full "addrevision" mechanism again. This is slower than "redeltaall" but this gives the opportunity for extensions to trigger special logic. For example, the lfs extensions can decide to promote some revision to lfs storage during the upgrade.

upgrade: use actual filelog to convert filelog Extensions can add extra logic related to the config, so we must use the actual class. The path used needs minimal transformation for this to work.

upgrade: more standard creation of the temporary repository By using the standard path to create a repository we fill some hole in the current initialization process. The one who triggered this changeset was the lack of extensions initialization.

upgrade: use the repository 'ui' as the base for the new repository The `repo.baseui` contains all the configuration but the one specific to the repository (so it can be used when dealing with local peer and sub- repository). However, we need the repository config to be taken into account when doing the upgrade. Otherwise, the upgrade related config that exists in the repository config won't be taken into account when performing the update. A buggy and surprising behavior. We had to work around protection set around `repo.ui.copy` since we are an uncommon case.

upgrade: add a test to show the repository config being ignored The upgrade process ignores the config within the repository. The next changeset fixes it, but we introduce this test before to show it actually tests our target.

upgrade: register compression as a format variants Compression is a promising vector for speedup, let us make it easier to check the compression used and upgrade existing repository.

debugformat: handle non-boolean value for variant