py3: use pycompat.bytestr() to convert str to bytes

py3: pass the memoryview object into bytes() to get the value

py3: use pycompat.bytestr instead of str

py3: use '%d' to convert integers to bytes

py3: slice over bytes to prevent getting it's ascii value

py3: use pycompat.strkwargs() to convert kwargs keys to str

py3: use r'' to prevent the addition of b'' by transformer There are cases in opts handling in debugcommands.py where we don't need to convert opts keys back to bytes as there are some handful cases and no other function using opts value. Using r'', we prevent the transformer to add a b'' which will keep the value str.

py3: use pycompat.byteskwargs() to convert kwargs' keys to bytes This is used where ever required like where kwargs are passed into ui.formatter(), scmutil.match() or cmdutil.openrevlog() which expects bytes.

tests: use cgienv to minimize environment setup at hgweb tests This patch follows other hgweb tests.

tests: avoid test failure for mangling path-like string by MSYS

py3: make sure commands name are bytes in tests

py3: add b'' to make the regex pattern bytes

py3: use hex() to convert the hash to bytes

py3: add b'' to make a triple quoted string bytes on Python 3 Transformer does not adds b'' in front of triple quoted strings to prevent converting docs to bytes.

py3: add tests to show `hg bookmarks` and `hg branches` work on Python 3

py3: fix kwargs handling for `hg bookmarks`

identify: provide changectx to templater

formatter: proxy fm.context() through converter Otherwise nested template formatter would not see the context objects. It's just a boolean flag now. We might want to change it to 'ctxs -> items' function so changectx attributes are populated automatically in JSON, but I'm not sure.

identify: change p1/p2 to a list of parents It makes sense because the nested data structure is a list of items.

scmutil: add a cleanupnodes method for developers It's now common that an old node gets replaced by zero or more new nodes, that could happen with amend, rebase, histedit, etc. And it's a common requirement to do bookmark movements, strip or obsolete nodes and even moving working copy parent. Previously, amend, rebase, history have their own logic doing the above. This patch is an attempt to unify them and future code. This enables new developers to be able to do "replace X with Y" thing correctly, without any knowledge about bookmarks, strip or obsstore. The next step will be migrating rebase to the new API, so it works inside a transaction, and its code could be simplified.

strip: add a delayedstrip method that works in a transaction For long, the fact that strip does not work inside a transaction and some code has to work with both obsstore and fallback to strip lead to duplicated code like: with repo.transaction(): .... if obsstore: obsstore.createmarkers(...) if not obsstore: repair.strip(...) Things get more complex when you want to call something which may call strip under the hood. Like you cannot simply write: with repo.transaction(): .... rebasemod.rebase(...) # may call "strip", so this doesn't work But you do want rebase to run inside a same transaction if possible, so the code may look like: with repo.transaction(): .... if obsstore: rebasemod.rebase(...) obsstore.createmarkers(...) if not obsstore: rebasemod.rebase(...) repair.strip(...) That's ugly and error-prone. Ideally it's possible to just write: with repo.transaction(): rebasemod.rebase(...) saferemovenodes(...) This patch is the first step towards that. It adds a "delayedstrip" method to repair.py which maintains a postclose callback in the transaction object.

workingfilectx: add audit() as a wrapper for wvfs.audit()

workingfilectx: add backgroundclose as a kwarg to write() This is necessary because some callers in merge.py pass backgroundclose=True when writing. As with previous changes in this series, this should be a no-op.

merge: change repo.wvfs.setflags calls to a new wctx[f].setflags function As with previous changes in this series, this should be a no-op.

merge: convert repo.wwrite() calls to wctx[f].write() As with the previous patch in this series, workingfilectx.write() is a direct call to repo.wwrite(), so this change should be a no-op.

merge: replace repo.wvfs.unlinkpath() with calls to wctx[f].remove() The code inside workingfilectx.remove() is a straight call to repo.wvfs.unlinkpath, so this should be a no-op.

merge: pass wctx to batchremove and batchget We would like to migrate direct calls of repo.wvfs/wwrite/wread/etc to a call on the relevant workingfilectx, both as a cleanup (to reduce the number of working copy functions on `repo`), and also to facilitate an in-memory merge that doesn't write to the working copy. In order to do that, the first step is to ensure we pass the target wctx around and perform our writes and reads on it. Later, this object might become a memctx.

revset: add depth limit to descendants() (issue5374) This is naive implementation using two-pass scanning. Tracking descendants isn't an easy problem if both start and stop depths are specified. It's impractical to remember all possible depths of each node while scanning from roots to descendants because the number of depths explodes. Instead, we could cache (min, max) depths as a good approximation and track ancestors back when needed, but that's likely to have off-by-one bug. Since this implementation appears not significantly slower, and is quite straightforward, I think it's good enough for practical use cases. The time and space complexity is O(n) ish. revisions: 0) 1-pass scanning with (min, max)-depth cache (worst-case quadratic) 1) 2-pass scanning (this version) repository: mozilla-central # descendants(0) (for reference) *) 0.430353 # descendants(0, depth=1000) 0) 0.264889 1) 0.398289 # descendants(limit(tip:0, 1, offset=10000), depth=1000) 0) 0.025478 1) 0.029099 # descendants(0, depth=2000, startdepth=1000) 0) painfully slow (due to quadratic backtracking of ancestors) 1) 1.531138

dagop: make walk direction switchable so it can track descendants # ancestors(tip) using hg repo 2) 0.068527 3) 0.069097

dagop: factor out generator of ancestor nodes # ancestors(tip) using hg repo 1) 0.068976 2) 0.068527

dagop: factor out pfunc from revancestors() generator This generator will be reused for tracking descendants with depth limit. # ancestors(tip) using hg repo 0) 0.065868 1) 0.068976

dagop: use smartset.min() in revdescendants() generator All callers pass the result of revset.getset(), which should be a smartset.

dagop: change revdescendants() to include all root revisions Prepares for adding depth support. I want to process depth=0 in revdescendants() to make things simpler. only() also calls dagop.revdescendants(), but it filters out root revisions explicitly. So this should cause no problem. # descendants(0) using hg repo 0) 0.052380 1) 0.051226 # only(tip) using hg repo 0) 0.001433 1) 0.001425

test-revset: add a few more tests of descendants() I'll add depth support to descendants().

dagop: unnest inner generator of revdescendants() This just moves iterate() to module-level function.

smartset: fix default value of abstractsmartset.sort() It's unused, but it shouldn't lie.

keyword: wrap functions only once at loading keyword extension Before this patch, some functions are wrapped in reposetup(), but this causes redundant nested wrapping, if two ore more repositories enable keyword extension (e.g. hgweb serves multiple repositories). Now, there is no need to define these wrapper functions in reposetup(), because previous patches made them not directly refer to kwtemplater instanciated in reposetup(). This patch factors these wrapper functions out from reposetup(), and uses them to wrap functions only at once at loading keyword extension.

keyword: use _keywordkwt of repository instead of kwtools['templater'] Now, kwtemplater instance can be obtained via _keywordkwt property of repository.

keyword: obtain kwtemplater instance via repository at runtime Wrapper functions of keyword extension are defined in reposetup(), because they refer to kwtemplater instantiated in reposetup(). This patch makes them obtain kwtemplater instance via repository at runtime. For reviewability, this patch focuses on wrapper functions, which handle generator. This is a part of preparations for defining them statically.

keyword: obtain kwtemplater instance via repository at runtime Wrapper functions of keyword extension are defined in reposetup(), because they refer to kwtemplater instantiated in reposetup(). This patch makes them obtain kwtemplater instance via repository at runtime. This is a part of preparations for defining them statically.

keyword: make wrapped repository and kwtemplater refer to each other Wrapper functions of keyword extension are defined in reposetup(), because they refer to kwtemplater instantiated in reposetup(). But these functions can be defined statically, if kwtemplater can be obtained via repository at runtime. This is a part of preparations for defining them statically. To avoid cyclic reference, this patch makes kwtemplater use weakref to refer related repository instance.

keyword: add test for keyword expansion at serving multiple repositories This is safety for subsequent (and future) patches, which change function wrapping.

keyword: make comparison webcommand suppress keyword expansion Before this patch, diff in "comparison" webcommand doesn't suppress keyword expansion as same as diff output of other webcommands.

keyword: restore kwtemplater.match at the end of wrapped webcommands Before this patch, kwweb_skip doesn't restore kwtemplater.match after wrapped webcommands. This suppresses keyword expansion at wrapped webcommands. Typical usecase of this issue is "file" webcommand after annotate, changeset, filediff or so on. To ensure kwtemplater.match=util.never while original webcommand running, this patch makes kwweb_skip yield values returned by it, because it returns generator object.

keyword: restore kwtemplater.restrict at the end of wrapped patch.diff Before this patch, kwdiff doesn't restore kwtemplater.restrict after invocation of wrapped patch.diff(). This suppresses keyword expansion at subsequent filelog.read(). Typical usecase of this issue is "hg cat" after "hg diff" with command server. In this case, kwtemplater.restrict=True is kept in command server process even after "hg diff". To ensure kwtemplater.restrict=True while original patch.diff() running, this patch makes kwdiff() yield values returned by it, because it returns generator object. Strictly speaking, if filelog.read() is invoked before completely evaluating the result of previous patch.diff(), keyword expansion is still suppressed, because kwtemplater.restrict isn't restored yet. But this fixing should be reasonable enough, because patch.diff() is consumed immediately, AFAIK.

debugrevlog: align chain length, reach, and compression ratio I think this is what the max(...) exists for.

configitems: register 'ui.interactive' That item default value is a bit special (None) so this adds a second proof that everything is still working fine.

config: use '_config' within 'configbytes' This will prevent bugs from using None as the sentinel value (eg: 'ui.interactive')

config: use '_config' within 'configbool' This will prevent bugs from using None as the sentinel value (eg: 'ui.interactive')

config: extract the core config logic into a private method This will make it easier for the other 'configxxx' function to detect unset value.

debugrevlog: also display the largest delta chain span Mercurial read all data between the base of the chain and the last delta when restoring content (including unrelated delta). To monitor this, we add data about the size of the "delta chain span" to debugrevlog.

rebase: clean up rebasestate from active transaction Previously, rebase assumes the following pattern: rebase: with transaction as tr: # top-level ... tr.__close__ writes rebasestate unlink('rebasestate') However it's possible that "rebase" was called inside a transaction: with transaction as tr1: rebase: with transaction as tr2: # not top-level ... tr2.__close__ does not write rebasestate unlink('rebasestate') tr1.__close__ writes rebasestate That leaves a rebasestate on disk incorrectly. This patch adds "removefilegenerator" to notify transaction code that the state file is no longer needed therefore fixes the issue.

test-rebase: add a test showing rebasestate left behind The test demonstrates that .hg/rebasestate is left behind if "rebase" was called inside an existing transaction.

identify: rename 'changed' keyword -> 'dirty' I meant to do this before sending the initial templater support, but forgot. I'm quite surprised that 'dirty' doesn't occur in more user facing contexts, but there are a few, like the help for blackbox. It also more obviously mirrors the '(clean)' state printed by the summary command. I also didn't like that it was just one letter off from {changes} in the {latesttags} sub-keywords, which has a totally different meaning.

dispatch: remove unused _loaded Now, there is no user for dispatch._loaded.

extensions: register functions always at loading extension (issue5601) Before this patch, functions defined in extensions are registered via extra loaders only in _dispatch(). Therefore, loading extensions in other code paths like below omits registration of functions. - WSGI service - operation across repositories (e.g. subrepo) - test-duplicateoptions.py, using extensions.loadall() directly To register functions always at loading new extension, this patch moves implementation for extra loading from dispatch._dispatch() to extensions.loadall(). AFAIK, only commands module causes cyclic dependency between extensions module, but this patch imports all related modules just before extra loading in loadall(), in order to centralize them. This patch makes extensions.py depend on many other modules, even though extensions.py itself doesn't. It should be avoided if possible, but I don't have any better idea. Some other places like below aren't reasonable for extra loading, IMHO. - specific function in newly added module: existing callers of extensions.loadall() should invoke it, too - hg.repository() or so: no-repo commands aren't covered by this. BTW, this patch removes _loaded.add(name) on relocation, because dispatch._loaded is used only for extraloaders (for similar reason, "exts" variable is removed, too).