py3: convert kwargs' keys to str before passing in cmdutil.getcommiteditor

diff: add a fast path to avoid loading binary contents When diffing binary contents, with certain configs, we can show "Binary file <name> has changed" without actual content. That allows a fast path where we could avoid providing actual binary contents. Note: in that case we still need to test if two contents are the same, that's done by using "filectx.cmp", which could have its own fast path.

diff: correct binary testing logic This seems to be more correct given the table drawn in the previous patch. Namely, "losedatafn" and "opts.git" are removed, "not opts.text" is added. - losedatafn: diff output (binary) should not be affected by "losedatafn" - opts.git: binary testing is helpful for detecting a fast path in the next path. the fast path can also be used if opts.git is False - opts.text: if it's set, we should treat the content as non-binary

diff: draw a table about binary diff behaviors The table should make it easier to reason about future changes.

diff: use fctx.size() to test empty fctx.size() could have a fast path that does not require loading content.

diff: use fctx.isbinary() to test binary The end goal is to avoid calling fctx.data() when unnecessary. For example, if diff.nobinary=1 and files are binary, the expected behavior is to print "Binary file has changed". That could avoid reading fctx.data() sometimes. This is mainly to enable an external LFS extension to skip expensive binary file loading sometimes (read: most of the time with diff.nobinary=1 and diff.text=0), without any behavior changes to mercurial (i.e. whether a file is LFS or not does not change any behavior, LFS could be 100% transparent to users).

pycompat: extract helper to raise exception with traceback It uses "raise excobj, None, tb" form which I think is simpler and more useful than "raise exctype, args, tb".

largefiles: make sure debugstate command is populated before wrapping Copied the hack from 869d660b8669, which seemed the simplest workaround. Perhaps debugcommands.py should have its own commands table.

check-code: ignore re-exports of os.environ in encoding.py These are valid uses of os.environ.

check-code: exclude demandimport.py and policy.py from Python 3 checks These modules can't depend on pycompat.py, which means we have to write Py3 hacks in them.

check-code: rewrite py3 exclusion pattern with negative lookahead I want to add more patterns, but negative lookbehind requires patterns of the same length so not useful.

cleanup: remove useless re-raises of KeyboardInterrupt KeyboardInterrupt is no longer a subclass of Exception since Python 2.5. https://docs.python.org/2/whatsnew/2.5.html#pep-352-exceptions-as-new-style-classes

make: drop deprecated rule to process temporary copy of pure modules Pure modules never be copied to mercurial/ since 511a4384b033.

help: describe about choice of :prompt as a fallback merge tool explicitly "merge-tools" help topic has described that the merge of the file fails if no tool is found to merge binary or symlink, since c77f6276c9e7 (or Mercurial 1.7), which based on (already removed) MergeProgram wiki page. But even at that revision, and of course now, merge of the file doesn't fail automatically for binary/symlink. ":prompt" (or equivalent logic) is used, if there is no appropriate tool configuration for binary/symlink.

wix: only one KeyPath is allowed per Component

dirstate: optimize walk() by using match.visitdir() We already have the logic for restricting directory walks in match.visitdir() that we use for treemanifests. We should take advantage of it when walking the working copy as well. This speeds up "hg st -I rootfilesin:." on the Firefox repo from 0.587s to 0.305s on warm disk (and much more on cold disk). More time is spent reading the dirstate than walking the working copy after. I tried to find scenarios where calling match.visitdir() would be a noticeable overhead, but I couldn't find any. I encourage the reader to try for themselves, since this is performance-critical code.

match: optimize visitdir() for patterns matching only root directory Because _rootsanddirs() returns a list of directories to visit recursively and a list of directories to visit non-recursively. For patterns such as 'rootfilesin:foo/bar', we clearly need to visit the directory foo/bar, but we also need to visit its parents. The method therefore uses util.dirs() to find the parent directories of 'foo/bar'. That method does not include the root directory, but since we obviously need to visit the root directory, we always added '.' to the set of directories to visit non-recursively. The visitdir() method had special handling to consider set(['.']) to mean that no includes had been specified and would thus visit all directories. However, when the pattern is 'rootfilesin:.', set(['.']) is actually the real set of directories to visit and the special handling of that set meant that all directories got visited instead of just the root directory. The fix is simple: add '.' to the set of parent directories in _rootsanddirs() and stop treating set(['.']) specially. This makes hg files -r . -I rootfilesin:. in a treemanifest version of the Firefox repo go from 1.5s to 0.26s on warm disk (and a *much* bigger improvement on cold disk). Note that the -I is necessary for no good reason. We just haven't optimized visitdir() for regular (non-include, non-exclude) patterns yet.

rebase: don't update state dict same way for each root The update statement does not depend on anything in the loop, so just move it before the loop and do it once. There are no cases where update would happen 0 times before (and 1 now); the function returns early in all such cases.

forget: access status fields by name, not index

demandimport: add urwid.command_map to ignore list The useful pudb debugger can be used with Mercurial, but its import of urwid fails when demandimport is enabled. Add urwid.command_map to the ignore list so pudb can be used with hg without disabling all of demandimport.

outgoing: run on filtered repo outgoing has been using an unfiltered repo since fe67107094fd (discovery: outgoing pass unfiltered repo to findcommonincoming (issue3776), 2013-01-28). If I'm reading code and history correctly, it should be safe to run _outgoing() on a filtered repo since c5456b64eb07 (discovery: run discovery on filtered repository, 2015-01-07). By running _outgoing() on a filtered repo, we can also remove the workaround there for ignoring filtered revisions.

manifest: remove check for non-contexts in _dirmancache It looks like the _dirmancache has contained only manifest contexts since d79c141fdf41 (manifest: remove usages of manifest.read, 2016-11-10).

bundle: factor the 'getchangegroup' out The call in the two branches is identical, so we can just issue it outside of the conditional.

bundle: avoid reset of the 'outgoing' variable We have a cleaner way to achieve the same effect. Not resetting the variable will help us to simplify the code.

changegroup: deprecate 'getlocalchangroup' (API) We have 'getchangegroup' with a shorter name for the exactly same feature. Now that all users are gone we can formally deprecate it.

tests: directly 'getchangegroup' It is identical to 'getlocalchangegroup' with a shorter name.

exchange: directly 'getchangegroup' It is identical to 'getlocalchangegroup' with a shorter name.

commands: directly 'getchangegroup' It is identical to 'getlocalchangegroup' with a shorter name.

changegroup: deduplicate 'getlocalchangegroup' The two functions 'getlocalchangegroup' and 'getchangegroup' have been strictly identical for multiple years ('getchangegroup' had a deprecated docstring) We'll drop one of them (getlocalchangegroup, since it has the longest name). However, we needs to migrate all users of the dropped one to the new one before we can deprecate it. In the mean time we drop one of the duplicated definition and the outdated docstring.

py3: add test to show 'hg log -Tjson' works

py3: add test to show 'hg log -G' works

py3: rename test-check-py3-commands.t to test-py3-commands.t test-check-*.t is a set of tests which tests certain coding style checks. So this test was wrongly named, thanks to marmoute for pointing this out.

py3: use list of bytes rather than bytestring while extending bytes into lists Python 2: >>> ls = [] >>> ls.extend(b'abc') >>> ls ['a', 'b', 'c'] Python 3: >>> ls = [] >>> ls.extend(b'abc') >>> ls [97, 98, 99] So to make sure things work fine in Py3, this patch does: >>> ls = [] >>> ls.extend([b'a', b'b', b'c']) >>> ls [b'a', b'b', b'c'] After this `hg log -G` works!

py3: use pycompat.byteskwargs to converts kwargs to bytes baseformatter._item must contain both keys and values in bytes. So to make sure that, we convert the opts back to bytes.

py3: make adefaults keys str to be compatible with getattr getattr passes a str value of the attribute to be looked and keys in adefaults dict are bytes which resulted in AttributeError. This patch abuses r'' to make the keys str.

py3: abuse r'' to access keys in keyword arguments

py3: use pycompat.bytechr instead of chr

py3: use %d to format integers into bytestrings

py3: use pycompat.bytestr instead of bytes

py3: slice over bytes to prevent getting ascii values

py3: use encoding.unitolocal instead of .encode(encoding.encoding)

rebase: use matcher to optimize manifestmerge The old merge code would call manifestmerge and calculate the complete diff between the source to the destination. In many cases, like rebase, the vast majority of differences between the source and destination are irrelevant because they are differences between the destination and the common ancestor only, and therefore don't affect the merge. Since most actions are 'keep', all the effort to compute them is wasted. Instead, let's compute the difference between the source and the common ancestor and only perform the diff of those files against the merge destination. When using treemanifest, this lets us avoid loading almost the entire tree when rebasing from a very old ancestor. This speeds up rebase of an old stack of 27 commits by 20x. In mozilla-central, without treemanifest, when rebasing a commit from default~100000 to default, this speeds up the manifestmerge step from 2.6s to 1.2s. However, the additional diff adds an overhead to all manifestmerge calls, especially for flat manifests. When rebasing a commit from default~1 to default it appears to add 100ms in mozilla-central. While we could put this optimization behind a flag, I think the fact that it makes merge O(number of changes being applied) instead of O(number of changes between X and Y) justifies it.

changegroup: delete unused 'bundlecaps' argument (API)

localrepo: reuse exchange.bundle2requested() It seems like localrepo.getbundle() is trying to do the same thing, so let's just call the method. That way we get the same condition as there (matching any "HG2" prefix, not only "HG20").

py3: use raw strings while accessing class.__dict__ The keys of class.__dict__ are unicodes on Python 3.

py3: handle opts correctly for `hg add` opts in add command were passed again to cmdutil.add() as kwargs so we need to convert them again to str. Intstead we convert them to bytes when passing scmutil.match(). Opts handling is also corrected for all the functions which are called from cmdutil.add().

py3: handle opts correctly for rollback dryrun and force are just check for None, the value is not used. So its better to leave opts as unicodes as that wont harm us.

py3: handle opts correctly for unbundle There is just a check whether the value is None or not. So even having optupdate as unicodes won't harm us.

py3: convert opts to bytes in cmdutil.dorecord() commands.commit() calls cmdutil.dorecord() where opts are passed as unicodes being keyword arguments. This patch converts them back to bytes as they are required.

py3: make sure opts are passed and used correctly in help command opts are converted back to bytes in help.help_() where they are used. Before that it's ensured that we have a bytes value for keep variable.

py3: handle opts uniformly in commands.py Since keyword arguments can't be bytes on Python 3, we converted then to unicodes before passing into different command functions. We need to adopt a certain pattern to convert opts back to bytes. Following are some of the functions which are called from inside these command functions and should always be feeded bytes to follow the right behaviour. ui.fomattter() scmutil.match() patch.diffallopts() hg.peer() cmdutil.{show_changeset|copy|graphrevs|checkunsupportedflag} server.{createservice|runservice} There are few commands which are left out where opts is again passed to a function as keyword arguments or converting opts back to bytes is kind of not necessary. Those are cat, revert, help, unbundle and rollback. Following patches will deal with them. This patch apart from these five commands, convert opts back to bytes for rest of the commands. This fixes a lot of things which are hidden like --git works now. Similarly more flags of commands which run on Python 3 currently get fixed.

merge with stable

help: call out specific replacement configuration settings As an aside, I'm having trouble parsing the help text meaning for HG when it is unset or empty. How can it be the frozen name or searched if it is empty?

help: spelling fixes

help: attempt to clarify that pager usage is not output length based This may be too subtle of a change to get the point across, but when I first read the original text, I thought maybe the pager would only be invoked if writing more than a screenful. The distinction between this and a pager that simply exits after printing less than a screenful is important on Windows, given the inability of `more` to color output.

help: document color/pager pitfalls on Windows Even though I figured this out a few weeks ago, I was initially puzzled where the color went when I upgraded to 4.2 on a different Windows machine. Let's point users reading the help into the right direction. I wonder if we should be even more explicit about cmd.exe/MSYS/pager/color interplay, but at least all of the breadcrumbs are here (I think).

webcommands: use fctx.isbinary

annotate: use fctx.isbinary

fileset: use fctx.isbinary instead of util.binary(fctx.data()) filectx provides "isbinary" to test if the data is binary. Let's use it. This enables other filectx implementations (like LFS) to override the isbinary test.

internals: document that "branches" is a legacy wire command Modern clients use a different discovery mechanism.