templatefilters: deprecate hgdate as {date|hgdate} is the default format

templater: make date wrapper support dot/map operations No idea if it will be useful, but it just works.

templater: introduce a wrapper for date tuple (BC) Strictly speaking, this is BC, but I believe the original string format (str(float(unixtime)) + str(int(tzoffset))) was just plain wrong.

templater: promote tomap() to an interface type I originally considered merging tomap() with itermaps()/getmember(), but decided to not. We might want to add support for chained map operations (e.g. {foo % func() % ...}), where func() will return a mappable object, and 'foo % func()' will be a mappedgenerator of mappable objects.

templater: rename mappable to hybriditem as it is the primary use case This frees up the name 'mappable' for new interface type.

serve: do not daemonize by default by --print-url Per discussion in D3649, -d/--daemon should be explicitly specified to avoid surprising result. Differential Revision: https://phab.mercurial-scm.org/D3703

mercurial: add .t files to .editorconfig The notable entry here prevents the stripping of trailing whitespace, which is important in .t tests. Differential Revision: https://phab.mercurial-scm.org/D3702

graft: add a new `--stop` flag to stop interrupted graft This patch adds a new flag `--stop` to `hg graft` command which stops the interrupted graft. The `--stop` flag takes back you to the last successful step i.e. it will keep your grafted commits, it will just clear the mergestate and interrupted graft state. The `--stop` is different from `--abort` flag as the latter also undoes all the work done till now which is sometimes not what the user wants. Suppose you grafted a lot of changesets, you encountered conflicts, you resolved them, did `hg graft --continue`, again encountered conflicts, continue, again encountered conflicts. Now you are tired of solving merge conflicts and want to resume this sometimes later. If you use the `--abort` functionality, it will strip your already grafted changesets, making you loose the work you have done resolving merge conflicts. A general goal related to this flag is to add this flag to `rebase` and `histedit` too. The evolve command already has this --stop flag. Tests are added for the new flag. .. feature:: `hg graft` now has a `--stop` flag to stop interrupted graft. Differential Revision: https://phab.mercurial-scm.org/D3668

graft: fix the help text to say `graft reapplies previous options` The options which graft accepts are: * rev * continue * edit * log * force * currentdate * currentuser * date * user * tool * dryrun Out of these following should be reapplied in case conflicts occur and user does `hg graft --continue`: * rev * log * force * currentuser * currentdate * date * user rev and force were already reapplied by --continue. Some recent refactors made graft to store the values of other options in graftstate and reapplies them at `hg graft --continue`. Now all the options which should be reapplied are taken care of, we can easily remove the help note which says that `--continue does not reapply earlier options` and help text saying that we reapplies earlier options. .. feature:: `hg graft --continue` now reapplies all the earlier options `hg graft` now preserves the value of all the flags and reapplies them in case of `hg graft --continue`. Differential Revision: https://phab.mercurial-scm.org/D3667

files: drop now-unnecessary filtering of removed dirstate files workingctx.matches() now filters out removed files. Differential Revision: https://phab.mercurial-scm.org/D3712

context: make workingctx.matches() filter our removed files (API) It seems surprising that workingctx.matches() includes files that had been removed in the working copy. The callers don't want that either (besides the `hg locate` that was changed in the previous patch). The only observable difference that I'm aware of is that `hg log -T 'wdir()' -r '{files(...)}'` will no longer include removed files (an improvement, IMO). That matches `hg files` (but does not match the deprecated `hg locate`). Differential Revision: https://phab.mercurial-scm.org/D3711

locate: explicitly use dirstate.matches() for working copy `hg locate` is odd in that it includes files that have been removed from the working copy. It relies on workingctx.matches() for that. I want to make workingctx.matches() not include removed files, so this patch makes `hg locate` handle the working copy differently instead. Differential Revision: https://phab.mercurial-scm.org/D3710

revsets: define a none() revset This can be useful for default values in corp-wide configuration (e.g. for commits to hide, which defaults to none()), and perhaps in scripts. I named it none() rather than empty() to match all(). Differential Revision: https://phab.mercurial-scm.org/D3713

statprof: fix save and load Fix these functions even if they don't are used at the moment. Thanks to Yuya Nishihara for spotting that.

perftemplating: abort on incompatible version This is what the other debug commands do. Thanks to Yuya Nishihara for pointing this out.

templater: fix string representation of wrapped None flatten() and stringify() skip None, which means wrappedvalue(None).show() must return '' instead of 'None'. This isn't a problem right now, but we'll encounter it once we start using wrapped types extensively.

templater: show slightly better hint on map operation error It was super hard to find where the templater went wrong if a keyword was evaluated to '' or None.

templater: inline unwraphybrid() flatten() is the solo user of this function.

templater: drop hybrid-ness on unwrapvalue() Proxy methods are no longer necessary as list/dict functions go through the wrapped interface.

templater: mark .keytype as a private attribute

templater: abstract ifcontains() over wrapped types This allows us to make .keytype private. There's a minor BC that a hybrid dict/list of keytype=None now strictly checks the type of the needle. For example, {ifcontains(rev, files)} no longer matches a file named "1" at the rev=1. I made this change for consistency with the get(dict, key) function. We can restore the old behavior by making keytype=bytes the default if desired.

templater: inline wraphybridvalue() All wraphybridvalue() calls are now handled in the hybrid class.

templater: abstract min/max away I'm not certain how many get*() functions I'll add to the wrapped types, but getmin() and getmax() will allow us to optimize a revset wrapper.

stringutil: fix prettyrepr() to not orphan foo=<...> line

debugwalk: pretty-print nested matcher While porting filesets to composition of matchers, I found the original one-liner was hard to read.

debugwalk: show matcher output only if -v/--verbose And drop "egrep -v". This matches the behavior of "hg debugrevspec".

stringutil: promote smartset.prettyformat() to utility function It will be used by debugwalk.

profiling: introduce a "profiling.time-track" option This option allows to switch the time used by the statistical profiler from cpu to real time. Our goal is to profile IO intensive operations using the mercurial profiler.

statprof: also gather wall time Gathering wall time information is useful for operations that are IO (files, network) intensive. For now we only expose it in the final summary.

perftemplating: allow to specify the template to test This is useful to test the evolution of rendering time for a given template.

perftemplating: stop going through the log command Only benchmark the rendering phase by moving steps outside of the timed function: * revisions resolution, * template parsing

perftemplating: move revision argument to flag only The revision we want to render are less important than the templates. We move revision specification behind the usual `--rev` flag.

perftemplating: drop usage of buffer The buffer can consume a lot of memory and change various internal behaviors. Writing to dev/null seems more appropriate when it comes to benchmark.

perftemplating: move template formating into its own function The lambda is not practical when time will come to alter the benchmark code.

perftemplating: fix the revisions argument Before this change, the argument passed to log where a string instead of a list of string. This meant only single character rev were supported... We now properly accepts argument of any length. We also make it possible to specify multiple REV arguments in the same go.

py3: make sure util.username() always returns bytes Differential Revision: https://phab.mercurial-scm.org/D3706

py3: add b'' prefix to make the regex bytes # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3705

phases: use "published" in the phase movement message Using "published" seems smoother than "became public" and more in line with the "phase.publishing" configuration.

phase: clarify the message about movement on command changeset The current message is a bit generic. Since we only print it for phase movement on changeset already common before the pull, we add "local" to the message in and attempt to clarify what changeset the phase movement affected.

fuzz: try and generate an interesting mpatch seed from a repo Differential Revision: https://phab.mercurial-scm.org/D3697

fuzz: structured helpers for creating mpatch seed corpus entries Maybe this is better. I'm not sold, honestly. Differential Revision: https://phab.mercurial-scm.org/D3696

fuzz: new fuzzer for the mpatch code Differential Revision: https://phab.mercurial-scm.org/D3695

run-tests: follow-up on the test-case format It turns out the original regex doesn't support real test cases names like the one Mercurial is using. Update the regex to being able to precisely select them on the command line. Differential Revision: https://phab.mercurial-scm.org/D3699

templater: resolve type of dict key in getmember() This seems more correct and is consistent with the future wrapped.contains() function, where a key type has to be resolved depending on a container type.

templater: promote getmember() to an interface of wrapped types

templater: move getdictitem() to hybrid class Since a raw dict will never be returned by evalwrapped(), we don't need to support d.get(key).

templater: add try-except stub to runmember() New wrapped interface will raise ParseError if the underlying object doesn't support dict-like lookup operation.

templater: do dict lookup over a wrapped object Dict/list lookup operations will be moved to a wrapped interface so that a returned element can inherit hybrid-ness automatically. wraphybridvalue() will be inlined.

merge with stable

graft: reuse the --log value passed initially in `hg graft --continue` (BC) We now stores the value of --log flag passed initially in the graftstate and reuse that value when doing `hg graft --continue` which is a nice behavior. The test updates demonstrate the fix. Since we now preserve the value by default, drop the mention of `--log` flag from the hint which we see after conflicts. Differential Revision: https://phab.mercurial-scm.org/D3662

graft: add test showing --continue not preserving --log passed earlier This patch shows that when we do `hg graft --continue`, the value of --log flag passed when initial graft was run was not used. Next patch will fix this and tests will help us in realising that. Differential Revision: https://phab.mercurial-scm.org/D3661

test-fuzz-targets: look for clang-6.0 binary as well Debian sid is still shipped with clang 4.0 by default. This allows me to run the test without replacing the system clang to clang-6.0.

test-fuzz-targets: rebuild executable silently This makes sure the build is not broken.

fuzz: fix "make clean" to pass even if no binaries built yet

fuzz: compile xdiff.cc with -std=c++17 Otherwise the build would fail as follows: In file included from xdiff.cc:13: ./fuzzutil.h:23:10: fatal error: 'third_party/absl/types/optional.h' file not found #include "third_party/absl/types/optional.h"

fuzz: fix use of undeclared function memcpy()

fuzz: fix the default make target

fuzz: expand variables by make Otherwise it wouldn't work unless CC and CXX were exported from the environment.

templater: always map over a wrapped object _checkeditermaps() is no longer necessary as the hgweb issue was resolved.

templater: consistently join() string-like object per character (BC) The old behavior was copied from join() of a lazy generator string, which was unified to the behavior of join() of a byte string by the previous patch. This patch fixes the mappable type to do the same.

templater: always join() over a wrapped object (BC) This is a behavior change in a sense that join() of a byte string is no longer "implementation dependent." Before, if a byte string was backed by a lazy generator, join() would concatenate each chunk with the specified separator, which seems wrong. The new behavior is always join() each byte. TypeError on join() over uniterable is also fixed.

templater: add wrapped types for pure non-list/dict values These wrapper types will allow us to get rid of some isinstance() business. A bytes object needs to support sequence-like operations (e.g. join(), ifcontains(), etc.) That's why we have two wrapper classes. Tests will be added later.

templater: unify unwrapvalue() with _unwrapvalue() All weird generators got removed from the hgweb codebase. We still have inconsistent behavior regarding join() of a byte string, which will be addressed later.

templater: inline unwrapvalue() The current unwrapvalue() will be superseded by _unwrapvalue(). Note that _unwrapvalue() can simply return thing.tovalue() if thing is a wrapped object. That's because tovalue() is guaranteed to not return a generator of strings.

run-tests: update the test case name format Manually typing parenthesis and spaces will be tedious when trying to launch a specific test case. I'm proposing a simpler format that is less hard to remember and type right. There was other possibilities envisaged like `::` or `!`, I think `#` is slight easier to type but I'm open to any suggestion on the new format. Differential Revision: https://phab.mercurial-scm.org/D3556

run-tests: add support for running specific test cases Differential Revision: https://phab.mercurial-scm.org/D3555

configitems: drop the deprecated alias for commands.update.check experimental.updatecheck was deprecated and should have been removed once 4.4 was released. Let's drop it now. Differential Revision: https://phab.mercurial-scm.org/D3679

graft: correct documentation about options can be reapplied Now options --user and --date can be reapplied after we hit a conflict. Differential Revision: https://phab.mercurial-scm.org/D3678

cmdutil: use internal separators when building the terse list Status uses internal separators, so this is more correct. See c974320d20b9 and 362096cfdb1f. A utility method is still needed, so that the paths starting with '/' aren't created when `self.path` is empty.

advanceboundary: add dryrun parameter Added logic to find those csets whose phase will be changed (when running without --dryrun) while advancing boundary and return those csets. Differential Revision: https://phab.mercurial-scm.org/D3671

grep: enable passing wdir as a revision When you pass wdir() to the -r flag, it catches the WdirUnsupported error and falls back to an alternate path. Differential Revision: https://phab.mercurial-scm.org/D3673

outgoing: drop an extraneous URL parse This was left over from dfb888aae17a. 'branches' wasn't used, and the only thing parseurl() would do is slice any branch fragment off 'dest'. But path.loc and path.pushloc already have the fragment removed.

config: rename allow_archive to allow-archive As part of ConfigConsolidationPlan [1], rename the option according to the new UI guidelines [2] and add an alias for backward compatibility. [1]: https://www.mercurial-scm.org/wiki/ConfigConsolidationPlan [2]: https://www.mercurial-scm.org/wiki/UIGuideline#adding_new_options

hgweb: fill {rename} even if ?linerange is specified Otherwise, {rename % ...} would fail. Hopefully this is the last hgweb patch so I can start fixing templater stuff.

hgweb: wrap {earlycommands} and {othercommands} of help with mappinggenerator They were generators of mappings.

hgweb: wrap {topics}es of help with mappinggenerator or mappinglist The former 'topics' was a generator function, and the latter was a list of mappings.

hgweb: wrap {edges} of {nodes} of graph with mappinglist This was a list of mappings.

hgweb: wrap {nodes} of graph with mappinggenerator It was a generator of mappings, which shouldn't be put in template mappings.

hgweb: adapt {jsdata} of graph to mappinggenerator The laziness is handled by the mappinggenerator class.

hgweb: wrap {nextentry} of graph with mappinglist It's a 0/1-length list of a mapping.

setup: write version constant as bytes literal Spotted while bulk-rewriting string literals to b''s.

spartan: don't show '[up]' in file view when in root directory already

monoblue: don't show '[up]' in file view when in root directory already

gitweb: don't show '[up]' in file view when in root directory already

paper: don't show '[up]' in file view when in root directory already

spartan: add missing closing tags Browsers apparently don't care about this, but let's fix it for humans.

templatefilters: add commonprefix The commonprefix filter takes a list of files names like files() and returns the longest directory name common to all elements. Differential Revision: https://phab.mercurial-scm.org/D3439

py3: add 3 new passing tests to whitelist We ave crossed the 500 mark! Differential Revision: https://phab.mercurial-scm.org/D3638

serve: add option print-url It can be used to open hgweb in a web browser. Example : xdg-open "`hg serve --print-url`". Differential Revision: https://phab.mercurial-scm.org/D3649

chg: fix an undefined behavior about memcpy Spot by Wez Furlong. `memcpy(x, NULL, 0)` is undefined according to [1]. [1]: https://stackoverflow.com/questions/5243012 Differential Revision: https://phab.mercurial-scm.org/D3698

Added signature for changeset 9c5ced5276d6

Added tag 4.6.1 for changeset 9c5ced5276d6

mpatch: avoid integer overflow in combine() (SEC) All the callers of this function can handle a NULL return, so that appears to be the "safe" way to report an error.

mpatch: avoid integer overflow in mpatch_decode (SEC)

mpatch: fix UB integer overflows in discard() (SEC)

mpatch: fix UB in int overflows in gather() (SEC)

mpatch: introduce a safesub() helper as well Same reason as safeadd().

mpatch: introduce a safeadd() helper to work around UB int overflow We're about to make extensive use of this. This change duplicates some stdbool.h portability hacks from cext/util.h. We should probably clean that up in the future, but we'll skip that for now in order to make security backports easier.

mpatch: ensure fragment start isn't past the end of orig (SEC) Caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0004. A CVE has not been obtained as of this writing.

mpatch: protect against underflow in mpatch_apply (SEC) Also caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0002. A CVE has not been obtained as of this writing.

mpatch: be more careful about parsing binary patch data (SEC) It appears to have been possible to trivially walk off the end of an allocated region with a malformed patch. Oops. Caught when writing an mpatch fuzzer for oss-fuzz. This defect is OVE-20180430-0001. A CVE has not been obtained as of this writing.

zstandard: pull in bug fixes from upstream 0.9.1 release (issue5884) This changeset contains the meaningful code changes from python-zstandard's 0.9.1 release. The main fix is to restore support for compiling with mingw.

templatefuncs: only render text portion of minirst.format() result When "keep" argument is provided, the function returns (text, pruned), where pruned is a list of sections from the original plain text that were pruned from the rendered result. Let's not output it together with the rendered HTML.

tests: demonstrate that hgweb renders "pruned" that minirst.format() returns Notice at the bottom of the help text there's "windows". It's a section that is in the original help text, but was pruned (because hgweb didn't ask for it).

rebase: avoid RevlogError when computing obsoletenotrebased (issue5907) The key to reproducing this seems to be missing an obsolete node that is not an ancestor of the destination.

rebase: prioritize indicating an interrupted rebase over update (issue5838) This should also cover the transplant extension, and any other non clearable states.

tests: demonstrate inconsistent messaging around interrupted rebases

tests: adapt test-check-pylint to run on Windows The line endings are explicitly converted because this was ending up with '\r (no-eol) (esc)' lines, in addition to the usual '\r (esc)' lines. I've seen the fakerc output on other recently installed systems though (10.13 and/or Fedora 26). Unfortunately, the output here uses '\\' on Windows, so glob away the whole path.

hghave: avoid a deadlock reading the child process's output The output of `pylint` is voluminous enough that it fills the buffer on Windows, and waited for the parent to read it out. But the parent was waiting on the child to exit. I'm not sure what the intent of setting `ret = -1` in the exception handler just above this was...

lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902) There are only a handful of methods that access repo attributes that are applied in reposetup(). The `diff` test covers all of the commands that call scmutil.prefetchfiles(). Along the way, I saw that adding files and upgrading the repo format were also problems (also tested here). I don't think running `hg serve` through the commandserver is sane, but I conditionalized both the capabilities and the wsgirequest handler because it's trivially correct. It doesn't look like there has ever been a caller of candownload(), so there's no test for that path. The upload case isn't testable, because uploadblobs() bails if there are no pointers. The requirement should be added any time pointers are introduced, and that would force the extension to be loaded specifically for the repo. This covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is promoted to LFS), and uploadblobsfromrevs(), which can be called by other extensions. I think readfromstore() and writetostore() are only reachable as a flag processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as that is seen, so I don't think those are a problem.

revlog: disallow setting uncompressed length to None Backed out changeset a4942675de6b

unionrepo: fill in uncompressed length of revlog entry It can be either -1 or a valid length, but shouldn't be None. IIUC, we can simply trust the length retrieved from the revlog to be overlaid. I don't bother thinking whether the compressed length can be copied as well. We'll need to fix it later.

fuzz: add clean target Differential Revision: https://phab.mercurial-scm.org/D3677

fuzzutil: make it possible to use absl when C++17 isn't supported Differential Revision: https://phab.mercurial-scm.org/D3676

fuzz: extract some common utilities and use modern C++ idioms Alex Gaynor suggested we should probably copy the left and right sides of diffs to new blocks so we can detect over-reads in the diffing code, and I agree. Once I got into that, I realized we should do things with C++17 idioms rather than keep using malloc() and free(). This change is the result. I tried to split it more than this and failed. Everything still compiles and works in the oss-fuzz container, so I think we can count on C++17 being available! Differential Revision: https://phab.mercurial-scm.org/D3675

tests: update fuzzer tests to include both fuzzers Differential Revision: https://phab.mercurial-scm.org/D3674

transaction-summary: show phase changes statistics in pull/unbundle Upon pull or unbundle, we display a message with the number of changesets which phase became public. Noticeably, this new message would appear even if no new changeset were added (below the "no changes found" message), thus indicating that something actually happened to the local repository.

outgoing: pay attention to `default:pushurl` for bookmarks and subrepos The problem here was that `default:pushurl` and `default` get translated to a single entry in `ui.paths` named 'default', with an attribute for 'pushloc', 'loc', and 'rawloc'. ui.expandpath() then always takes the `rawloc` attribute. Maybe the ui.expandpath() API is busted and should be removed? Or maybe getpath() should return a copy that adds an attribute reflecting the URL of the path chosen? I thought that I could remove the code in hg._outgoing() and pass the location resolved in commands.py as `dest`, but unfortunately that code is needed there to resolve #branch type URLs. Maybe that should be pulled up to commands.py, because I can't see any reasonable behavior for a subrepo path that's constructed out of that type of URL. The push command already resolves this early, so that works properly. But it looks like bundle, histedit, largefiles, patchbomb, and summary use a similar pattern, so they are likely similarly affected.

revlog: make chainbase cache its result for the correct revision Previously, as 'rev' was our iterator, we were always caching the chain base for the second revision of the chain, or for the base itself.

revlog: make getcandidaterevs more consistent about updating tested revs set Like in previous cases, update the set of tested revisions after yielding

templatekw: make getrenamed() return only filename, not nodeid No callers cared about the nodeid and I want to make getrenamed() not look up the node (although it's currently free, I hope to store copy info in changesets and not include the nodeid). Differential Revision: https://phab.mercurial-scm.org/D3666

py3: wrap tempfile.NamedTemporaryFile() to return bytes fp.name Unlike its name, tempfile.NamedTemporaryFile is not a class, so I renamed the pycompat version to look like a plain function. Since temp.name uses in the infinitepush extension aren't bytes-safe, this patch leaves them unmodified. Another weird thing is tempfile.mktemp(), which does not accept bytes suffix nor prefix. Sigh.

py3: wrap tempfile.mkdtemp() to use bytes path This also flips the default to use a bytes path on Python 3.

py3: wrap tempfile.mkstemp() to use bytes path This patch just flips the default to use a bytes path on Python 3. ca1cf9b3cce7 is backed out as the bundlepath should be bytes now.

extensions: remove strip_init=True from _disabledpaths() It's no longer used.

extensions: peek command table of disabled extensions without importing With chg where demandimport disabled, and if disk cache not warm, it took more than 5 seconds to get "unknown command" error when you typo a command name. This is horrible UX. The new implementation is less accurate than the original one as Python can do anything at import time and cmdtable may be imported from another module, but I think it's good enough. Note that the new implementation has to parse .py files, which is slightly slower than executing .pyc if demandimport is enabled.

lfs: clarify pointer validation error messages It wasn't obvious that LFS was involved from the error messages when `hg verify` fails.

terse: pconvert() entries added to the temporary terse dict for Windows Recent additional testing revealed this problem on Windows: --- tests/test-status.t.err +++ tests/test-status.t.err @@ -109,7 +109,7 @@ tweaking defaults works $ hg status --cwd a --config ui.tweakdefaults=yes - ? . + ? ../a/ ? ../b/ ? ../in_root $ HGPLAIN=1 hg status --cwd a --config ui.tweakdefaults=yes @@ -120,7 +120,7 @@ ? b/in_b (glob) ? in_root $ HGPLAINEXCEPT=tweakdefaults hg status --cwd a --config ui.tweakdefaults=yes - ? . + ? ..\a\ ? ../b/ ? ../in_root (glob) AFAICT, the status list (input and output here) is always in '/' format. The '\' printed output on Windows is because each file is run through repo.pathto() -> dirstate.pathto() -> util.pathto(). (And that function states that the argument uses '/' separators.) I fixed a similar issue in 362096cfdb1f, and given the apparent need for these strings to be in '/' format, I wonder if cmdutil.dirnode() should be rewritten to avoid os.path.join(). But it looks like all entries added to the temporary terse dict should use '/' now, and cmdutil.tersedir() looks like the only user.

filemerge: don't pass function name as loadpath's module_name param

phases: define an official tuple of phases we do not share Same motivation as for `mutablephases`, having a single definition helps with updating phases logic.

phases: define an official tuple of mutable phases Such tuple was already manually defined in a couple of place. Having an official definition makes it easy to introduce of new phases.

repoview: use 'phasecache.getrevset' when computing 'unserved' filter This reuses a precomputed set, being much faster than manual iteration. Computing the "unserved" filter speed up by 55%. before: wall 0.014671 comb 0.020000 user 0.020000 sys 0.000000 (best of 180) after: wall 0.006623 comb 0.010000 user 0.010000 sys 0.000000 (best of 401) The "unserved" filter is used to restrict the revisions client can pull from a server. (eg: secret changesets)

graft: drop --user and --date values info from hint in case of conflicts After previous patch, we have started preserving user and date values in graftstate and reusing them during `hg graft --continue`. Now passing --user and --date again with --continue makes no sense. Let's drop them from the hint. Differential Revision: https://phab.mercurial-scm.org/D3660

graft: reuse --user and --date values in `hg graft --continue` (BC) Reading the user and date information from graftstate during `hg graft --continue` will help us in preserving the user and date arguments passed when `hg graft` was called. This patch reads that information and reuses that while running `hg graft --continue`. So after this patch, --user and --date values are preserved even if conflicts occur and user don't need to pass them again. The test changes demonstrate the fix. This is a backward incompatible change but I think of this more as a bug fix. Also thinking about removing the line from `hg help graft` which says --continue does not reapply other flags but need to check what are the other flags which needs to be preserved. Differential Revision: https://phab.mercurial-scm.org/D3659

tests: add test showing --continue not preserving --date and --user flags This patch adds test showing that we don't preserve the user passed --date and --user values in `hg graft`. I was fixing that and realized this is untested. Adding tests before so that behavior change or the fix is easy to realize. Differential Revision: https://phab.mercurial-scm.org/D3658

graft: store user passed date and user information in graftstate Right now, `hg help graft` says: The -c/--continue option does not reapply earlier options, except for --force. which should be treated as a bug. A good user experience is that the commands remember the arguments passed initially and preserve them during `hg graft --continue`. This patch starts storing the user and date information in graftstate if user passed it. Upcoming patches will make sure we preserve that information during --continue and them don't allow user to pass any new arguments with --continue. I don't think there is any another `--continue` flag which allows new options to be passed with it. Differential Revision: https://phab.mercurial-scm.org/D3657

graft: use cmdstate.delete() to delete the state file Previous patches start using state.cmdstate() class for statefiles. The class has a function delete() to delete the state file. This patch replaces the existing repo.vfs.unlink() with that delete() function. Differential Revision: https://phab.mercurial-scm.org/D3656

graph: add outputgraph() function, called by ascii() to print the graph to the ui. This allows a cleaner entrypoint for extensions to tweak the graph output without needing to rewrite all of ascii(), or needing to manually guess where the graph nodes/edges end and the rev note portion begins. This patch does not affect graph output or behavior in any way. Differential Revision: https://phab.mercurial-scm.org/D3655

graft: add test for reading old graftstate files with new mechanism This tests the reading of old graftstate file using the new logic. The tests shows that if user is in middle of a graft and then updates their mercurial to the version where we have new graftstate format, we can still read the old graft state format files correctly. Differential Revision: https://phab.mercurial-scm.org/D2597

graft: start using the cmdstate class to read and write data to graftstate This patch replaces the logic to read and write data to graftstate file to use the state.cmdstate() class. The previous graftstate format didn't had any version number on top of that, so we have to catch the CorruptedState error and then read the graftstate in case of old state files. This will help us to implement nice additions to graft commands like `--no-commit`, `--abort`, `--stop` flags. Passing on test-graft.t shows that things are working fine. Differential Revision: https://phab.mercurial-scm.org/D3654

graft: use state.cmdstate() to check whether graftstate exists This is a step towards make graft use the new state.cmdstate() class. This patch replaces the ugly try-except with nice if-else conditionals. Differential Revision: https://phab.mercurial-scm.org/D3652

graft: factor out function to read graft state in separate function Fatcoring out the logic in a separate function will help us in adding conditional logic for different versions of graft state files. Differential Revision: https://phab.mercurial-scm.org/D3651

state: removing remaining instances of opts class variable The cmdstate class used to have a class variable opts which used to be a dict which stored all the data for the state. Recent cleanups removed the use of that variable. There were couple of instances left which are removed by this patch. Differential Revision: https://phab.mercurial-scm.org/D3653

help: mention pattern syntax of latesttag() template function

py3: replace str.format(x) with `str % x` in githelp

githelp: drop the trailing period from single sentence output for consistency There are several instances of multiple sentence output, which I left alone. That is already nonstandard style, so dropping the period doesn't seem like an improvement.

hgweb: wrap {entries}* of filelog with mappinglist They were lists of mappings.

hgweb: don't use dict(key=value) to build a mapping dict in filelog It wasn't Py3 compatible because mapping keys must be bytes.

hgweb: wrap {diffopts} of annotate by hybriddict() The diffopts here is a plain dict, which should be wrapped by hybriddict.

hgweb: wrap {parents} of {annotate} with mappinggenerator It's a generator of at most two mappings, which has to be wrapped.

hgweb: wrap {annotate} with mappinggenerator No bare generator of mappings should be put in a template mapping.

hgweb: wrap {bookmarks} of summary with mappinggenerator No bare generator of mappings should be put in a template mapping.

hgweb: wrap {lastchange} of bookmarks with mappinglist It was an 1-length list of a mapping, can be wrapped with a mappinglist.

hgweb: wrap {entries}* of bookmarks with mappinggenerator They were functions returning a generator of mappings. The laziness is handled by the mappinggenerator class.

state: raise CorruptedState error isntead of ProgrammingError There are old state files which don't have a version number in top of them and hence we have to read them to check whether they are good or not. ProgrammingError is not apt for this case. Thanks to Yuya for suggesting CorruptedState error. Differential Revision: https://phab.mercurial-scm.org/D3644

localrepo: add docstring to _makedirstate to make it less likely to be removed Differential Revision: https://phab.mercurial-scm.org/D3643

scmutil: move repair.stripbmrevset as scmutil.bookmarkrevs (API)

githelp: cleanup one more abort message This makes the string localizable, uses the more standard hint argument, quotes the problem option so it stands out, and kills a stray apostrophe.

githelp: do not concatenate i18n messages dynamically so they can be collected

narrow: only wrap dirstate functions once, instead of per-reposetup chg will call reposetup multiple times, and we would end up double-wrapping (or worse) the dirstate functions; this can cause issues like OSError 'No such file or directory' during rebase operations, when we go to double-delete our narrowspec backup file. Differential Revision: https://phab.mercurial-scm.org/D3559

state: temporary silence pyflakes warning by removing variable assignment The variable 'version' is going to be used in upcoming series where we will be using the version number to read a certain state file. However currently, pyflakes fails because of the variable not being used. Let's remove the assignment temporarily so that buildbots and test-suite is happy until I iterate over remaining part of the series. Differential Revision: https://phab.mercurial-scm.org/D3641

state: fix usage of an unassigned variable The variable iv was used in earlier iterations of the patches and was dropped since we made sure 'version' is always int. This usage of 'iv' should have been replaced by 'version' in the original patch only. Differential Revision: https://phab.mercurial-scm.org/D3640

py3: add b'' prefixes in tests/test-revset2.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3637

py3: use encoding.strfromlocal() instead of pycompat.sysstr() opts['dateformat'] is provided by the user and can be encoded in local encoding. Thanks to Yuya for catching. Differential Revision: https://phab.mercurial-scm.org/D3642

tests: stabilize test-patch.t on Windows $PYTHON needs to be quoted when invoking with cmd.exe, because the value expands to c:/Python27/python.exe, which seems to be interpreted as 'c' being a command. We can't just convert to '\', because there are a few places that run $PYTHON directly in MSYS. If unquoted there, it results in c:Python27python.exe being run. I wonder if we should bake the quotes into the environment variable to avoid this. It also wasn't happy with the quoting around exit1.py: c:/Python27/python.exe: can't open file ''$TESTTMP/d/exit1.py'': [Errno 22] Invalid argument

githelp: lowercase the start of output messages for consistency I left 'Mercurial' as a proper name capitalized.

githelp: rewrite a Yoda conditional

githelp: make several strings localizable

githelp: cleanly abort if the `svn` command is unknown Previously, the warning was lost above the sea of KeyError stacktrace output.

githelp: fail gracefully in a couple cases where arguments are missing I didn't bother adding tests because the other commands that already handled missing arguments don't test these edge cases. I didn't read over all of the code, rather I scanned for `args` not being checked before indexing.

revlog: isgooddeltainfo takes the whole revinfo object Future changes will need other information about te revision.

revlog: make variable name 'd' more explicit in _isgooddeltainfo d -> deltainfo

debugdeltachain: r.start and r.length can be retrieved outside the loop

revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta

pathencode: remove unused variable Caught by Yuya during review of 92ac9cf78dba. Differential Revision: https://phab.mercurial-scm.org/D3618

py3: add r'' prefixes to fix kwargs handling in hgext/sparse.py This fixes two of sparse tests which were failing on Python 3. # skip-blame because just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3634

py3: use bytes in tests/printenv.py This patch add b'' prefixes and adds some .encode() calls to convert str to bytes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3633

py3: use stringutil.pprint() to prevent b'' prefixes in output This patch uses stringutil.pprint to print a list with bytes in it to prevent the b'' prefixes in the output. Differential Revision: https://phab.mercurial-scm.org/D3632

py3: use pycompat.fsencode to convert path to bytes Differential Revision: https://phab.mercurial-scm.org/D3631

py3: add b'' prefixes in tests/test-ssh.t and tests/test-ssh-bundle1.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3630

py3: add b'' prefixes in contrib/hg-ssh # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3629

state: write the version number in plain text on top of state files We will soon be using CBOR format to write the data in state files. But we should not write the version number of the state files in CBOR format and we should rather write it in plain text because in future we can change the format of state files and we should be able to parse the version number of state file without requiring to understand a certain format. This will help us in making sure we have a good compatibility story with other versions of state files. Differential Revision: https://phab.mercurial-scm.org/D3579

state: set canonical=True to write deterministically Passing canonical=True to cbor.dump() will help in writing the data deterministically. This will sort all the sets and dicts before writing to the file. Thanks indygreg for recommending to use it. Differential Revision: https://phab.mercurial-scm.org/D3578

state: don't have a dict like interface for cmdstate class This patch changes the cmdstate class to stop having a dict like interface and delete the __nonzero__ function. After this patch, the save fuction takes a dict to store the data and read function returns a dict of the data stored. Differential Revision: https://phab.mercurial-scm.org/D3572

state: import the file to write state files from evolve extension The current way of writing state files is very obscure with each state file having it's own format to store state files. There is no centralized way to write state files in a good format. Moreover the current state files are not extensible, you cannot add more data to store in state files in reliable ways. To solve the problem, I wrote my own serialization and deserialization format, looked into existing formats like Protobuf, MessagePack, JSON but CBOR looks very promising and is suggested by people in the community. The current interface to store state files is to directly write data in files when things abort. Using the class imported by this commit, we can create objects which has a dict like interface and can store data on the object and store it on the file when things abort. The evolve extension is using the state file for `evolve`, `grab` commands and using it for resolution of orphaness, phase-divergence and content-divergence. The file is moved from changeset e4ac2e2c2086f977afa35e23a62f849e9305a225 of the evolve extension which is also tagged as 7.3.0. The following changes are made to the file while moving to core: * import util from current directory as this file in mercurial/ now * make cmdstate class extend object * removed mutable default value for opts in cmdstate.__init__ * some doc changes to replace out of core things with in-core ones evolve extension can be found at https://bitbucket.org/marmoute/mutable-history Differential Revision: https://phab.mercurial-scm.org/D2591

revlog: suggest other parent when a parent was refused for a delta (issue5481) Without aggressivemergedeltas, ensure that when we decline the closest parent (by revision number), the other parent is examined too.

status: add default of --terse=u to tweakdefaults (BC) This is in line with both Git and Subversion, and strikes me as a more humane behavior. Test output changes are expected once you read test-status.t, and I feel like we've still got adequate coverage on things. Differential Revision: https://phab.mercurial-scm.org/D3628

status: add a config knob for setting default of --terse I want --terse=u basically 100% of the time, but there's not a good way to do that before this patch. I'm very unhappy with how the default value for --terse looks rigt now, but it does *work*. The alternative would be to define an "optional string" flag type using fancyopts.customopt and then use that, leaving the default as None. Does anyone have a strong preference for that, or a better idea? Differential Revision: https://phab.mercurial-scm.org/D3627

tests: port inline extensions in test-obsolete.t to Python 3 Still some lingering failure in here, but it's much better. Differential Revision: https://phab.mercurial-scm.org/D3626

py3: consolidate down some (case ...) entries Differential Revision: https://phab.mercurial-scm.org/D3625

py3: re-sort test whitelist Differential Revision: https://phab.mercurial-scm.org/D3624

tests: fix test-subrepo-paths.t on Python 3 The Python 3 group reference error message is a little more helpful, so we'll allow the extra output. I tried matching this with an (re) line, but decided it wasn't worth the effort. Differential Revision: https://phab.mercurial-scm.org/D3623

tests: fix test-parseindex2 on Python 3 parsers.versionerrortext is a sysstr, but it's only ever used in this test on the Python side, so I'm okay to just handle it like this. Differential Revision: https://phab.mercurial-scm.org/D3622

tests: use [(] instead of \( to avoid warning on py3 From a recent test run of mine: run-tests.py:1633: DeprecationWarning: invalid escape sequence '\(' This lets us encode the expectation without the badness. Differential Revision: https://phab.mercurial-scm.org/D3621

tests: fix test-patch.t on pickier /bin/sh implementations FreeBSD sh(1) doesn't accept -d, so we weren't testing what we expected there. Let's just use a simple Python script instead. Differential Revision: https://phab.mercurial-scm.org/D3620

patch: add debug message to show external patch tool invocation Differential Revision: https://phab.mercurial-scm.org/D3619

py3: add 13 new passing tests to whitelist Differential Revision: https://phab.mercurial-scm.org/D3615

py3: check for None before comparing with integers Comparing None and integers on Python 3 is not allowed and raise error. Differential Revision: https://phab.mercurial-scm.org/D3614

py3: use `except error as e` instead of `except error, e` The latter throws SyntaxError on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3613

py3: use pycompat.fsencode() to convert tempfile name to bytes Differential Revision: https://phab.mercurial-scm.org/D3612

py3: add b'' prefixes in tests/test-parseindex.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3611

py3: fix .write() calls in few tests This patch adds b'' prefixes to make sure we write bytes and add `and None` in the end to suppress the output by .write() calls. Differential Revision: https://phab.mercurial-scm.org/D3610

py3: bytestr() bytes to get bytechar while iterating on it Iterating on bytes give you ascii values instead of bytechr so we need to wrap the bytes in pycompat.bytestr() to get bytechr while iterating. Differential Revision: https://phab.mercurial-scm.org/D3609

py3: slice over bytes to prevent getting the ascii values Differential Revision: https://phab.mercurial-scm.org/D3608

py3: use utils.stringutil.forcebytestr to convert error to bytes Differential Revision: https://phab.mercurial-scm.org/D3607

py3: make sure we open files in bytes mode we internally deal with bytes and should read files in bytes mode. Differential Revision: https://phab.mercurial-scm.org/D3606

py3: add b'' prefix in test/test-fileset.t # skip-blame because just a b'' prefix Differential Revision: https://phab.mercurial-scm.org/D3605

py3: suppress the output from .write() calls in few tests Differential Revision: https://phab.mercurial-scm.org/D3604

py3: use print as a function in tests/test-parseindex.t Differential Revision: https://phab.mercurial-scm.org/D3603

py3: use print as a function in tests/test-pull.t Differential Revision: https://phab.mercurial-scm.org/D3602

py3: add b'' prefixes in tests/test-rebuildstate.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3601

py3: add b'' prefixes in tests/test-progress.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3600

py3: add b'' prefixes in tests/test-profile.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3599

py3: add b'' prefixes in tests/test-pager.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3598

py3: add b'' prefixes in tests/test-hgrc.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3597

py3: add b'' prefixes in tests/test-extensions-afterloaded.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3596

py3: add b'' prefixes in tests/test-custom-filters.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3595

py3: add b'' prefixes in tests/test-bugzilla.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3594

py3: fix kwargs handling in hgext/split.py Differential Revision: https://phab.mercurial-scm.org/D3593

py3: add b'' prefixes in hgext/convert/subversion.py Litrerals starting with triple quotes or prepended with r'' do not get prepended by b'' by the transformer. # skip-blame because just adding b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3592

py3: use '%d' for integers in hgext/infinitepush/__init__.py Differential Revision: https://phab.mercurial-scm.org/D3591

py3: make sure we pass str to date.strftime() Differential Revision: https://phab.mercurial-scm.org/D3590

py3: use pycompat.bytestr() to convert str into bytes the keys of opts are str here, so we have to convert it to bytes before passing into error.Abort() Differential Revision: https://phab.mercurial-scm.org/D3617

py3: fix kwargs handling in hgext/rebase.py We add r'' prefixes to prevent the transformer adding b'' prefixes. # skip-blame because just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3616