perftemplating: abort on incompatible version This is what the other debug commands do. Thanks to Yuya Nishihara for pointing this out.

templater: fix string representation of wrapped None flatten() and stringify() skip None, which means wrappedvalue(None).show() must return '' instead of 'None'. This isn't a problem right now, but we'll encounter it once we start using wrapped types extensively.

templater: show slightly better hint on map operation error It was super hard to find where the templater went wrong if a keyword was evaluated to '' or None.

templater: inline unwraphybrid() flatten() is the solo user of this function.

templater: drop hybrid-ness on unwrapvalue() Proxy methods are no longer necessary as list/dict functions go through the wrapped interface.

templater: mark .keytype as a private attribute

templater: abstract ifcontains() over wrapped types This allows us to make .keytype private. There's a minor BC that a hybrid dict/list of keytype=None now strictly checks the type of the needle. For example, {ifcontains(rev, files)} no longer matches a file named "1" at the rev=1. I made this change for consistency with the get(dict, key) function. We can restore the old behavior by making keytype=bytes the default if desired.

templater: inline wraphybridvalue() All wraphybridvalue() calls are now handled in the hybrid class.

templater: abstract min/max away I'm not certain how many get*() functions I'll add to the wrapped types, but getmin() and getmax() will allow us to optimize a revset wrapper.

stringutil: fix prettyrepr() to not orphan foo=<...> line

debugwalk: pretty-print nested matcher While porting filesets to composition of matchers, I found the original one-liner was hard to read.

debugwalk: show matcher output only if -v/--verbose And drop "egrep -v". This matches the behavior of "hg debugrevspec".

stringutil: promote smartset.prettyformat() to utility function It will be used by debugwalk.

profiling: introduce a "profiling.time-track" option This option allows to switch the time used by the statistical profiler from cpu to real time. Our goal is to profile IO intensive operations using the mercurial profiler.

statprof: also gather wall time Gathering wall time information is useful for operations that are IO (files, network) intensive. For now we only expose it in the final summary.

perftemplating: allow to specify the template to test This is useful to test the evolution of rendering time for a given template.

perftemplating: stop going through the log command Only benchmark the rendering phase by moving steps outside of the timed function: * revisions resolution, * template parsing

perftemplating: move revision argument to flag only The revision we want to render are less important than the templates. We move revision specification behind the usual `--rev` flag.

perftemplating: drop usage of buffer The buffer can consume a lot of memory and change various internal behaviors. Writing to dev/null seems more appropriate when it comes to benchmark.

perftemplating: move template formating into its own function The lambda is not practical when time will come to alter the benchmark code.

perftemplating: fix the revisions argument Before this change, the argument passed to log where a string instead of a list of string. This meant only single character rev were supported... We now properly accepts argument of any length. We also make it possible to specify multiple REV arguments in the same go.

py3: make sure util.username() always returns bytes Differential Revision: https://phab.mercurial-scm.org/D3706

py3: add b'' prefix to make the regex bytes # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3705

phases: use "published" in the phase movement message Using "published" seems smoother than "became public" and more in line with the "phase.publishing" configuration.

phase: clarify the message about movement on command changeset The current message is a bit generic. Since we only print it for phase movement on changeset already common before the pull, we add "local" to the message in and attempt to clarify what changeset the phase movement affected.

fuzz: try and generate an interesting mpatch seed from a repo Differential Revision: https://phab.mercurial-scm.org/D3697

fuzz: structured helpers for creating mpatch seed corpus entries Maybe this is better. I'm not sold, honestly. Differential Revision: https://phab.mercurial-scm.org/D3696

fuzz: new fuzzer for the mpatch code Differential Revision: https://phab.mercurial-scm.org/D3695

run-tests: follow-up on the test-case format It turns out the original regex doesn't support real test cases names like the one Mercurial is using. Update the regex to being able to precisely select them on the command line. Differential Revision: https://phab.mercurial-scm.org/D3699

templater: resolve type of dict key in getmember() This seems more correct and is consistent with the future wrapped.contains() function, where a key type has to be resolved depending on a container type.

templater: promote getmember() to an interface of wrapped types

templater: move getdictitem() to hybrid class Since a raw dict will never be returned by evalwrapped(), we don't need to support d.get(key).

templater: add try-except stub to runmember() New wrapped interface will raise ParseError if the underlying object doesn't support dict-like lookup operation.

templater: do dict lookup over a wrapped object Dict/list lookup operations will be moved to a wrapped interface so that a returned element can inherit hybrid-ness automatically. wraphybridvalue() will be inlined.

merge with stable

graft: reuse the --log value passed initially in `hg graft --continue` (BC) We now stores the value of --log flag passed initially in the graftstate and reuse that value when doing `hg graft --continue` which is a nice behavior. The test updates demonstrate the fix. Since we now preserve the value by default, drop the mention of `--log` flag from the hint which we see after conflicts. Differential Revision: https://phab.mercurial-scm.org/D3662

graft: add test showing --continue not preserving --log passed earlier This patch shows that when we do `hg graft --continue`, the value of --log flag passed when initial graft was run was not used. Next patch will fix this and tests will help us in realising that. Differential Revision: https://phab.mercurial-scm.org/D3661

test-fuzz-targets: look for clang-6.0 binary as well Debian sid is still shipped with clang 4.0 by default. This allows me to run the test without replacing the system clang to clang-6.0.

test-fuzz-targets: rebuild executable silently This makes sure the build is not broken.

fuzz: fix "make clean" to pass even if no binaries built yet

fuzz: compile xdiff.cc with -std=c++17 Otherwise the build would fail as follows: In file included from xdiff.cc:13: ./fuzzutil.h:23:10: fatal error: 'third_party/absl/types/optional.h' file not found #include "third_party/absl/types/optional.h"

fuzz: fix use of undeclared function memcpy()

fuzz: fix the default make target

fuzz: expand variables by make Otherwise it wouldn't work unless CC and CXX were exported from the environment.

templater: always map over a wrapped object _checkeditermaps() is no longer necessary as the hgweb issue was resolved.

templater: consistently join() string-like object per character (BC) The old behavior was copied from join() of a lazy generator string, which was unified to the behavior of join() of a byte string by the previous patch. This patch fixes the mappable type to do the same.

templater: always join() over a wrapped object (BC) This is a behavior change in a sense that join() of a byte string is no longer "implementation dependent." Before, if a byte string was backed by a lazy generator, join() would concatenate each chunk with the specified separator, which seems wrong. The new behavior is always join() each byte. TypeError on join() over uniterable is also fixed.

templater: add wrapped types for pure non-list/dict values These wrapper types will allow us to get rid of some isinstance() business. A bytes object needs to support sequence-like operations (e.g. join(), ifcontains(), etc.) That's why we have two wrapper classes. Tests will be added later.

templater: unify unwrapvalue() with _unwrapvalue() All weird generators got removed from the hgweb codebase. We still have inconsistent behavior regarding join() of a byte string, which will be addressed later.

templater: inline unwrapvalue() The current unwrapvalue() will be superseded by _unwrapvalue(). Note that _unwrapvalue() can simply return thing.tovalue() if thing is a wrapped object. That's because tovalue() is guaranteed to not return a generator of strings.

run-tests: update the test case name format Manually typing parenthesis and spaces will be tedious when trying to launch a specific test case. I'm proposing a simpler format that is less hard to remember and type right. There was other possibilities envisaged like `::` or `!`, I think `#` is slight easier to type but I'm open to any suggestion on the new format. Differential Revision: https://phab.mercurial-scm.org/D3556

run-tests: add support for running specific test cases Differential Revision: https://phab.mercurial-scm.org/D3555

configitems: drop the deprecated alias for commands.update.check experimental.updatecheck was deprecated and should have been removed once 4.4 was released. Let's drop it now. Differential Revision: https://phab.mercurial-scm.org/D3679

graft: correct documentation about options can be reapplied Now options --user and --date can be reapplied after we hit a conflict. Differential Revision: https://phab.mercurial-scm.org/D3678

cmdutil: use internal separators when building the terse list Status uses internal separators, so this is more correct. See c974320d20b9 and 362096cfdb1f. A utility method is still needed, so that the paths starting with '/' aren't created when `self.path` is empty.

advanceboundary: add dryrun parameter Added logic to find those csets whose phase will be changed (when running without --dryrun) while advancing boundary and return those csets. Differential Revision: https://phab.mercurial-scm.org/D3671

grep: enable passing wdir as a revision When you pass wdir() to the -r flag, it catches the WdirUnsupported error and falls back to an alternate path. Differential Revision: https://phab.mercurial-scm.org/D3673

outgoing: drop an extraneous URL parse This was left over from dfb888aae17a. 'branches' wasn't used, and the only thing parseurl() would do is slice any branch fragment off 'dest'. But path.loc and path.pushloc already have the fragment removed.

config: rename allow_archive to allow-archive As part of ConfigConsolidationPlan [1], rename the option according to the new UI guidelines [2] and add an alias for backward compatibility. [1]: https://www.mercurial-scm.org/wiki/ConfigConsolidationPlan [2]: https://www.mercurial-scm.org/wiki/UIGuideline#adding_new_options

hgweb: fill {rename} even if ?linerange is specified Otherwise, {rename % ...} would fail. Hopefully this is the last hgweb patch so I can start fixing templater stuff.

hgweb: wrap {earlycommands} and {othercommands} of help with mappinggenerator They were generators of mappings.

hgweb: wrap {topics}es of help with mappinggenerator or mappinglist The former 'topics' was a generator function, and the latter was a list of mappings.

hgweb: wrap {edges} of {nodes} of graph with mappinglist This was a list of mappings.

hgweb: wrap {nodes} of graph with mappinggenerator It was a generator of mappings, which shouldn't be put in template mappings.

hgweb: adapt {jsdata} of graph to mappinggenerator The laziness is handled by the mappinggenerator class.

hgweb: wrap {nextentry} of graph with mappinglist It's a 0/1-length list of a mapping.

setup: write version constant as bytes literal Spotted while bulk-rewriting string literals to b''s.

spartan: don't show '[up]' in file view when in root directory already

monoblue: don't show '[up]' in file view when in root directory already

gitweb: don't show '[up]' in file view when in root directory already

paper: don't show '[up]' in file view when in root directory already

spartan: add missing closing tags Browsers apparently don't care about this, but let's fix it for humans.

templatefilters: add commonprefix The commonprefix filter takes a list of files names like files() and returns the longest directory name common to all elements. Differential Revision: https://phab.mercurial-scm.org/D3439

py3: add 3 new passing tests to whitelist We ave crossed the 500 mark! Differential Revision: https://phab.mercurial-scm.org/D3638

serve: add option print-url It can be used to open hgweb in a web browser. Example : xdg-open "`hg serve --print-url`". Differential Revision: https://phab.mercurial-scm.org/D3649

revlog: disallow setting uncompressed length to None Backed out changeset a4942675de6b

unionrepo: fill in uncompressed length of revlog entry It can be either -1 or a valid length, but shouldn't be None. IIUC, we can simply trust the length retrieved from the revlog to be overlaid. I don't bother thinking whether the compressed length can be copied as well. We'll need to fix it later.

fuzz: add clean target Differential Revision: https://phab.mercurial-scm.org/D3677

fuzzutil: make it possible to use absl when C++17 isn't supported Differential Revision: https://phab.mercurial-scm.org/D3676

fuzz: extract some common utilities and use modern C++ idioms Alex Gaynor suggested we should probably copy the left and right sides of diffs to new blocks so we can detect over-reads in the diffing code, and I agree. Once I got into that, I realized we should do things with C++17 idioms rather than keep using malloc() and free(). This change is the result. I tried to split it more than this and failed. Everything still compiles and works in the oss-fuzz container, so I think we can count on C++17 being available! Differential Revision: https://phab.mercurial-scm.org/D3675

tests: update fuzzer tests to include both fuzzers Differential Revision: https://phab.mercurial-scm.org/D3674

transaction-summary: show phase changes statistics in pull/unbundle Upon pull or unbundle, we display a message with the number of changesets which phase became public. Noticeably, this new message would appear even if no new changeset were added (below the "no changes found" message), thus indicating that something actually happened to the local repository.

outgoing: pay attention to `default:pushurl` for bookmarks and subrepos The problem here was that `default:pushurl` and `default` get translated to a single entry in `ui.paths` named 'default', with an attribute for 'pushloc', 'loc', and 'rawloc'. ui.expandpath() then always takes the `rawloc` attribute. Maybe the ui.expandpath() API is busted and should be removed? Or maybe getpath() should return a copy that adds an attribute reflecting the URL of the path chosen? I thought that I could remove the code in hg._outgoing() and pass the location resolved in commands.py as `dest`, but unfortunately that code is needed there to resolve #branch type URLs. Maybe that should be pulled up to commands.py, because I can't see any reasonable behavior for a subrepo path that's constructed out of that type of URL. The push command already resolves this early, so that works properly. But it looks like bundle, histedit, largefiles, patchbomb, and summary use a similar pattern, so they are likely similarly affected.

revlog: make chainbase cache its result for the correct revision Previously, as 'rev' was our iterator, we were always caching the chain base for the second revision of the chain, or for the base itself.

revlog: make getcandidaterevs more consistent about updating tested revs set Like in previous cases, update the set of tested revisions after yielding

templatekw: make getrenamed() return only filename, not nodeid No callers cared about the nodeid and I want to make getrenamed() not look up the node (although it's currently free, I hope to store copy info in changesets and not include the nodeid). Differential Revision: https://phab.mercurial-scm.org/D3666

py3: wrap tempfile.NamedTemporaryFile() to return bytes fp.name Unlike its name, tempfile.NamedTemporaryFile is not a class, so I renamed the pycompat version to look like a plain function. Since temp.name uses in the infinitepush extension aren't bytes-safe, this patch leaves them unmodified. Another weird thing is tempfile.mktemp(), which does not accept bytes suffix nor prefix. Sigh.

py3: wrap tempfile.mkdtemp() to use bytes path This also flips the default to use a bytes path on Python 3.

py3: wrap tempfile.mkstemp() to use bytes path This patch just flips the default to use a bytes path on Python 3. ca1cf9b3cce7 is backed out as the bundlepath should be bytes now.

extensions: remove strip_init=True from _disabledpaths() It's no longer used.

extensions: peek command table of disabled extensions without importing With chg where demandimport disabled, and if disk cache not warm, it took more than 5 seconds to get "unknown command" error when you typo a command name. This is horrible UX. The new implementation is less accurate than the original one as Python can do anything at import time and cmdtable may be imported from another module, but I think it's good enough. Note that the new implementation has to parse .py files, which is slightly slower than executing .pyc if demandimport is enabled.

lfs: clarify pointer validation error messages It wasn't obvious that LFS was involved from the error messages when `hg verify` fails.

terse: pconvert() entries added to the temporary terse dict for Windows Recent additional testing revealed this problem on Windows: --- tests/test-status.t.err +++ tests/test-status.t.err @@ -109,7 +109,7 @@ tweaking defaults works $ hg status --cwd a --config ui.tweakdefaults=yes - ? . + ? ../a/ ? ../b/ ? ../in_root $ HGPLAIN=1 hg status --cwd a --config ui.tweakdefaults=yes @@ -120,7 +120,7 @@ ? b/in_b (glob) ? in_root $ HGPLAINEXCEPT=tweakdefaults hg status --cwd a --config ui.tweakdefaults=yes - ? . + ? ..\a\ ? ../b/ ? ../in_root (glob) AFAICT, the status list (input and output here) is always in '/' format. The '\' printed output on Windows is because each file is run through repo.pathto() -> dirstate.pathto() -> util.pathto(). (And that function states that the argument uses '/' separators.) I fixed a similar issue in 362096cfdb1f, and given the apparent need for these strings to be in '/' format, I wonder if cmdutil.dirnode() should be rewritten to avoid os.path.join(). But it looks like all entries added to the temporary terse dict should use '/' now, and cmdutil.tersedir() looks like the only user.

filemerge: don't pass function name as loadpath's module_name param

phases: define an official tuple of phases we do not share Same motivation as for `mutablephases`, having a single definition helps with updating phases logic.

phases: define an official tuple of mutable phases Such tuple was already manually defined in a couple of place. Having an official definition makes it easy to introduce of new phases.

repoview: use 'phasecache.getrevset' when computing 'unserved' filter This reuses a precomputed set, being much faster than manual iteration. Computing the "unserved" filter speed up by 55%. before: wall 0.014671 comb 0.020000 user 0.020000 sys 0.000000 (best of 180) after: wall 0.006623 comb 0.010000 user 0.010000 sys 0.000000 (best of 401) The "unserved" filter is used to restrict the revisions client can pull from a server. (eg: secret changesets)

graft: drop --user and --date values info from hint in case of conflicts After previous patch, we have started preserving user and date values in graftstate and reusing them during `hg graft --continue`. Now passing --user and --date again with --continue makes no sense. Let's drop them from the hint. Differential Revision: https://phab.mercurial-scm.org/D3660

graft: reuse --user and --date values in `hg graft --continue` (BC) Reading the user and date information from graftstate during `hg graft --continue` will help us in preserving the user and date arguments passed when `hg graft` was called. This patch reads that information and reuses that while running `hg graft --continue`. So after this patch, --user and --date values are preserved even if conflicts occur and user don't need to pass them again. The test changes demonstrate the fix. This is a backward incompatible change but I think of this more as a bug fix. Also thinking about removing the line from `hg help graft` which says --continue does not reapply other flags but need to check what are the other flags which needs to be preserved. Differential Revision: https://phab.mercurial-scm.org/D3659

tests: add test showing --continue not preserving --date and --user flags This patch adds test showing that we don't preserve the user passed --date and --user values in `hg graft`. I was fixing that and realized this is untested. Adding tests before so that behavior change or the fix is easy to realize. Differential Revision: https://phab.mercurial-scm.org/D3658

graft: store user passed date and user information in graftstate Right now, `hg help graft` says: The -c/--continue option does not reapply earlier options, except for --force. which should be treated as a bug. A good user experience is that the commands remember the arguments passed initially and preserve them during `hg graft --continue`. This patch starts storing the user and date information in graftstate if user passed it. Upcoming patches will make sure we preserve that information during --continue and them don't allow user to pass any new arguments with --continue. I don't think there is any another `--continue` flag which allows new options to be passed with it. Differential Revision: https://phab.mercurial-scm.org/D3657

graft: use cmdstate.delete() to delete the state file Previous patches start using state.cmdstate() class for statefiles. The class has a function delete() to delete the state file. This patch replaces the existing repo.vfs.unlink() with that delete() function. Differential Revision: https://phab.mercurial-scm.org/D3656

graph: add outputgraph() function, called by ascii() to print the graph to the ui. This allows a cleaner entrypoint for extensions to tweak the graph output without needing to rewrite all of ascii(), or needing to manually guess where the graph nodes/edges end and the rev note portion begins. This patch does not affect graph output or behavior in any way. Differential Revision: https://phab.mercurial-scm.org/D3655

graft: add test for reading old graftstate files with new mechanism This tests the reading of old graftstate file using the new logic. The tests shows that if user is in middle of a graft and then updates their mercurial to the version where we have new graftstate format, we can still read the old graft state format files correctly. Differential Revision: https://phab.mercurial-scm.org/D2597

graft: start using the cmdstate class to read and write data to graftstate This patch replaces the logic to read and write data to graftstate file to use the state.cmdstate() class. The previous graftstate format didn't had any version number on top of that, so we have to catch the CorruptedState error and then read the graftstate in case of old state files. This will help us to implement nice additions to graft commands like `--no-commit`, `--abort`, `--stop` flags. Passing on test-graft.t shows that things are working fine. Differential Revision: https://phab.mercurial-scm.org/D3654

graft: use state.cmdstate() to check whether graftstate exists This is a step towards make graft use the new state.cmdstate() class. This patch replaces the ugly try-except with nice if-else conditionals. Differential Revision: https://phab.mercurial-scm.org/D3652

graft: factor out function to read graft state in separate function Fatcoring out the logic in a separate function will help us in adding conditional logic for different versions of graft state files. Differential Revision: https://phab.mercurial-scm.org/D3651

state: removing remaining instances of opts class variable The cmdstate class used to have a class variable opts which used to be a dict which stored all the data for the state. Recent cleanups removed the use of that variable. There were couple of instances left which are removed by this patch. Differential Revision: https://phab.mercurial-scm.org/D3653

help: mention pattern syntax of latesttag() template function

py3: replace str.format(x) with `str % x` in githelp

githelp: drop the trailing period from single sentence output for consistency There are several instances of multiple sentence output, which I left alone. That is already nonstandard style, so dropping the period doesn't seem like an improvement.

hgweb: wrap {entries}* of filelog with mappinglist They were lists of mappings.

hgweb: don't use dict(key=value) to build a mapping dict in filelog It wasn't Py3 compatible because mapping keys must be bytes.

hgweb: wrap {diffopts} of annotate by hybriddict() The diffopts here is a plain dict, which should be wrapped by hybriddict.

hgweb: wrap {parents} of {annotate} with mappinggenerator It's a generator of at most two mappings, which has to be wrapped.

hgweb: wrap {annotate} with mappinggenerator No bare generator of mappings should be put in a template mapping.

hgweb: wrap {bookmarks} of summary with mappinggenerator No bare generator of mappings should be put in a template mapping.

hgweb: wrap {lastchange} of bookmarks with mappinglist It was an 1-length list of a mapping, can be wrapped with a mappinglist.

hgweb: wrap {entries}* of bookmarks with mappinggenerator They were functions returning a generator of mappings. The laziness is handled by the mappinggenerator class.

state: raise CorruptedState error isntead of ProgrammingError There are old state files which don't have a version number in top of them and hence we have to read them to check whether they are good or not. ProgrammingError is not apt for this case. Thanks to Yuya for suggesting CorruptedState error. Differential Revision: https://phab.mercurial-scm.org/D3644

localrepo: add docstring to _makedirstate to make it less likely to be removed Differential Revision: https://phab.mercurial-scm.org/D3643

scmutil: move repair.stripbmrevset as scmutil.bookmarkrevs (API)

githelp: cleanup one more abort message This makes the string localizable, uses the more standard hint argument, quotes the problem option so it stands out, and kills a stray apostrophe.

githelp: do not concatenate i18n messages dynamically so they can be collected

narrow: only wrap dirstate functions once, instead of per-reposetup chg will call reposetup multiple times, and we would end up double-wrapping (or worse) the dirstate functions; this can cause issues like OSError 'No such file or directory' during rebase operations, when we go to double-delete our narrowspec backup file. Differential Revision: https://phab.mercurial-scm.org/D3559

state: temporary silence pyflakes warning by removing variable assignment The variable 'version' is going to be used in upcoming series where we will be using the version number to read a certain state file. However currently, pyflakes fails because of the variable not being used. Let's remove the assignment temporarily so that buildbots and test-suite is happy until I iterate over remaining part of the series. Differential Revision: https://phab.mercurial-scm.org/D3641

state: fix usage of an unassigned variable The variable iv was used in earlier iterations of the patches and was dropped since we made sure 'version' is always int. This usage of 'iv' should have been replaced by 'version' in the original patch only. Differential Revision: https://phab.mercurial-scm.org/D3640

py3: add b'' prefixes in tests/test-revset2.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3637

py3: use encoding.strfromlocal() instead of pycompat.sysstr() opts['dateformat'] is provided by the user and can be encoded in local encoding. Thanks to Yuya for catching. Differential Revision: https://phab.mercurial-scm.org/D3642

tests: stabilize test-patch.t on Windows $PYTHON needs to be quoted when invoking with cmd.exe, because the value expands to c:/Python27/python.exe, which seems to be interpreted as 'c' being a command. We can't just convert to '\', because there are a few places that run $PYTHON directly in MSYS. If unquoted there, it results in c:Python27python.exe being run. I wonder if we should bake the quotes into the environment variable to avoid this. It also wasn't happy with the quoting around exit1.py: c:/Python27/python.exe: can't open file ''$TESTTMP/d/exit1.py'': [Errno 22] Invalid argument

githelp: lowercase the start of output messages for consistency I left 'Mercurial' as a proper name capitalized.

githelp: rewrite a Yoda conditional

githelp: make several strings localizable

githelp: cleanly abort if the `svn` command is unknown Previously, the warning was lost above the sea of KeyError stacktrace output.

githelp: fail gracefully in a couple cases where arguments are missing I didn't bother adding tests because the other commands that already handled missing arguments don't test these edge cases. I didn't read over all of the code, rather I scanned for `args` not being checked before indexing.

revlog: isgooddeltainfo takes the whole revinfo object Future changes will need other information about te revision.

revlog: make variable name 'd' more explicit in _isgooddeltainfo d -> deltainfo

debugdeltachain: r.start and r.length can be retrieved outside the loop

revlog: in _getcandidaterevs, shorten revlog._generaldelta to gdelta

pathencode: remove unused variable Caught by Yuya during review of 92ac9cf78dba. Differential Revision: https://phab.mercurial-scm.org/D3618

py3: add r'' prefixes to fix kwargs handling in hgext/sparse.py This fixes two of sparse tests which were failing on Python 3. # skip-blame because just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3634

py3: use bytes in tests/printenv.py This patch add b'' prefixes and adds some .encode() calls to convert str to bytes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3633

py3: use stringutil.pprint() to prevent b'' prefixes in output This patch uses stringutil.pprint to print a list with bytes in it to prevent the b'' prefixes in the output. Differential Revision: https://phab.mercurial-scm.org/D3632

py3: use pycompat.fsencode to convert path to bytes Differential Revision: https://phab.mercurial-scm.org/D3631

py3: add b'' prefixes in tests/test-ssh.t and tests/test-ssh-bundle1.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3630

py3: add b'' prefixes in contrib/hg-ssh # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3629

state: write the version number in plain text on top of state files We will soon be using CBOR format to write the data in state files. But we should not write the version number of the state files in CBOR format and we should rather write it in plain text because in future we can change the format of state files and we should be able to parse the version number of state file without requiring to understand a certain format. This will help us in making sure we have a good compatibility story with other versions of state files. Differential Revision: https://phab.mercurial-scm.org/D3579

state: set canonical=True to write deterministically Passing canonical=True to cbor.dump() will help in writing the data deterministically. This will sort all the sets and dicts before writing to the file. Thanks indygreg for recommending to use it. Differential Revision: https://phab.mercurial-scm.org/D3578

state: don't have a dict like interface for cmdstate class This patch changes the cmdstate class to stop having a dict like interface and delete the __nonzero__ function. After this patch, the save fuction takes a dict to store the data and read function returns a dict of the data stored. Differential Revision: https://phab.mercurial-scm.org/D3572

state: import the file to write state files from evolve extension The current way of writing state files is very obscure with each state file having it's own format to store state files. There is no centralized way to write state files in a good format. Moreover the current state files are not extensible, you cannot add more data to store in state files in reliable ways. To solve the problem, I wrote my own serialization and deserialization format, looked into existing formats like Protobuf, MessagePack, JSON but CBOR looks very promising and is suggested by people in the community. The current interface to store state files is to directly write data in files when things abort. Using the class imported by this commit, we can create objects which has a dict like interface and can store data on the object and store it on the file when things abort. The evolve extension is using the state file for `evolve`, `grab` commands and using it for resolution of orphaness, phase-divergence and content-divergence. The file is moved from changeset e4ac2e2c2086f977afa35e23a62f849e9305a225 of the evolve extension which is also tagged as 7.3.0. The following changes are made to the file while moving to core: * import util from current directory as this file in mercurial/ now * make cmdstate class extend object * removed mutable default value for opts in cmdstate.__init__ * some doc changes to replace out of core things with in-core ones evolve extension can be found at https://bitbucket.org/marmoute/mutable-history Differential Revision: https://phab.mercurial-scm.org/D2591

revlog: suggest other parent when a parent was refused for a delta (issue5481) Without aggressivemergedeltas, ensure that when we decline the closest parent (by revision number), the other parent is examined too.

status: add default of --terse=u to tweakdefaults (BC) This is in line with both Git and Subversion, and strikes me as a more humane behavior. Test output changes are expected once you read test-status.t, and I feel like we've still got adequate coverage on things. Differential Revision: https://phab.mercurial-scm.org/D3628

status: add a config knob for setting default of --terse I want --terse=u basically 100% of the time, but there's not a good way to do that before this patch. I'm very unhappy with how the default value for --terse looks rigt now, but it does *work*. The alternative would be to define an "optional string" flag type using fancyopts.customopt and then use that, leaving the default as None. Does anyone have a strong preference for that, or a better idea? Differential Revision: https://phab.mercurial-scm.org/D3627

tests: port inline extensions in test-obsolete.t to Python 3 Still some lingering failure in here, but it's much better. Differential Revision: https://phab.mercurial-scm.org/D3626

py3: consolidate down some (case ...) entries Differential Revision: https://phab.mercurial-scm.org/D3625

py3: re-sort test whitelist Differential Revision: https://phab.mercurial-scm.org/D3624

tests: fix test-subrepo-paths.t on Python 3 The Python 3 group reference error message is a little more helpful, so we'll allow the extra output. I tried matching this with an (re) line, but decided it wasn't worth the effort. Differential Revision: https://phab.mercurial-scm.org/D3623

tests: fix test-parseindex2 on Python 3 parsers.versionerrortext is a sysstr, but it's only ever used in this test on the Python side, so I'm okay to just handle it like this. Differential Revision: https://phab.mercurial-scm.org/D3622

tests: use [(] instead of \( to avoid warning on py3 From a recent test run of mine: run-tests.py:1633: DeprecationWarning: invalid escape sequence '\(' This lets us encode the expectation without the badness. Differential Revision: https://phab.mercurial-scm.org/D3621

tests: fix test-patch.t on pickier /bin/sh implementations FreeBSD sh(1) doesn't accept -d, so we weren't testing what we expected there. Let's just use a simple Python script instead. Differential Revision: https://phab.mercurial-scm.org/D3620

patch: add debug message to show external patch tool invocation Differential Revision: https://phab.mercurial-scm.org/D3619

py3: add 13 new passing tests to whitelist Differential Revision: https://phab.mercurial-scm.org/D3615

py3: check for None before comparing with integers Comparing None and integers on Python 3 is not allowed and raise error. Differential Revision: https://phab.mercurial-scm.org/D3614

py3: use `except error as e` instead of `except error, e` The latter throws SyntaxError on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3613

py3: use pycompat.fsencode() to convert tempfile name to bytes Differential Revision: https://phab.mercurial-scm.org/D3612

py3: add b'' prefixes in tests/test-parseindex.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3611

py3: fix .write() calls in few tests This patch adds b'' prefixes to make sure we write bytes and add `and None` in the end to suppress the output by .write() calls. Differential Revision: https://phab.mercurial-scm.org/D3610

py3: bytestr() bytes to get bytechar while iterating on it Iterating on bytes give you ascii values instead of bytechr so we need to wrap the bytes in pycompat.bytestr() to get bytechr while iterating. Differential Revision: https://phab.mercurial-scm.org/D3609

py3: slice over bytes to prevent getting the ascii values Differential Revision: https://phab.mercurial-scm.org/D3608

py3: use utils.stringutil.forcebytestr to convert error to bytes Differential Revision: https://phab.mercurial-scm.org/D3607

py3: make sure we open files in bytes mode we internally deal with bytes and should read files in bytes mode. Differential Revision: https://phab.mercurial-scm.org/D3606

py3: add b'' prefix in test/test-fileset.t # skip-blame because just a b'' prefix Differential Revision: https://phab.mercurial-scm.org/D3605

py3: suppress the output from .write() calls in few tests Differential Revision: https://phab.mercurial-scm.org/D3604

py3: use print as a function in tests/test-parseindex.t Differential Revision: https://phab.mercurial-scm.org/D3603

py3: use print as a function in tests/test-pull.t Differential Revision: https://phab.mercurial-scm.org/D3602

py3: add b'' prefixes in tests/test-rebuildstate.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3601

py3: add b'' prefixes in tests/test-progress.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3600

py3: add b'' prefixes in tests/test-profile.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3599

py3: add b'' prefixes in tests/test-pager.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3598

py3: add b'' prefixes in tests/test-hgrc.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3597

py3: add b'' prefixes in tests/test-extensions-afterloaded.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3596

py3: add b'' prefixes in tests/test-custom-filters.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3595

py3: add b'' prefixes in tests/test-bugzilla.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3594

py3: fix kwargs handling in hgext/split.py Differential Revision: https://phab.mercurial-scm.org/D3593

py3: add b'' prefixes in hgext/convert/subversion.py Litrerals starting with triple quotes or prepended with r'' do not get prepended by b'' by the transformer. # skip-blame because just adding b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3592

py3: use '%d' for integers in hgext/infinitepush/__init__.py Differential Revision: https://phab.mercurial-scm.org/D3591

py3: make sure we pass str to date.strftime() Differential Revision: https://phab.mercurial-scm.org/D3590

py3: use pycompat.bytestr() to convert str into bytes the keys of opts are str here, so we have to convert it to bytes before passing into error.Abort() Differential Revision: https://phab.mercurial-scm.org/D3617

py3: fix kwargs handling in hgext/rebase.py We add r'' prefixes to prevent the transformer adding b'' prefixes. # skip-blame because just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3616

hgweb: wrap {entries}* of tags with mappinggenerator They were functions returning a generator of mappings. The laziness is handled by the mappinggenerator class.

hgweb: wrap {fentries} and {dentries} of manifest with mappinggenerator They were functions returning a generator of mappings.

hgweb: wrap {entries}* of changelog with mappinglist They were lists of mappings.

hgweb: wrap {lines} of filerevision with mappinggenerator No bare generator of mappings should be put in a template mapping.

hgweb: drop unused argument 'tmpl' from webutil.diffstat()

hgweb: use template context to render {diffstat} This is a preferred way to process nested templates.

hgweb: wrap {diffstat} with mappedgenerator This can't be a mappinggenerator since the associated templates are switched per item. We already have webutil.diffstatgen(). That's why the generator function is named as _diffstattmplgen().

pathencode: improve error messages slightly Both of these are conditions we should never hit, so we can afford to be more verbose to make debugging less baffling. Differential Revision: https://phab.mercurial-scm.org/D3589

pathencode: hashlib.sha1() takes bytes not str on Python 3 Differential Revision: https://phab.mercurial-scm.org/D3588

pathencode: fix importing hashlib on Python 3 I'm curious why PyImport_Import is returning NULL, but PyImport_ImportModule bypasses some of the regular import path, probably including demandimport, which I suspect is the issue here. Differential Revision: https://phab.mercurial-scm.org/D3587

context: fix %-formatting on Python 3 changeid could be an int or a bytestr, so use our pycompat wrapper. Differential Revision: https://phab.mercurial-scm.org/D3585

patch: fix import-time syntax error in test-check-module-imports.t This is more correct, though we were getting lucky in practice with our rewriter saving us. # skip-blame just a b prefix Differential Revision: https://phab.mercurial-scm.org/D3584

py3: whitelist two more passing tests observed by buildbot Differential Revision: https://phab.mercurial-scm.org/D3583

py3: use stringutil.pprint() to format a list to print Differential Revision: https://phab.mercurial-scm.org/D3582

py3: fix kwargs handling in qgurad in hgext/mq.py # skip-blame because just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3581

py3: use .startswith() instead of bytes[0] bytes[0] returns the ascii value of character at 0 index. Differential Revision: https://phab.mercurial-scm.org/D3580

crecord: fallback to text mode if diffs are too big for curses mode crecord uses curses.newpad to create a region that we can then scroll around in by moving the main 'screen' as a veiwport into the (probably larger than the actual screen) pad. Internally, at least in ncurses, pads are implemented using windows, which have their dimensions limited to a certain size. Depending on compilation options for ncurses, this size might be pretty small: (signed) short, or it might be larger ((signed) int). crecord wants to have enough room to have all of the contents of the main area of the chunkselector in the pad; this means that the full size with everything expanded must be less than these (undocumented, afaict) limits. It's not easy to write tests for this because the limits are platform- and installation- dependent and undocumented / unqueryable, as far as I can tell. Differential Revision: https://phab.mercurial-scm.org/D3577

narrow: filter copies in core Differential Revision: https://phab.mercurial-scm.org/D3576

narrow: filter set of files to check for case-folding to core Differential Revision: https://phab.mercurial-scm.org/D3575

narrow: filter merge actions in core Part of the ongoing effort to move narrow into core. Differential Revision: https://phab.mercurial-scm.org/D3574

hgweb: extract code for emitting multiple changelist records changelistentry() exists so extensions can easily monkeypatch the function to add additional metadata. In at least one case at Mozilla, we have an extension doing this where performance would greatly benefit if we were able to "batch fetch" metadata for all revisions that will eventually have their data set. By extracting the logic for "obtain records for N revisions" into a standalone function, we make it possible for a monkeypatched function to more efficiently bulk fetch data for N revisions. Differential Revision: https://phab.mercurial-scm.org/D3573

phabricator: register config settings I didn't bother registering the deprecated phabricator.auth.*, and I'm not sure if the two flagged as 'developer config' should be moved to [devel] (or why there's a distinction between `repophid` and `callsign`).

filemerge: support specifying a python function to custom merge-tools Eliminates the need to specify a python executable, which may not exist on system. Additionally launching script inprocess aids portability on systems that can't execute python via the shell. Example usage "merge-tools.myTool.executable=python:c:\myTool.py:mergefn" where myTool.py contains a function: "def mergefn(ui, repo, args, **kwargs):" where args is list of args passed to merge tool. (by default, expanded: $local $base $other) Invoking the specified python function was done by exposing and invoking (hook._pythonhook -> hook.pythonhook)

test-http-branchmap: fix stdio mode on Windows

test-http-branchmap: fix encoding test to wrap the server stream Wrapping sys.std* streams has no effect since 39d13b8c101d, "py3: bulk replace sys.stdin/out/err by util's."

test-merge-tools: create repo directory to free $TESTTMP for temporary files

notify: add option to include function names in the diff output This is a localized version of diff.showfunc. Differential Revision: https://phab.mercurial-scm.org/D3553

tests: update py3 test since json fields are now sorted The fields have been sorted since 814151cd8c4a (logcmdutil: rewrite jsonchangeset printer to be backed by jsonformatter, 2018-04-13), and the test case has thus been failing since that commit. Differential Revision: https://phab.mercurial-scm.org/D3570

tests: test-parseindex.t works just fine with chg In 538e850ae737 I annotated tests that failed with chg with "#require no-chg", but did not investigate why the tests were failing. This test was failing for unrelated reasons (I forgot to rebuild hg) and so should not be marked as no-chg. Test Plan: make local pushd contrib/chg make clean; make popd cd tests for f in '' --with-chg=../contrib/chg/chg; do ./run-tests.py --local $f test-parseindex.t done Differential Revision: https://phab.mercurial-scm.org/D3571

py3: convert the report to bytes Differential Revision: https://phab.mercurial-scm.org/D3569

json: reject unicode on py2 as well This makes it consistent with the behavior on py3. Differential Revision: https://phab.mercurial-scm.org/D3536

dispatch: mask negative exit code recorded in blackbox log That's what we do for the exit code delivered to the environment.

dispatch: fix exit code of unhandled exception recorded in blackbox log Spotted by Martin von Zweigbergk. We might want to change the exit code to -1 (i.e. 255) because 1 means non-abort error in hg, but that's another issue.

tests: mark tests that fail when using chg as #require no-chg As far as I can tell, most of these failures are due to using $HGPORT, which it seems chg might be using itself? I don't know enough to debug these failures to fix them properly. Differential Revision: https://phab.mercurial-scm.org/D3562

py3: use pycompat.bytestr() instead of str() in extensions.py Differential Revision: https://phab.mercurial-scm.org/D3568

py3: use pycompat.bytestr() on bytes before %r-ing it Differential Revision: https://phab.mercurial-scm.org/D3567

py3: add b'' prefixes in tests/test-extension.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3564

py3: suppress the value returned by .write() calls These return values are not present on Python 2. Differential Revision: https://phab.mercurial-scm.org/D3563

tests: test failure reporting in blackbox code Differential Revision: https://phab.mercurial-scm.org/D3560

export: add -B option to select a bookmark Just like `hg email -B`, `hg strip -B`, supports -B in export to select a list of changesets reachable from a bookmark.

packaging: move most packaging targets to own Makefile Since we're putting everything packaging under one roof, let's define the make targets related to packaging there as well. I didn't move the "osx" target because it is non-trivial. Some targets did change slightly as part of the move. But it was mostly around path normalization. Differential Revision: https://phab.mercurial-scm.org/D3552

packaging: make packaging scripts less reliant on pwd The scripts currently assume they are executed from the repo root. This feels like an arbitrary restriction. Let's fix that. Differential Revision: https://phab.mercurial-scm.org/D3551

packaging: move contrib/macosx to contrib/packaging/ And update various references throughout the code. Differential Revision: https://phab.mercurial-scm.org/D3550

packaging: move linux-wheel-centos5-blacklist to contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3549

packaging: move build-linux-wheels.sh to contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3548

packaging: move contrib/debian to contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3547

packaging: move buildrpm to contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3546

packaging: move mercurial.spec to contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3545

packaging: move builddeb into contrib/packaging/ Differential Revision: https://phab.mercurial-scm.org/D3544

packaging: move packagelib.sh into contrib/packaging/ This required some minor path adjustments in scripts referencing this path. Differential Revision: https://phab.mercurial-scm.org/D3543

packaging: move some docker scripts into contrib/packaging/ This required some minor path adjustments in the scripts. Differential Revision: https://phab.mercurial-scm.org/D3542

packaging: move most of contrib/docker to contrib/packaging/docker contrib/ is kind of a dumping ground for all kinds of random things. Many of the files in contrib/ are related to packaging. But sorting out how everything is related is a lot of effort since it is all mixed together. This commit starts the process of refactoring everything related to packaging so it all lives under one roof. Most content from contrib/docker is simply moved to contrib/packaging/docker and all references are updated. Differential Revision: https://phab.mercurial-scm.org/D3541

setup: reformat options argument This makes things a bit more easier to manage IMO. Differential Revision: https://phab.mercurial-scm.org/D3540

contrib: remove fixpax.py This was used as part of bdist_mpkg. And we stopped using bdist_mpkg after db5084d27df5. Differential Revision: https://phab.mercurial-scm.org/D3539

revlog: handle errors from index_node() in nt_insert() and index_slice_del() Same idea as in a9d9802d577e (revlog: don't say "not found" on internal error, 2018-05-04). Differential Revision: https://phab.mercurial-scm.org/D3558

help: add new deprecated topic Add a deprecated topic which lists most of the commands and options that are not recommended anymore.

phabricator: split auth.url into the standard auth.schemes and auth.prefix It seems better to reuse the existing function to find the proper [auth] block, even if not all of the possible settings may be of interest. The other callers of readauthforuri() make a trip through the password database to fetch the user from the URI. But in the little experimenting I did here, the username always came back as None. Since readauthforuri() wants it to make sure that user@prefix matches user@url, it seems that parsing the URL and pulling out the user component should be equivalent.

phabricator: migrate [phabricator.auth] to [auth] This resurrects the warning mechanism removed in 20a4543e9a2b. I'm not worried about the copy/paste of the code for a brief transitional period.

dispatch: unify handling of None returned by a command function A command function may return None in place of 0 just for convenience, but dispatch() doesn't need to inherit that property. This patch makes it be friendly to callers.

dispatch: minor code refactor This hopefully makes things slightly easier to read. Differential Revision: https://phab.mercurial-scm.org/D3537

hgweb: drop unused argument 'tmpl' from webutil.compare()

hgweb: convert {comparison} to a mappinggenerator with named template No bare generator.

hgweb: rename 'context' argument of webutil.compare() to avoid name conflicts The meaning of 'context' depends on context. Here it is the number of the context lines in unified diff.

hgweb: convert comparison {lines} to a mappinggenerator with named template Bare generator can't be restarted.

hgweb: move getblock() closure out of compare() (Please use 'hg diff -w' to get readable diff from this patch.)

hgweb: move compline() closure out of compare()

hgweb: convert {diff} to a mappinggenerator with named template No more bare generator. Fortunately, this one is associated with a single template, so it can be a mappinggenerator.

hgweb: use template context to render {lines} of {diff} This is a preferred way to process nested templates.

hgweb: wrap {lines} of {diff} with mappedgenerator This can't be a mappinggenerator as it switches the templates conditionally.

hgweb: move prettyprintlines() closure out of diffs() This will be wrapped with mappedgenerator.

remotenames: enable the storage config option if extension is enabled Before this patch, the config option to store remotenames was set to False by default and remotenames extension does not set it True. So if you enable remotenames extension without setting 'experimental.remotenames=True', you won't get the remotenames. This patch makes remotenames enable the config option if then extension is enabled which makes sense. Differential Revision: https://phab.mercurial-scm.org/D3520

tests: clone the repo over ssh in tests/test-logexchange.t Cloning a local repo in tests, and normally too does not perform the actual clone, rather it copies the file contents. This makes remotenames information not being transferred. Let's clone over ssh so that we can test remotenames correctly. Differential Revision: https://phab.mercurial-scm.org/D3519

remotenames: check the remotepath with url containing user information too Current logic to match a remotepatch to a user defined path first removes the authentication information from the url and then tries to match it. However this is not true every time. If we clone a repo using ssh, the default path contains the user information too. Differential Revision: https://phab.mercurial-scm.org/D3518

shortest: avoid magic number "41" As suggested by Yuya. Feel free to fold into D3502. Differential Revision: https://phab.mercurial-scm.org/D3531

revlog: handle error from node lookup I missed this in D3499. Thanks to Yuya for spotting it. Differential Revision: https://phab.mercurial-scm.org/D3530

debugextensions: process extensions loaded from the local repository too

phabricator: drop support for the deprecated `phabricator.token` config

help: load module doc of disabled extension in extensions.disabledcmd() This helps rewriting _finddisablecmd() to not load the module.

extensions: extract closure that looks for commands from disabled module I'll rewrite this function to not load disabled extensions.

extensions: drop dead code trying to exclude deprecated disabled commands Here 'ext' is a tuple, which means ext.__doc__ describes about the Python tuple type. Fixing this would break 'hg help glog', which currently shows some nice explanation, so this patch just removes the dead code. $ hg help glog 'glog' is provided by the following extension: graphlog command to view revision graphs from a shell (DEPRECATED) (use 'hg help extensions' for information on enabling extensions)

util: drop remainder of dateutil/procutil aliases (API)

shortest: move revnum-disambiguation out of revlog I want to be able to change how we disambiguate and I think having revlog.shortest() worry only about finding a prefix that's unambiguous among nodeids makes more sense. This slows down `hg log -T '{shortest(node,1)}\n'` from 4.0s to 4.1s. Differential Revision: https://phab.mercurial-scm.org/D3502

shortest: remove unnecessary check for revnum in isvalid() We now always disambiguating against revnums after we've found an otherwise valid prefix, so we can safely remove the early isrev() check in isvalid(). Differential Revision: https://phab.mercurial-scm.org/D3501

shortest: make pure code also disambigute against revnums at end This makes the pure code more similar to the native code in that it first finds a prefix that's unambiguous among nodeids and then adds hex digits until it no longer looks like a revnum. It will allow us to even better separate the disambiguation with revnums in a later patch. With this patch `hg log -r 0::50 -T '{shortest(node,1)}'` with no native code goes from 25s to 43s. It wasn't exactly usable to begin with, so I don't feel too bad about it. Differential Revision: https://phab.mercurial-scm.org/D3500

revlog: use node tree (native code) for shortest() calculation I want to rewrite revlog.shortest() to disambiguate only among hex nodeids and then disambiguate the result with revnums at a higher level (in scmutil). However, that would slow down `hg log -T '{shortest(node,1)}\n'` from 5.0s to 6.8s, which I wasn't sure would be acceptable. So this patch makes revlog.shortest() use the node tree for finding the length of the shortest prefix that's unambiguous among nodeids. Once that has been found, it makes it longer until it is also not ambiguous with a revnum. This speeds up `hg log -T '{shortest(node,1)}\n'` from 5.0s to 4.0s. Differential Revision: https://phab.mercurial-scm.org/D3499

shelve: reduce scope of merge tool config override The config override seems to have a much greater scope than it needed to. I *think* it's only relevant in the while merging files. The rebase step also cares about the merge tool, but we seem to be explicitly passing it to rebase (around line 755). Differential Revision: https://phab.mercurial-scm.org/D3517

util: drop _deprecatedfunc() It was only needed for the previously removed forwarding.

util: drop deprecated forwarding to procutil (API)

tests: migrate demandimport.py away from deprecated `util` module symbols

templatekw: drop deprecated showlist() and showdict() (API)

util: drop deprecated forwarding to dateutil (API)

util: drop deprecated forwarding to stringutil (API)

scmutil: remove deprecated revpairnodes method (API)

merge: drop support for using updateresults as tuples (API)

cmdutil: drop deprecated log helper methods (API)

cmdutil: drop deprecated precursor of registrar.command (API)

cmdutil: remove deprecated _revertprefetch code (API)

tests: fix deprecation warning in test-url.py

hgweb: remove unused argument 'tmpl' from listfilediffs()

hgweb: use template context to render {files} of changelist entries This is a preferred way to process nested templates.

hgweb: wrap {files} of changelist entries with mappedgenerator This also switches the associated templates conditionally, which can't be a mappinggenerator.

hgweb: use template context to render {files} of changesetentry() This is a preferred way to process nested templates.

hgweb: wrap {files} of changesetentry() with mappedgenerator This can't be easily adapted to mappinggenerator because the associated templates are switched per item.

hgweb: extract generator of {files} from changesetentry() This will be wrapped with mappedgenerator.

templatefilters: document the json filter I don't know the exact reason why it was undocumented, but that would be because unicode is difficult for humans. Since the json filter can now get localstr back to UTF-8 as long as it is unprocessed (i.e. {desc|json} works, but {desc|firstline|json} doesn't), it seems good enough to advertise the json filter.

encoding: introduce tagging type for non-lossy non-ASCII string This fixes the weird behavior of toutf8b(), which would convert a local string back to UTF-8 *only if* it was lossy in the system encoding. Before b7b26e54e37a "encoding: avoid localstr when a string can be encoded losslessly (issue2763)", all local strings were wrapped by the localstr class. I think this would justify the round-trip behavior of toutf8b(). ASCII strings are special-cased, so the cost of wrapping with safelocalstr is negligible. (with mercurial repo) $ export HGRCPATH=/dev/null HGPLAIN= HGENCODING=latin-1 $ hg log --time --config experimental.evolution=all > /dev/null (original) time: real 11.340 secs (user 11.290+0.000 sys 0.060+0.000) time: real 11.390 secs (user 11.300+0.000 sys 0.080+0.000) time: real 11.430 secs (user 11.360+0.000 sys 0.070+0.000) (this patch) time: real 11.200 secs (user 11.100+0.000 sys 0.100+0.000) time: real 11.370 secs (user 11.300+0.000 sys 0.070+0.000) time: real 11.190 secs (user 11.130+0.000 sys 0.060+0.000)

encoding: fix toutf8b() to resurrect lossy characters even if "\xed" in it If 's' is a localstr, 's._utf8' must be returned to get the original UTF-8 sequence back. Because of this, it was totally wrong to test if '"\xed" not in s', which should be either '"\xed" not in s._utf8' or just omitted. This patch moves the localstr handling to top as the validity of 's._utf8' should be pre-checked by encoding.tolocal().

sshserver: redirect stdin/stdout early and use duplicated streams This is what we achieved with hook.redirect(True) plus ui.fout = ui.ferr. The hook.redirect() function can't be completely removed yet since hgweb still depends on it. I'm not sure if it is necessary for WSGI servers. CGI needs it, but does WSGI communicate over stdin/stdout channels?

sshserver: do setbinary() by caller (API) In most cases, stdio should be set to binary mode by the dispatcher, so the sshserver does not have to take care of that. The only exception was hg-ssh, which is fixed by this patch. .. api:: ``sshserver()`` no longer sets stdin and stdout to binary mode.

test-ssh: add some flush() to make output deterministic We shouldn't rely on buffering mode/state of file handles.

stringutil: flip the default of pprint() to bprefix=False If we use pprint() as a drop-in replacement for repr(), bprefix=False is more appropriate. Let's make it the default to remove bprefix=False noise.

stringutil: make pprint() forward uninteresting object to b'%r' We appear to start using pprint() as a replacement for repr(), so it's probably safer to support any Python objects instead of complaining about that.

tests: use stringutil.pprint instead of custom dumper in test-hgweb-auth.py Differential Revision: https://phab.mercurial-scm.org/D3529

tests: port test-simplekeyvaluefile.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3528

tests: port test-extensions-wrapfunction to Python 3 Differential Revision: https://phab.mercurial-scm.org/D3527

tests: port test-ui-config to Python 3 Slight test output changes to make life easier. Differential Revision: https://phab.mercurial-scm.org/D3526

config: fix py3 backslash escaping bug in parser caught by tests Differential Revision: https://phab.mercurial-scm.org/D3525

tests: port test-symlink-os-yes-fs-no.py to Python 3 # skip-blame just bytes/str changes Differential Revision: https://phab.mercurial-scm.org/D3524

posix: use inst.errno instead of inst[0] on OSError instances Differential Revision: https://phab.mercurial-scm.org/D3523

tests: port test-filelog.py to Python 3 # skip-blame just some bytes prefixes Differential Revision: https://phab.mercurial-scm.org/D3522

stringutil: teach pprint about tuples Differential Revision: https://phab.mercurial-scm.org/D3521

templatefuncs: show hint if extdata source is evaluated to empty (issue5843)

revlog: use literal -1 instead of variable that always has that value We were setting "ntrev" to "rev", but "rev" was always -1 at the end of the loop, so it's clearer to use a literal -1. Differential Revision: https://phab.mercurial-scm.org/D3498

revlog: extract function for fully populating the radix tree This code is currently used for partialmatch(), but I want to reuse it when I implement shortest() in native code. Differential Revision: https://phab.mercurial-scm.org/D3497

tests: port test-lfs-pointer.py to Python 3 The weird bit here is having to import /something/ from Mercurial before we touch hgext.* so that the module loader gets initialized. That's probably a bug we should explore at some point? Differential Revision: https://phab.mercurial-scm.org/D3514

tests: port test-ui-verbosity.py to Python 3 # skip-blame just some bytes/string wrangling Differential Revision: https://phab.mercurial-scm.org/D3515

lfs: stabilize error message values for Python 2 and 3 Differential Revision: https://phab.mercurial-scm.org/D3513

tests: port test-ui-color.py to Python 3 As in some other tests, I like confirming that we're dealing with bytes here, so the b prefix is now in the expected output on both 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D3512

tests: port test-dispatch.py to Python 3 Differential Revision: https://phab.mercurial-scm.org/D3511

tests: port test-wireproto.py to Python 3 A little bit of output churn because we now get b'' prefixes on output in both 2 and 3, but for this test I'm more comfortable knowing that we're using bytestrs everywhere. Differential Revision: https://phab.mercurial-scm.org/D3510

tests: port test-context.py to Python 3 This is a weird "minimally invasive" port. I think there's room to do better here, but I'm also not sure how often we'll touch this code... Differential Revision: https://phab.mercurial-scm.org/D3509

scmutil: fix __repr__ of status tuple We should probably start giving some thought to migrating the status tuple to attrs, but not now. Differential Revision: https://phab.mercurial-scm.org/D3508

update: print warning about hidden changeset after update When an attempt to update to a hidden changeset fails because the working copy is dirty, you may get a message like this: updating to a hidden changeset 343f6de32686 (hidden revision '343f6de32686' was rewritten as: 4ab941244072) abort: conflicting changes (commit or update --clean to discard changes) It's easy to miss the real error here. This patch moves the warning about the hidden changeset to after the update has happened. It changes the verb tense accordingly (and drops the "a" that I think it sounds better without). Of course, this means that the commit isn't actually hidden anymore when the message is printed. I think that's fine. Differential Revision: https://phab.mercurial-scm.org/D3479

py3: whitelist three passing cvs convert tests Differential Revision: https://phab.mercurial-scm.org/D3490

cvsps: convert encoding name to sysstr Differential Revision: https://phab.mercurial-scm.org/D3489

tweakdefaults: remove "bisect" from commands.status.skipstates If you forget to clear bisect state after you're done bisecting (as I've done twice in the last week or so), the next time you run `hg bisect --good/--bad`, it's going to tell you "The first bad revision is:" etc. It's probably not obvious to new users what's going on, and having the verbose output about an ongoing bisection will probably help them, so let's turn it back on by default. Differential Revision: https://phab.mercurial-scm.org/D3516

hgweb: wrap {instabilities} by hybridlist() This is a list of simple values, which can be a hybrid list.

hgweb: wrap {whyunstable} with mappinggenerator This is also a generator of mappings, which needs a wrapper.

hgweb: wrap {succsandmarkers} with mappinggenerator This is also a generator of mappings, which needs a wrapper.

hgweb: wrap {branches} and {entries} of branches with mappinggenerator Bare generator of mappings shouldn't be put in a template mapping because its type can't be determined without consuming it.

hgweb: drop tmpl argument from webutil.showtag() and showbookmark() It's replaced by a context argument passed to a mappinggenerator.

hgweb: wrap {changelogtag}, {changesettag}, and {changesetbookmark} These can't be hybrid lists as they've associated with named template, 't1'.

hgweb: drop useless **args from webutil.showtag() and showbookmark() Callers never pass excessive arguments to these functions.

hgweb: reuse graph node-related functions from templates The difference between templatekw.getgraphnode() and webutil.getgraphnode() is that the latter is not limited to 1 character.

templates: split getgraphnode() body into two functions getgraphnodecurrent() is checking if the node is currently checked out and getgraphnodesymbol() is checking properties that have more to do with the stored data and the DAG.

hgweb: wrap {branch} and {changesetbranch} by hybridlist() This is also a 0/1-length list of a simple value, can be a hybrid list. Appears that we have many {branch} variants.

hgweb: wrap {inbranch} by hybridlist() This is also a 0/1-length list of a simple value, can be a hybrid list.

hgweb: wrap {branches} by hybridlist() This is a 0/1-length list of a simple value, can be a hybrid list.

hgweb: wrap {bookmarks} by hybridlist() This is also a list of simple values.

hgweb: wrap {tags} by hybridlist() This one is a list of simple values, which can be a hybrid list.

hgweb: wrap {rename} with mappinglist No bare list of mappings should be put in a template mapping.

tests: fix test-status-inprocess.py on Python 3 The same print() hack as test-filecache.py. Differential Revision: https://phab.mercurial-scm.org/D3507

tests: port test-filecache.py to Python 3 Only remarkable bit is my wrapper around print(), which I regret a little, but not enough to go back and try to do something cleaner. Differential Revision: https://phab.mercurial-scm.org/D3506

tests: port test-cbor.py to Python 3 I suspect this b''.join() was a remnant of an earlier iteration of this code, as it was building a string from a string. Differential Revision: https://phab.mercurial-scm.org/D3505

cborutil: port to Python 3 The only problem lurking in here was sorts of mismatched types. The sorts are only for output stability in our tests (sigh), so we just build a phony sort key using the __name__ of types so that we only compare like types against each other. By pure luck, my awful sort key matches the behavior we get "for free" in Python 2, so no test output changes. Differential Revision: https://phab.mercurial-scm.org/D3504

lock: pass sysstr to warnings module This makes test-lock.py slightly less broken, but it's still pretty far from passing. Differential Revision: https://phab.mercurial-scm.org/D3503

tests: port test-revlog-raw.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3496

tests: migrate test-wsgirequest.py to Python 3 # skip-blame adding and removing b prefixes Differential Revision: https://phab.mercurial-scm.org/D3495

tests: port test-parseindex2.py to Python 3 # skip-blame just b prefixes and int() instead of long() Differential Revision: https://phab.mercurial-scm.org/D3494

tests: prefer string concatenation with () instead of \ in parseindex2 tests Differential Revision: https://phab.mercurial-scm.org/D3493

tests: port test-simplemerge.py to Python 3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3492

tests: replace use of assertEquals with assertEqual in test-simplemerge.py The former is deprecated. No functionality change. # skip-blame just removing an ess and some whitespace Differential Revision: https://phab.mercurial-scm.org/D3491

tests: b prefixes on some inline python in test-convert-cvs.t # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3488

cvsps: wrap bytes in bytestr before %r-ing it Differential Revision: https://phab.mercurial-scm.org/D3487

convcmd: make a copy of heads before mutating it Differential Revision: https://phab.mercurial-scm.org/D3486

cvsps: portably convert int to bytes Differential Revision: https://phab.mercurial-scm.org/D3485

cvsps: wrap cmp methods (deprecated) in functools.cmp_to_key Differential Revision: https://phab.mercurial-scm.org/D3484

tests: port inline extension in test-convert-cvs.t to py3 # skip-blame two b prefixes, nothing more Differential Revision: https://phab.mercurial-scm.org/D3483

cvsps: add b prefixes to regular expressions # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3482

minirst: fix bytes slicing defect on Python 3 This is the only lingering problem in minirst on Python 3. test-minirst.py now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3478

tests: port test-minirst.py to Python 3 There are a few failures here, mostly around formatting lists. I'll fix that in a subsequent change. Differential Revision: https://phab.mercurial-scm.org/D3477

tests: use stringutil.pprint instead of pprint.pprint in test-minirst.py Stabilizes some output on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3476

tests: port test-pathencode.py to Python 3 # skip-blame uninteresting changes, mainly b prefixes Differential Revision: https://phab.mercurial-scm.org/D3475

tests: port test-hgwebdir-paths.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3474

tests: port test-walkrepo.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3473

tests: port test-minifileset.py to Python 3 # skip-blame all b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3472

minifileset: fix on Python 3 Found by porting test-minifileset.py to Python 3. Differential Revision: https://phab.mercurial-scm.org/D3471

tests: port test-url.py to Python 3 # skip-blame it's all b prefixes and pycompat.bytestr() wrappers Differential Revision: https://phab.mercurial-scm.org/D3470

tests: fix error case in test-url.py's doctest This required some careful attention, so I wanted to split it out from the uninteresting bits that'll be in the next change. Differential Revision: https://phab.mercurial-scm.org/D3469

util: make util.url __repr__ consistent on Python 2 and 3 Differential Revision: https://phab.mercurial-scm.org/D3468

sslutil: fix some edge cases in Python 3 support Detected by fixing up test-url.py on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3467

tests: port test-hybridencode.py to unittest This was done predominantly through keyboard macros in emacs, so it's not always pretty, but it's enough to make the test easier to work with. There's a ton of room for improvement in this file, but it would be labor intensive and error-prone, so I went with the dumbest option that could work, so as to avoid transcription errors. Paranoia: $ egrep '^ def test' tests/test-hybridencode.py | wc -l 44 $ egrep '^ def test' tests/test-hybridencode.py | sort | uniq | wc -l so I'm pretty confident there aren't any shadowed test methods. This fixes the test on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3466

tests: make test-check-interfaces.py work on Python 3 # skip-blame just a bunch of bytes prefixes Differential Revision: https://phab.mercurial-scm.org/D3465

scmutil: clean up bytes/string cache decorator mess on Python 3 again The previous fix to this area worked, but was dropping bytes in __dict__ on Python 3. This was causing subtle breakage in test-check-interfaces.py, and probably other things too. Fixed now. Differential Revision: https://phab.mercurial-scm.org/D3464

directaccess: use resolvehexnodeidprefix() instead of _partialmatch() Same reasoning as previous commit: I want to make resolvehexnodeidprefix() move complex and don't want to duplicate that code in directaccess. Differential Revision: https://phab.mercurial-scm.org/D3463

revset: use resolvehexnodeidprefix() in id() predicate (BC) We now have a public method for this purpose, so we don't need to access the private revlog._partialmatch(). Also, I'll probably make some changes to resolvehexnodeidprefix() later, and I want those to be reflected by the id() predicate. Note that this breaks a test case, because we now resolve the prefix in the unfiltered repo and get an ambiguous lookup, which results in no revision being added to the revset. The test case was already documented to be broken even though it wasn't. It's important to note that {shortest(node)} already uses the unfiltered repo, so we're not going to break people who get the prefix from there. I think we may not want to ever use shortest() in the filtered repo. It seems unlikely to be enough of a win to matter much. For example, in my hg repo, it would save me only 0.2 hex digits. In another repo that only I modify, it saves a little more, but it's still only 0.29 hex digits. It seems unlikely that people will prune enough commits that only 1/16 of the commits are visible (which is what it would take a to save a single hex digit). Instead, I'm working on another approach: allow ambiguous matches to be disambiguated within a user-specified revset. Whether or not that pans out, I hope we're okay with this little change in behavior for now and we can decide what to do about it later. Differential Revision: https://phab.mercurial-scm.org/D3311

revset: make id() an empty set for ambiguous nodeid (BC) As Yuya pointed out in the review of D3311, id() (and rev()) does not raise an error when the input is an unknown identifier, so it doesn't make sense for it to do that when the input is ambiguous with a filtered node. However, it turned out that it already does raise an error when the input is ambiguous among the visible nodes. So let's start by fixing that. Differential Revision: https://phab.mercurial-scm.org/D3462

shortest: don't keep checking for longer prefix if node doesn't exist (API) If revlog.shortest() is called with an invalid nodeid, we keep checking if longer and longer prefixes are valid. We call revlog._partialmatch() for each prefix. That function will give us None if the node doesn't exist (and a RevlogError if it's ambiguous), so there's no need to keep checking. This patch instead makes revlog.shortest() raise a LookupError is the node does not exist, and updates the caller to handle it. Before this patch, revlog.shortest() would return the full hexnode for nonexistent nodeids. By the same reasoning as in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), it's not revlog.shortest() that should decide how to present nonexistent nodeids, so that's now moved to the template function. This should speed up cases where {shortest()} is applied to an invalid nodeid, but I couldn't think of a reasonable case where that would happen. Differential Revision: https://phab.mercurial-scm.org/D3461

shortest: extract function for checking if a prefix is a revnum Much of isvalid() was about testing if a prefix is a valid revnum. I want to reuse that soon, so let's move it out. There is no significant slowdown from the function call overhead. Differential Revision: https://phab.mercurial-scm.org/D3460

shortest: rename "test" variable to "prefix" Sorry if this is considered churn, but "prefix" just seems much clearer to me. Differential Revision: https://phab.mercurial-scm.org/D3459

shortest: move some safe code out of exception block The RevlogError and WdirUnsupported could be raised by _partialmatch(), but not by the rest of isvalid(), so let's move the rest out to make it clearer. Differential Revision: https://phab.mercurial-scm.org/D3458

revlog: don't say "not found" on internal error If index_node() returned NULL, then index_find_node() and and nt_partialmatch() used to return -2 to signal that the node was not found. However, we were passing in a revnum to index_node() that we knew should exist, so the only reason it could return NULL was due to some internal error or perhaps out of memory. Let's not use "not found" for these cases. I suppose we never noticed this because these error never happen in practice. I think there are more places where we should error out instead of reporting that the node was not found, but the cases mentioned above were all I cared about right now (because using the same error code for all failures simplified some future patches). Differential Revision: https://phab.mercurial-scm.org/D3457

revlog: extract function for getting node from known-to-exist rev Many of the calls to index_node() (which converts a rev to a nodeid) are done with a rev that's know to exist. If the function fails, there's something really wrong and we should just abort. This was done in only one place. This patch starts by extracting that code to a function that we can reuse in later patches. Differential Revision: https://phab.mercurial-scm.org/D3456

shortest: make {shortest("fffffffff")} work again {shortest("fffffffff")} should shorten it to the shortest unambiguous prefix for the working directory. It used to do that until I broke it in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), when we started returning the full hex nodeid for any working directory prefix shorter than 40 hex digits. This patch fixes it by catching WdirUnsupported specifically. Differential Revision: https://phab.mercurial-scm.org/D3455

revlog: use radix tree also for matching keys shorter than 4 hex digits I don't know what the reason for the 4-digit limit was, and I can't think of any real disadvantages of using the radix tree also when the requested minimum length is short. This speeds up `hg log -T '{shortest(node,1)}\n'` from 2m16s to 4.5s by making that not fall back to pure code. Differential Revision: https://phab.mercurial-scm.org/D3453

context: convert to hex for error message only for 20-byte changeid Now that 20-byte strings unambiguously mean binary (or a bug), we can specialize the conversion to hex for that case. Differential Revision: https://phab.mercurial-scm.org/D3452

context: clarify that only one attempt is made to interpret changeid We can now tell what type of revision specifier we have just by looking at it (we no longer attempt to interpret it in one way after the other -- that's now in scmutil.revsymbol()). Let's clarify this in the code by swithing to if/elif. Differential Revision: https://phab.mercurial-scm.org/D3451

context: only bother looking for broken dirstate for 20-byte changeid If we fail to look up a changeid in changectx.__init__, we check if it exactly matches any of the dirstate parents, and if it does, we print a more specific message ("working directory has unknown parent '...'!" instead of "unknown revision '...'"). The dirstate parents are always 20 bytes, so there's no need to check for a match when the given changeid is not 20 bytes. (And now that all the other allowed forms of changeid have been moved out of the constructor, there's no risk that a changeid that did match a dirstate parent was actually a valid bookmark.) Differential Revision: https://phab.mercurial-scm.org/D3450

context: drop support for looking up context by ambiguous changeid (API) This removes support for using the changectx constructor (and thereby repo[x]) for looking up contexts by a stringified int, a namespace key (e.g. a bookmark), or a partial hex nodeid. This means that e.g. repo[<hex nodeid>] will now fail even if a bookmark with the same name exists (which is a good thing IMO). It also means that doing repo[<non-existent node>] no longer ends up loading namespaces (which was a surprising side-effect of creating of failing to create a context object that I recently ran into while debugging something unrelated to this series). Differential Revision: https://phab.mercurial-scm.org/D3449

tests: pass parent revision as integer to repo[x] in test-context-metadata.t Support for looking up by stringified int is going away. It's already deprecated, but I think I didn't notice this case because the test case pipes through `grep`. Differential Revision: https://phab.mercurial-scm.org/D3448

bookmarks: cache reverse mapping (issue5868) I chose a simpler implementation. If the initial cost of building reverse mapping is significant, we'll have to move it under @propertycache. The nodemap could be a dict of sets, but I think keeping a sorted list is better since each node is likely to have zero/one bookmark. Micro-benchmark with 1001 bookmarks and 1001 revisions: $ for n in `seq 0 1000`; do touch $n; hg book book$n; hg ci -qAm$n; done $ hg bookmarks --time > /dev/null (orig) time: real 0.040 secs (user 0.050+0.000 sys 0.000+0.000) (new) time: real 0.040 secs (user 0.040+0.000 sys 0.010+0.000) $ hg log -T '{bookmarks}\n' --time > /dev/null (orig) time: real 0.160 secs (user 0.160+0.000 sys 0.000+0.000) (new) time: real 0.090 secs (user 0.100+0.000 sys 0.000+0.000)

bookmarks: make argument names of _set/_del() more specific

bookmarks: extract function that looks up bookmark names by node

bookmarks: hide dict behind bmstore class This should make it clearer that the bmstore doesn't expose all dict APIs.

tests: remove pid file by default Previously, killdaemons.py would kill PIDs listed in a file then leave the file lingering around. If the PIDs are killed, then there's no point leaving the PID file around. In the worst case, a later invocation of killdaemons.py (run-tests.py invokes killdaemons.py after running a test) could kill a separate process whose PID conflicted with a previously-killed process. By removing the PID file, we eliminate this possibility. Some tests were manually removing the PID file after calling killdaemons.py. So we update these tests to not do this. Differential Revision: https://phab.mercurial-scm.org/D3443

tests: remove #require killdaemons The killdaemons hghave feature has returned True since it was introduced in 448d0c452140. As such, "#require killdaemons" has no effect and is superfluous. So we remove instances of it. Differential Revision: https://phab.mercurial-scm.org/D3442

pycompat: export queue module instead of symbols in module (API) Previously, pycompat and util re-exported individual symbols from the queue module. This had the side-effect of forcing the loading of the queue module whenever pycompat/util was imported. These symbols aren't used very often. So importing the module to get a handle on the symbols is wasteful. This commit changes pycompat so it no longer exports the individual symbols in the queue module. Instead, we make the imported module a "public" symbol. We drop the individual symbol aliases from the util module. All consumers are updated to use pycompat.queue.* instead. This change makes 300 invocations of `hg log -r. -T '{rev}\n'` a little faster: before: 18.44s after: 17.87s Differential Revision: https://phab.mercurial-scm.org/D3441

demandimport: make module ignores a set (API) The list of modules to ignore is used for membership testing. Yet it is defined as a list. Sets are more efficient for membership testing. So this commit converts the module list to a set. Since we took an API hit, I renamed the variable to further clarify the change. This appears to reduce the CPU time for running 300 invocations of `hg log -r. -T '{rev}'` on my i7-6700K: before: 18.64s after: 18.44s Differential Revision: https://phab.mercurial-scm.org/D3440

cmdutil: remove unused 'confirmopts' constant Follows up f10cb49951e1.

formatter: ditch namedtuple in favor of attr

log: consume --stat/patch options at constructor of changesetprinter The variable name, self.diffopts, was confusing. Let's split it to two booleans.

log: cache diffopts instance It appears that calling patch.diff*opts() repeatedly has some cost. $ hg log -T '{rev}\n' -R mercurial --time > /dev/null (orig) time: real 4.430 secs (user 4.370+0.000 sys 0.050+0.000) (new) time: real 1.950 secs (user 1.880+0.000 sys 0.060+0.000) 'diffopts or {}' isn't necessary as patch.diff*opts() accepts opts=None.

merge with stable

encoding: alias cp65001 to utf-8 on Windows As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know how different it is from the UTF-8, but Python 3 appears to have introduced new codec for cp65001, so the alias is enabled only for Python 2. https://bugs.python.org/issue13216 This patch is untested, but hopefully fixes the following issue. https://bitbucket.org/tortoisehg/thg/issues/5127/

windows: fix incorrect detection of broken pipe when writing to pager Paging e.g. hg incoming on Windows and quitting the pager before the output is consumed will print 'abort: Invalid argument'. This is because the windows error 0xE8 (ERROR_NO_DATA) is mapped to EINVAL even though it is documented as 'The pipe is being closed'. Note that this fix assumes that Windows' last error code is still valid in the exception handler. It works correctly in all my tests. A simpler fix would be to just map EINVAL to EPIPE, like was done is flush previously, but that would be less precise. This error was not observed previously, when pager was an extension.

Added signature for changeset 0b63a6743010

Added tag 4.6.2 for changeset 0b63a6743010

procutil: use unbuffered stdout on Windows Windows doesn't support line buffering, treating it as fully buffered. This causes output of slow commands to stutter. We use unbuffered instead.

merge: do not fill manifest of committed revision with pseudo node (issue5526) Since a75d24539aba "convert: fix convert dropping p2 contents during filemap merge", wctx is not always a committablectx because the convert extension passes in repo[n] as wctx. If wctx is a committed changeset, its manifest dict shouldn't be mutated reflecting to the working directory.

manifest: fix possible SEGV caused by uninitialized lazymanifest fields Before, uninitialized self->pydata would be passed to lazymanifest_dealloc() on OOM, and Py_DECREF(self->pydata) would crash if we were unlucky. It's still wrong to do malloc() thingy in tp_init because __init__() may be called more than once [1], but I don't want to go a step further in stable branch. [1]: https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new "The tp_new function should ... do only as much further initialization as is absolutely necessary. Initialization that can safely be ignored or repeated should be placed in the tp_init handler."

tests: replace `echo -n` with `printf` per check-code Differential Revision: https://phab.mercurial-scm.org/D3749

crecord: fix line number in hunk header (issue5917) `@@ -1,1 +-1,0 @@` is not a valid patch hunk header. Change it to `@@ -1,1 +0,0 @@`. Differential Revision: https://phab.mercurial-scm.org/D3737

lazymanifest: don't crash when out of memory (issue5916) self->lines can be NULL if we failed to allocate memory for it.

cext: stop worrying and love the free(NULL) There is no need to check for a NULL pointer before calling free since free(NULL) is defined by C standards as a no-op. Lots of software relies on this behavior so it is completely safe to call even on the most obscure of systems.

tests: fix test-patch.t on pickier /bin/sh implementations This is a graft of 0b39edeff033 and f44306940c94 from default because I'm tired of seeing the FreeBSD build be red on stable. See those revisions for details on what's going on here.

chg: fix an undefined behavior about memcpy Spot by Wez Furlong. `memcpy(x, NULL, 0)` is undefined according to [1]. [1]: https://stackoverflow.com/questions/5243012 Differential Revision: https://phab.mercurial-scm.org/D3698

Added signature for changeset 9c5ced5276d6

Added tag 4.6.1 for changeset 9c5ced5276d6

mpatch: avoid integer overflow in combine() (SEC) All the callers of this function can handle a NULL return, so that appears to be the "safe" way to report an error.

mpatch: avoid integer overflow in mpatch_decode (SEC)

mpatch: fix UB integer overflows in discard() (SEC)

mpatch: fix UB in int overflows in gather() (SEC)

mpatch: introduce a safesub() helper as well Same reason as safeadd().

mpatch: introduce a safeadd() helper to work around UB int overflow We're about to make extensive use of this. This change duplicates some stdbool.h portability hacks from cext/util.h. We should probably clean that up in the future, but we'll skip that for now in order to make security backports easier.

mpatch: ensure fragment start isn't past the end of orig (SEC) Caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0004. A CVE has not been obtained as of this writing.

mpatch: protect against underflow in mpatch_apply (SEC) Also caught by oss-fuzz fuzzer during development. This defect is OVE-20180430-0002. A CVE has not been obtained as of this writing.

mpatch: be more careful about parsing binary patch data (SEC) It appears to have been possible to trivially walk off the end of an allocated region with a malformed patch. Oops. Caught when writing an mpatch fuzzer for oss-fuzz. This defect is OVE-20180430-0001. A CVE has not been obtained as of this writing.

zstandard: pull in bug fixes from upstream 0.9.1 release (issue5884) This changeset contains the meaningful code changes from python-zstandard's 0.9.1 release. The main fix is to restore support for compiling with mingw.

templatefuncs: only render text portion of minirst.format() result When "keep" argument is provided, the function returns (text, pruned), where pruned is a list of sections from the original plain text that were pruned from the rendered result. Let's not output it together with the rendered HTML.

tests: demonstrate that hgweb renders "pruned" that minirst.format() returns Notice at the bottom of the help text there's "windows". It's a section that is in the original help text, but was pruned (because hgweb didn't ask for it).

rebase: avoid RevlogError when computing obsoletenotrebased (issue5907) The key to reproducing this seems to be missing an obsolete node that is not an ancestor of the destination.

rebase: prioritize indicating an interrupted rebase over update (issue5838) This should also cover the transplant extension, and any other non clearable states.

tests: demonstrate inconsistent messaging around interrupted rebases

tests: adapt test-check-pylint to run on Windows The line endings are explicitly converted because this was ending up with '\r (no-eol) (esc)' lines, in addition to the usual '\r (esc)' lines. I've seen the fakerc output on other recently installed systems though (10.13 and/or Fedora 26). Unfortunately, the output here uses '\\' on Windows, so glob away the whole path.

hghave: avoid a deadlock reading the child process's output The output of `pylint` is voluminous enough that it fills the buffer on Windows, and waited for the parent to read it out. But the parent was waiting on the child to exit. I'm not sure what the intent of setting `ret = -1` in the exception handler just above this was...

lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902) There are only a handful of methods that access repo attributes that are applied in reposetup(). The `diff` test covers all of the commands that call scmutil.prefetchfiles(). Along the way, I saw that adding files and upgrading the repo format were also problems (also tested here). I don't think running `hg serve` through the commandserver is sane, but I conditionalized both the capabilities and the wsgirequest handler because it's trivially correct. It doesn't look like there has ever been a caller of candownload(), so there's no test for that path. The upload case isn't testable, because uploadblobs() bails if there are no pointers. The requirement should be added any time pointers are introduced, and that would force the extension to be loaded specifically for the repo. This covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is promoted to LFS), and uploadblobsfromrevs(), which can be called by other extensions. I think readfromstore() and writetostore() are only reachable as a flag processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as that is seen, so I don't think those are a problem.

help: correct signature of separate() template function Without the dots, it looked as if separate() would take a list of arguments.

hgweb: do not try to replace signal handlers while locking According to the issue 5889, mod_wsgi issues a warning on signal.signal() call, and we wouldn't want to see it in error log. The problem addressed by d77c3b023393 could potentially occur in web session, but that would be less likely than in user processes.

lock: add internal config to not replace signal handlers while locking signal.signal() is blocked in some WSGI environments, and a horrible warning is sent to the server log. So we need a way to disable it, and I think abusing ui.config is the simplest workaround.

merge with i18n

i18n-pt_BR: synchronized with 32a75a8a5b0f

i18n-ja: fix block indentation

i18n-ja: synchronized with 32a75a8a5b0f

httppeer: declare 'dbg' at the function level As we just saw in the previous changeset, having the variable defined into a branch creates bug. This is a cheap to move it at the function level.

httppeer: properly gate debug usage behind debug flag check The "dbg" local variable is only defined if the 'debugflag' is set to True. However, it was used indiscriminately later in the function. We hide its usage behind the 'debugflag' value to avoid raising a NameError.

push: continue without locking on lock failure other than EEXIST (issue5882) This code was added by 3f5e75c22585 "push: make locking of source optional (issue3684)", but EACCES isn't the only error that could be triggered by filesystem permission. I think catching LockUnavailable is more appropriate than testing errno value by caller.

bdiff: fix yet more fallout from xdiff long/int64 conversion (issue5885) "l" in Py_BuildValue's format string means long, so passing int64_t instead results in fireworks on 32bit architectures. Differential Revision: https://phab.mercurial-scm.org/D3538

revset: pass in lookup function to matchany() (issue5879) Silly mistake in f83cb91b052e.

test-hgweb: add test for foo-bar name lookup This is broken since f83cb91b052e "revset: pass in lookup function instead of repo (API)."

bundle2: mark the bundle2 part as advisory (issue5872) It blocks old clients to read bundle including this part. Differential Revision: https://phab.mercurial-scm.org/D3481

debugbundle: also display if a part is mandatory or advisory Most parts are mandatory but when introducing new parts, they should be advisory if included by default or old clients won't be able to process it. Differential Revision: https://phab.mercurial-scm.org/D3480

Added signature for changeset 6614cac550ae

Added tag 4.6 for changeset 6614cac550ae

filelog: don't crash on invalid copy metadata (issue5748) "copy" and "copyrev" are both supposed to appear next to each other. However, a user report demonstrated a crash that indicates that something in the wild is producing "copy" without "copyrev" (probably `hg convert`). While we should definitely fix the source of the bad metadata, the bad code causing the crash is already in the wild and who knows how many repositories are impacted. So let's be more defensive when accessing the file revision metadata.

httppeer: detect redirect to URL without query string (issue5860) 197d10e157ce subtly changed the HTTP peer's handling of HTTP redirects. Before that changeset, we instantiated an HTTP peer instance and performed the capabilities lookup with that instance. The old code had the following relevant properties: 1) The HTTP request layer would automatically follow HTTP redirects. 2) An encountered HTTP redirect would update a peer instance variable pointing to the repo URL. 3) The peer would automagically perform a "capabilities" command request if a caller requested capabilities but capabilities were not yet defined. The first HTTP request issued by a peer is for ?cmd=capabilities. If the server responds with an HTTP redirect to a ?cmd=capabilities URL, the HTTP request layer automatically followed it, retrieved a valid capabilities response, and the peer's base URL was updated automatically so subsequent requests used the proper URL. In other words, things "just worked." In the case where the server redirected to a URL without the ?cmd=capabilities query string, the HTTP request layer would follow the redirect and likely encounter HTML. The peer's base URL would be updated and the unexpected Content-Type would raise a RepoError. We would catch RepoError and immediately call between() (testing the case for pre 0.9.1 servers not supporting the "capabilities" command). e.g. try: inst._fetchcaps() except error.RepoError: inst.between([(nullid, nullid)]) between() would eventually call into _callstream(). And _callstream() made a call to self.capable('httpheader'). capable() would call self.capabilities(), which would see that no capabilities were set (because HTML was returned for that request) and call the "capabilities" command to fetch capabilities. Because the base URL had been updated from the redirect, this 2nd "capabilities" command would succeed and the client would immediately call "between," which would also succeed. The legacy handshake succeeded. Only because "capabilities" was successfully executed as a side effect did the peer recognize that it was talking to a modern server. In other words, this all appeared to work accidentally. After 197d10e157ce, we stopped calling the "capabilities" command on the peer instance. Instead, we made the request via a low-level opener, detected the redirect as part of response handling code, and passed the redirected URL into the constructed peer instance. For cases where the redirected URL included the query string, this "just worked." But for cases where the redirected URL stripped the query string, we threw RepoError and because we removed the "between" handshake fallback, we fell through to the "is a static HTTP repo" check and performed an HTTP request for .hg/requires. While 197d10e157ce was marked as backwards incompatible, the only intended backwards incompatible behavior was not performing the "between" fallback. It was not realized that the "between" command had the side-effect of recovering from an errant redirect that dropped the query string. This commit restores the previous behavior and allows clients to handle a redirect that drops the query string. In the case where the request is redirected and the query string is dropped, we raise a special case of RepoError. We then catch this special exception in the handshake code and perform another "capabilities" request against the redirected URL. If that works, all is well. Otherwise, we fall back to the "is a static HTTP repo" check. The new code is arguably better than before 197d10e157ce, as it is explicit about the expected behavior and we avoid performing a "between" request, saving a server round trip. Differential Revision: https://phab.mercurial-scm.org/D3433

hgweb: prevent triggering dummy href="#" handler Follow up for the previous patch.

paper: add href="#" to links with click handlers This restores the styling that was accidentally removed by the previous change to these files. Differential Revision: https://phab.mercurial-scm.org/D3438

paper: don't register click handlers with inline javascript (issue5812) The use of inline href="javascript:" undermines CSP policies that don't allow inline javascript. This commit changes the registering of the diffstat and line wrapping toggle handlers to the the global DOMContentLoaded handler, thus eliminating all inline javascript from the paper template. Differential Revision: https://phab.mercurial-scm.org/D3437

hgweb: allow Content-Security-Policy header on 304 responses (issue5844) A side-effect of 98baf8dea553 was that the Content-Security-Policy header was set on all HTTP responses by default. This header wasn't in our list of allowed headers for HTTP 304 responses. This would trigger a ProgrammingError when a 304 response was issued via hgwebdir. This commit adds Content-Security-Policy to the allow list of headers for 304 responses so we no longer encounter the error. Differential Revision: https://phab.mercurial-scm.org/D3436

hgweb: discard Content-Type header for 304 responses (issue5844) A side-effect of 98baf8dea553 was that hgwebdir always sets a global default for the Content-Type header. HTTP 304 responses don't allow the Content-Type header. So a side-effect of this change was that HTTP 304 responses served via hgwebdir resulted in a ProgrammingError being raised. This commit teaches our 304 response issuing code to drop the Content-Type header. Differential Revision: https://phab.mercurial-scm.org/D3435

tests: add tests demonstrating ISE for HTTP 304 responses with hgwebdir There are two separate failures here. One for the Content-Type header. Another for the Content-Security-Policy header. Differential Revision: https://phab.mercurial-scm.org/D3434

hgweb: guard against empty Content-Length header Discussion in issue 5860 seems to indicate this can occur. Differential Revision: https://phab.mercurial-scm.org/D3432

test-push-http: do not clear pid file It's okay now, but we'll end up leaking daemon processes if we add some more.

debugcolor: fix crash by empty styles (issue5856)

tests: explicitly define compression engines for tests The zstd compression engine requires C extensions and isn't present in pure Python builds. The compression engine list leaks into the server capabilities string. Unless we're testing functionality specific to a compression format, the set of compression formats supported by a server doesn't matter much. So this commit explicitly defines the server's compression engines for some tests so behavior is consistent between pure and non-pure builds. Differential Revision: https://phab.mercurial-scm.org/D3431

tests: update no-zstd branch of test-treediscovery.t as in 330ada7e8ea5 This side of the test got overlooked. We should probably consider having a way to run some of our tests through a "no-zstd" case just like we run some things through a "no-obsmarkers" case, but that's not an appropriate thing for stable. Differential Revision: https://phab.mercurial-scm.org/D3430

tests: glob away content-length changes relating to missing zstd bindings This doesn't fix everything in these two tests around missing zstd: we still get some changes in the CBOR payload in ways that I think we probably shouldn't bother to glob around. Maybe we should just disable zstd support in some of these lower-level wireproto tests? Differential Revision: https://phab.mercurial-scm.org/D3429

revlog: make pure version of _partialmatch() support 40-byte hex nodeids Without this patch, test-histedit-arguments.t would fail when run with --pure. It turned out to be because the pure version of _partialmatch() does not support full 40-byte hex nodeids. When histedit's instructions include things like "pick tip", it resolves the "tip" revision early to a full nodeid (but plain hex nodeid prefixes are not resolved to full nodeids). Then the nodeid (full or not) is looked up using to a full nodeid later. This step is what fails in pure mode. It has been failing since my c4131138eadb (histedit: look up partial nodeid as partial nodeid, 2018-04-06). I haven't verified, but I suspect histedit instructions like "pick <full hex nodeid>" would have been failing before my commit too, though. The fix is trivial: change a "< 40" to "<= 40". Differential Revision: https://phab.mercurial-scm.org/D3428

hgweb: reuse body file object when hgwebdir calls hgweb (issue5851) An unintended side-effect of f0a851542a05 was that the request body file object (which uses a util.cappedreader) was constructed twice when hgwebdir called into hgweb. Since we attempt to read all remaining data from this file object when Content-Length is defined and since there were two instances of this object and the client supplied no additional data to read, this resulted in deadlock. The fix implemented in this commit is to reuse the request body file object when it is passed from hgwebdir to hgweb. A test demonstrating `hg clone` and `hg push` via hgwebdir has been added. Without this patch, the test hangs when doing `hg clone`. Surprisingly, this must mean that we have effectively no test coverage of the wire protocol when run via hgwebdir. Differential Revision: https://phab.mercurial-scm.org/D3427

remotenames: mark the extension as EXPERIMENTAL I still don't feel confident about locking the behavior of all the things in the remotenames extension. Moreover the extension was introduced in this cycle only. Let's mark this extension EXPERIMENTAL for now so that we can change things especially the storage layer if required in next cycle. I will like to use cbor at storage layer too. Differential Revision: https://phab.mercurial-scm.org/D3426

tests: fix test-check-commit.t when all commits are public I'm 99% sure this is a portable use of /bin/[, and it seems to fix the issue I noticed on the buildbot on my machine.

import: fix crash on --exact check of empty commit (issue5702)

tests: mark test-check-interfaces.py as requiring a repo This was failing our 4.6rc1 build like this: mercurial.error.RepoError: repository /tmp/build-debs.zMTRhC/src-4.6rc1 not found Differential Revision: https://phab.mercurial-scm.org/D3425

sshpeer: reflect actual command activity one handshake The output from devel-peer-request is expected to give data about request and roundtrip done to the server. Changeset a9cffd14aa04 changed some of that by grouping hello and between commands call. However, the old sequence of command was "emulated" in sshpeer. Update the sshpeer to reflect this grouping of commands and update the tests that use it.

tests: drop a useless glob in test-infinite-bundlestore.t With the previous breakage tamed, the lack of test output difference was causing the test runner to report "no result code from test" because of this glob.

infinitepush: ensure fileindex bookmarks use '/' separators (issue5840) After loading up with status messages, I noticed that the subsequent matcher was rejecting 'scratch\mybranch' on Windows. No bookmarks were reported back, and the tests subsequently failed. I did a search for 'match', and nothing else looks like it needs to be fixed up, but someone who understands this code should also take a look. I also tried setting `infinitepush.branchpattern=re:scratch\\.*` in library-infinitepush.sh without this change, but that didn't work. Still, should we ban '\' in these bookmarks to avoid confusion? I thought I saw code that sandwiches a pattern between 're:^' and '.*', so perhaps regex characters will need special care? I also noticed comments in externalbundlestore.{read,write} that it won't work on Windows because of opening an open file. But I don't see a test failure, so this may lack test coverage.

interfaceutil: module to stub out zope.interface The startup time of `hg` increased during the 4.6 development cycle. A cause of that was importing more modules and doing more work at module import time. The import of zope.interface and the declaring of various interfaces is partially responsible for the startup time regression. Our current usage of zope.interface doesn't do much at run time: we are merely declaring interfaces and stating that certain types implement various interfaces. Core Mercurial is not (yet) using of any of zope.interface features that actually require that interface plumbing be defined. The only place we actually need the interface metadata is in test-check-interfaces.py. This commit establishes a new interfaceutil module. It exposes the subset of the zope.interface API that we currently use. By default, the APIs no-op. But if an environment variable is set, we export the real zope.interface APIs. Existing importers of zope.interface have been converted to use the new module. test-check-interfaces.py has been updated to define the environment variable so the real zope.interface is used. The net effect of this change is we stop importing 9 zope.interface.* modules and we no longer perform interface bookkeeping when registering interfaces. On my i7-6700K on Linux, a shell loop that runs `hg log -r .` 300 times on a repo with 1 commit shows a significant CPU time improvement (average of 4 runs): 4.5: 14.814s before: 19.028s after: 16.945s And with `run-tests.py -j10` (single run): 4.5: ~3100s (~51.7m) before: ~4450s (~74.2m) after: ~3980s (~66.3m) So this claws back about half of the regressions in 4.6. Differential Revision: https://phab.mercurial-scm.org/D3419

test-fix: normalize precision of mtime copied by 'cp -p' Appears that MSYS cp only copies mtime in seconds.

merge stable heads

Added signature for changeset 1ec874717d8a

Added tag 4.6rc1 for changeset 1ec874717d8a

internals: correct capitalization of 'compression' stream level parameter

test-check-code: prevent from adding Python modules shadowed by ancient C

diffhelper: rename module to avoid conflicts with ancient C module (issue5846) Historically we had had C extensions in mercurial/, which shadows the pure Python modules of the same name forever unless we do clean build/install. I'm sloppy to think about new name, so just dropped the "s".

diffhelpers: backport 9e40bc4c1bde from C implementation 9e40bc4c1bde just says "harden testhunk." I don't think this would be the case, but it makes some sense to avoid negative index.

version: make parser more robust for rc variants and ill-formed strings

phabricator: specify some metadata compatibly with arc Differential Revision: https://phab.mercurial-scm.org/D3414

diff: restore original color scheme for worddiff I'm not young. "red dim" is nearly invisible, and "red" vs "red dim" is too subtle to see difference. And, "underline" is necessary to highlight whitespace changes.

test-fix: fix use of 'f --newer' to check that foo.whole is not updated Here we want to assert not 'mtime(foo.whole.orig) < mtime(foo.whole)'. The condition has to be inverted since 'f --newer' says "newer" if the mtimes match. Alternatively, we could insert 'sleep 2' before 'cp' to ensure that foo.whole.orig is newer than foo.whole, but a fewer sleeps should be better.

context: translate FilteredIndex/LookupError at repo[changeid] (API) This partially backs out ecd3f6909184. It seems layering violation for repo[changeid] to raise storage-level exceptions transparently. Otherwise, we would have to rewrite callers to catch all of them. try: repo[rev_or_node] except (error.RepoLookupError, error.FilteredIndexError, error.FilteredLookupError): pass This would also fix filectx._changectx(), which catches FilteredRepoLookupError to fall back to the unfiltered path.

inifinitepush: fix filebundlestore to close file

notify: only notify for non-filtered revision This should fix issue5821 for hook targeting individual revision.

notify: access the initial revision on an unfiltered repository (issue5821) This should avoid crash when the first revision pushed end up being hidden.

notify: use changelog API to iterate over revision number (issue5821) This will avoid iterating over filtered revision.

infinitepush: open files in binary mode This fixes the scary looking abort in test-infinitepush-ci.t when unbundling: --- tests/test-infinitepush-ci.t +++ tests/test-infinitepush-ci.t.err @@ -84,15 +84,12 @@ $ hg unbundle .hg/scratchbranches/filebundlestore/a4/c2/a4c202c147a9c4bb91bbadb56321fc5f3950f7f2 adding changesets - adding manifests - adding file changes - added 1 changesets with 1 changes to 1 files - new changesets 6cb0989601f1 - (run 'hg update' to get a working copy) - - $ hg glog - o 1:6cb0989601f1 added a - | public + transaction abort! + rollback completed + abort: stream ended unexpectedly (got 68 bytes, expected 218759168) + [255] + + $ hg glog @ 0:67145f466344 initialcommit public This was found by grepping for '"r', "'r", '"w' and "'w" after manually creating a bundle from the same revision, diffing against the corrupt one, and seeing CRs sprinkled around. Sadly, the missing bookmarks are still a problem in the two remaining test failures.

tests: stabilize test-fix.t for Windows

Added signature for changeset ed5448edcbfa

Added tag 4.6rc0 for changeset ed5448edcbfa

merge with default to begin 4.6 freeze # no-check-commit because of many vendored packages

rebase: introduce support for automatically rebasing orphan changes _destautorebase(SRC) is based on the _destrestack(SRC) revset from fbamend. The supporting _possibledestination function is extracted from evolve, with minor cleanups. We've considered some alternatives here: * This change, but with --auto as the flag name. We're hedging our bets on this a little in this change so that if this ends up being the wrong direction we haven't burned the valauble --auto name on rebase. * --destination auto: I've got reservations about the discoverability of this, and we don't currently have a good story for a revset alias of sorts that changes behavior depending on the context in which it's used. * A "rebase presets" feature, where we could use the currently-an-error positional argument space for the rebase command to define presets, so that users could define a 'linearize' preset that specifies --revision='orphan()-obsolete()' and --dest=_destautoorphanrebase(SRC). Personally, I find the third option somewhat appealing, but am hesitant to "spend" the functionality space of positional arguments to the rebase command. We should revisit the way we expose this functionality sometime in the 4.7 cycle once we've had a chance to vet the implementation of the functionality. Differential Revision: https://phab.mercurial-scm.org/D2668

py3: whitelist six new passing tests Differential Revision: https://phab.mercurial-scm.org/D3406

wireproto: rename wireproto to wireprotov1server (API) We have wireprotov2server, wireprotov1peer, and wireprotov2peer. wireproto only contains server functionality. So it makes sense to rename it to wireprotov1server so the naming aligns with everything else. Differential Revision: https://phab.mercurial-scm.org/D3400

wireproto: move version 2 commands dict to wireprotov2server This was the final piece of version 2 referenced in wireproto. The break between server implementations is now much cleaner. Differential Revision: https://phab.mercurial-scm.org/D3399

wireproto: move supportedcompengines out of wireproto This function is used by both version 1 and version 2. It belongs in a common module. "wireprototypes" may not be the best module name. I may rename it... Differential Revision: https://phab.mercurial-scm.org/D3398

wireproto: reimplement dispatch() for version 2 server The code is minimal. I'm trying to create a cleaner break between version 1 and version 2 server code. Differential Revision: https://phab.mercurial-scm.org/D3397

wireproto: move command registration types to wireprototypes These are shared across wire protocol implementations. wireprototypes is our module for common code. Differential Revision: https://phab.mercurial-scm.org/D3396

wireproto: make version 2 @wireprotocommand an independent function Previously, the code for this decorator was shared between version 1 and version 2 commands. Very few parts of the function were identical. So I don't think sharing is justified. wireprotov2server now has its own @wireprotocommand decorator function. Because the decorator is no longer shared, code for configuring the transport policy has been removed. i.e. commands must have separate implementations for each wire protocol version. Differential Revision: https://phab.mercurial-scm.org/D3395

wireproto: don't pass transportpolicy argument The default is version 1 only. So we don't need to pass this argument when declaring commands. Differential Revision: https://phab.mercurial-scm.org/D3394

forget: rename --confirm to --interactive Differential Revision: https://phab.mercurial-scm.org/D3405

notify: add maxdiffstat option to truncate long file lists Large scale changes like a new GCC version can easily result in 1MB+ emails due to diffstat alone. The new maxdiffstat option truncates the list similar to what maxdiff already provides for the diffs. Differential Revision: https://phab.mercurial-scm.org/D3402

log: fix crash on empty revision with --copies switch If a revset is empty, .max() raises ValueError. I don't see any reason to recompute the revs, so I made it reuse the one returned by logcmdutil.getrevs(). If no revs specified by command line, the endrev will be smartset.spanset(repo) + 1, which is basically the same as len(repo), the default of getrenamedfn(). If --follow specified, revs.max() points to the working parent, which seems more correct.

revsetlang: do not pass in non-bytes to parse() Since parse() isn't a simple function, we shouldn't expect it would raise TypeError or ValueError for invalid inputs. Before, TypeError was raised at 'if pos != len(spec)', which was quite late to report an error. This patch also makes tokenize() detect invalid object before converting it to a py3-safe bytes. Spotted while adding the 'revset(...)' hack to _parsewith().

fix: use templater to substitute values in command string bytes.format() isn't supported on Python 3. Luckily, our template syntax is similar so we can reuse it. We need a hack to disable \-escapes as '\' is a directory separator on Windows.