revsetbenchmark: convert update to proper subprocess call

revsetbenchmark: fix semicolon

revsetbenchmark: simplify and convert the script to python The script is now in python. That translation is very raw, more improvement to comes: The "current code" and "base" entry have been dropped. This is trivial to get same result using a tagged revision or "." in the list of benchmarked revision.

revsetbenchmapi: fix template The revision description missed an \n

perf: unroll the result in perfrevset With the new lazy revset implementation, we need to actually read all elements to trigger all the computations. Otherwise a no-op if of course much faster than the full work.

_addset: add a __len__ method Back in the time where repo.revs(...) returned a list, calling `len(...)` on the result was quite common. We reinstall this on _addset. There is absolutely no easy way to test this from the command line. The commands using this in the evolve extension will eventually land into core.

bundle2: implement the mandatory/advisory logic for parameter Parameter starting with an upper case are mandatory, the one starting with a lower case are advisory and may be ignored.

bundle2: print debug information during unbundling The unbundler class is now feed with an ui object and use it to transmit data about the unbundling process.

bundle2: print debug information during bundling The bundler class is now feed with an ui object and use it to transmit data about the bundling process.

bundle2: add ability to write to a file to the test Next changesets are about improving debug output during bundling. We won't be able to simply bundle to stdin in this case. The bundle2 stream would be interleaved with debug output.

subrepo: factor out Git version check to add doctests Followup to 6a2acb0d9352::e5641536e4d5.

contrib: add "hgperf" command to measure performance of commands easily Newly added "hgperf" command repeats "dispatch.runcommand()" for specified Mercurial command and measure performance of it in the same manner of "contrib/perf.py" extension. Users (mainly developers) can examine performance of the target command easily, without adding new entry for it to "contrib/perf.py" extension.

grep: exit loop immediately, if matching is found in the file for "hg grep -l" Before this patch, internal function "display()" of "hg grep" is not efficient for "-l"/"--files-with-matches", because loop is continued, even after the first matching is found in the specified file. This patch exits loop immediately, if matching is found for "--files-with-matches". In this case, "before is None" is equal to "opts.get('files_with_matches')".

grep: use "found" instead of "filerevmatches" examination for efficiency Before this patch, internal function "display()" of "hg grep" stores whether matching is already found or not into the dictionary "filerevmatches" by "(fn, rev)" tuple as the key. But this is redundant, because: - "filerevmatches" is local variable of "display()", so each "display()" invocations don't affect others - both "fn" and "rev" (gotten from "ctx" argument) are never changed in each "display()" invocations Then, "filerevmatches" should have only one entry at most, and "(fn, rev) in filerevmatches" should be equal to "found". This patch uses "found" instead of "filerevmatches" examination for efficiency.

grep: reuse the first "util.binary()" result for efficiency Before this patch, to check whether the file in the specified revision is binary or not, "util.binary()" is invoked via internal function "binary()" of "hg grep" once per a line of "hg grep" output, even though binary-ness is not changed in the same file. This patch reuses the first "util.binary()" invocation result by annotating internal function "binary()" with "@util.cachefunc". Performance improvement measured by "hgperf grep -r 88d8e568add1 vfs mercurial/scmutil.py": before this patch: ! wall 0.024000 comb 0.015600 user 0.015600 sys 0.000000 (best of 118) after this patch: ! wall 0.023000 comb 0.015600 user 0.015600 sys 0.000000 (best of 123) Status of recent(88d8e568add1) "mercurial/scmutil.py": # of lines: 919 (may affect cost of search) # of bytes: 29633 (may affect cost of "util.binary()") # of matches: 22 (may affect frequency of "util.binary()")

util: add the code path to "cachefunc()" for the function taking no arguments Before this patch, "util.cachefunc()" caches the value returned by the specified function into dictionary "cache", even if the specified function takes no arguments. In such case, "cache" has at most one entry, and distinction between entries in "cache" is meaningless. This patch adds the code path to "cachefunc()" for the function taking no arguments for efficiency: to store only one cached value, using list "cache" is a little faster than using dictionary "cache".

help: add examples to incoming

revset: fix generatorset race condition If two things were iterating over a generatorset at the same time, they could miss out on the things the other was generating, resulting in incomplete results. This fixes it by making it possible for two things to iterate at once, by always checking the _genlist at the beginning of each iteration. I was only able to repro it with pending changes from my other commits, but they aren't ready yet. So I'm unable to add a test for now.

tests: don't hardcode path to bash interpreter Use the env binary to figure out the correct bash to use. Certain systems ships with an ancient version of bash, but the user might have installed a newer one that is earlier in $PATH. For example the current version of Mac OS X ships version 3.2.51 of bash, which does not understand new fancy builtins such as readarray. A user might install a newer version of bash, use that as their shell and add that path before bin.

contrib: don't hardcode path to bash interpreter Use the env binary to figure out the correct bash to use. Certain systems ships with an ancient version of bash, but the user might have installed a newer one that is earlier in $PATH. For example the current version of Mac OS X ships version 3.2.51 of bash, which does not understand new fancy builtins such as readarray. A user might install a newer version of bash, use that as their shell and add that path before bin.

contrib: explicitly enable perf extension for revset tests If a developer doesn't have the perf extension enabled the revset benchmarking will fail. This patch explicitly enables the perf extension when running the tests.

merge with stable

revset: improve performance of _generatorset.__contains__ (issue 4201) _generatorset.__contains__ and __contains__ from child classes were calling into __iter__ to look for values. Since all previously-encountered values from the generator were cached and checked in __contains__ before this iteration, __contains__ was effectively performing iteration busy work which could lead to an explosion of redundant work. This patch changes __contains__ to be more intelligent. Instead of looking at all values via __iter__, __contains__ will instead go straight to "new" values from the underlying generator. On a clone of the Firefox repository with around 200,000 changesets, this patch decreases the execution time of the revset '::(200067)::' from ~100s to ~4s on the author's machine. Rebase operations (which use the aforementioned revset), speed up accordingly.

localrepo: omit ".hgsubstate" also from "added" files Before this patch, "localrepository.commit()" omits ".hgsubstate" from "modified" (changes[0]) and "removed" (changes[2]) file list before checking subrepositories, but leaves one in "added" (changes[1]) as it is. Then, "localrepository.commit()" adds ".hgsubstate" into "modified" or "removed" list forcibly, according to subrepository statuses. If "added" contains ".hgsubstate", the committed context will contain two ".hgsubstate" in its "files": one from "added" (not omitted one), and another from "modified" or "removed" (newly added one). How many times ".hgsubstate" appears in "files" changes node hash, even though revision content is same, because node hash calculation uses the specified "files" directly (without duplication check or so). This means that node hash of committed revision changes according to existence of ".hgsubstate" in "added" at "localrepository.commit()". ".hgsubstate" is treated as "added", not only in accidental cases, but also in the case of "qpush" for the patch adding ".hgsubstate". This patch omits ".hgsubstate" also from "added" files before checking subrepositories. This patch also omits ".hgsubstate" exclusion in "qnew"/"qrefresh" introduced by changeset d666da075b91, because this patch makes them meaningless. "hg parents --template '{files}\n'" newly added to "test-mq-subrepo.t" enhances checking unexpected multiple appearances of ".hgsubstate" in "files" of created/refreshed MQ revisions.

dispatch: only do __import__(debugger) when a debugger is requested When having ui.debugger=somedebugger in one's ~/.hgrc, this then somedebugger would be imported for every hg command. With this patch, this import only happens if the --debugger parameter is passed.

hg: use "os.path.join()" to join path components which may be empty (issue4203) Changset 2d0ab571b822 rewriting "hg.copystore()" with vfs uses 'dstbase + "/lock"' instead of "os.path.join()", because target files given from "store.copyfiles()" already uses "/" as path separator But in the repository using revlog format 0, "dstbase" becomes empty ("data" directory is located under ".hg" directly), and 'dstbase + "/lock"' is treated as "/lock": in almost all cases, write access to "/lock" causes "permission denied". This patch uses "os.path.join()" to join path components which may be empty in "hg.copystore()".

revset: document the regular expression support for tag(name) This has been supported since 20f55613fb2a, in 2.3.

help: allow -k to find debug commands

help: let 'hg help debug' show the list of secret debug commands The names of the debug commands are not easy to remember and they are not easy to find.

tests: add run-tests --changed option for running tests changed in revisions Convenient when polishing patches and changing details of how they change test output. This will probably break in weird ways for revsets with special quoting ... but it is good enough for run-tests. Usage example: yes | ./run-tests.py -li --changed qparent

repo: rephrase the "missing requirement" error message Unknown requirements will now be reported as: abort: repository requires features unknown to this Mercurial: largefiles! (see http://mercurial.selenic.com/wiki/MissingRequirement for more information) Some features of this phrasing: * avoid double ':' in abort message * make it more clear who requires and knows what * don't quote the requirement names - it is not something the user entered or need the exact spelling of ... and it is "identifiers" that are unambiguous anyway * remove double hint by removing "(upgrade Mercurial)" comment * don't mention upgrading Mercurial without mentioning enabling the feature - instead, just refer to wiki page for both * don't just talk about "details", talk about "more information"

revpair: drop useless conditional The `if not revs:` case is tested at the beginning of the function.

merge with stable

subrepo: add trailing newlines to warnings

subrepo: convert matched string to integer before comparing Git version (1, '4') is greater than (1, 5) so the version check never actually worked.

subrepo: only retrieve the first two components of the Git version This makes the version detection compatible with Git versions like '1.9-rc0'. We only cared about the first two components of the version anyway.

bundle2: force the first char of parameter to be an letter. We need a case sensitive character to convey mandatory/advisory parameter semantic in a later patches.

bundle2: refuse empty parameter name The bundle2 now raise value error when seeing invalid parameter names. The first introduced rules is: no empty parameter. The test extension is improve to properly abort when ValueError are encountered.

bundle2: urlunquote stream parameter name and value during unbundling Align to new escaping used during bundling.

bundle2: urlquote stream parameter name and value This introduces support for arbitrary characters in stream parameters name and value. The urlquote format has been chosen because it is: - simple, - standard, - no-op on simple alphanumerical entry.

bundle2: support for unbundling parameter value The unbundler now understand parameter value. introduced in the previous changeset.

bundle2: support for bundling parameter value Parameter can now have a value. We use a `<name>=<value>` form inspired from capabilities. There is still no kind of escaping in the name or value, yet.

bundle2: clarify stream parameter design in the documentation Stream level parameter have very restricted use case. Clarify why we chosen a textual format and point that applicative data goes in applicative parts.

repoview: add non-global tags to candidate list for blocking hidden changesets Previously, only bookmarks would be considered for blocking a changeset from being hidden. Now, we also consider non-global tags. This is helpful if we have local tags that might be hard to find once they are hidden, or tag that are added by extensions (e.g. hggit or remotebranches).

histedit: select the lowest rev when looking for a root in a revset (bc) When we specify a revision or a revset we just get the last element from the list. For revsets this can lead to unintended effects where you specify a revset like only() but instead histedit selects the highest revision in the set as root. Therefore we should always use the lowest revision number as root.

bundle2: support for unbundling simple parameter the unbundler now understand simple list of parameter.

bundle2: support bundling simple parameter This changeset add bundling capacity for simple parameters, not value or any special case are handled.

bundle2: make sure the unbundler refuse non bundle2 stream We now make use of the magic string at the beginning of the file.

bundle2: a very first version of bundle2 unbundler This changeset introduce an unbundler class to match the bundle2 bundler. It is currently able to unbundle an empty bundle2 only and will gain more feature at the same pace than the bundler. It also comes with its special extension command in test.

bundle2: very first version of a bundle2 bundler This changeset is the very first of a long series. It create a new bundle2 module and add a simple class that generate and empty bundle2 container. The module is documented with the current state of the implementation. For information about the final goal you may want to consult the mercurial wiki page: http://mercurial.selenic.com/wiki/BundleFormat2 The documentation of the module will be updated with later patches adding more and more feature to the format. This patches also introduce a test case. This test case build and use its own small extension that use the new bundle2 module. Since the new format is unable to do anything right now, we could not use real mercurial code to test it. Moreover, some advanced feature of the bundle2 spec will not be used by core mercurial at all. So we need to have them tested.

clone: abort if default destination has no meaningful name (BC) If source URL has no path, default destination is resolved as '.'. It is surprising than useful, and perhaps an unexpected behavior. This change does not solve issue3880, but can avoid to clone into current directory by accident.

clone: add doctest for default destination

scmutil: fix revrange when multiple revs are specified revrange was trying to add a list to a revset class, but revset classes only support adding with other revset classes. So wrap the lists in basesets.

parsers: fix compiler errors on MSVC 2008 This broke in 3681de20b0a7.

check-code: check for argument passing py2.6ism

merge with stable

sshpeer: only print out 'running ssh' messages in debug mode (BC) Previously, if another command was run with --verbose, and for whatever reason that invoked sshpeer, we'd get a 'running ssh' message from sshpeer. This extra line would interfere with that command's output and cause dumb parsers to break. For example, hg annotate can be run with --verbose to get full usernames. This, combined with the third-party remotefilelog extension which can cause ssh connections to be created, leads to an extra 'running ssh' line that breaks most parsers. This patch is (BC) because hg pull --verbose will no longer print out exactly what ssh command it is running. No tests are affected by this change.

tests: small refactoring of run-tests' handling of tests list

merge: mark mergestate as dirty when resolve changes _state Correctness - no visible difference so far.

backout: improve confusing 'cannot backout change on a different branch' abort These days 'branch' usually refer to a named branch. Instead, abort with 'cannot backout change that not is an ancestor'.

config: set a 'source' in most cases where config don't come from file but code Some extensions set configuration settings that showed up in 'hg showconfig --debug' with 'none' as source. That was confusing. Instead, they will now tell which extension they come from. This change tries to be consistent and specify a source everywhere - also where it perhaps is less relevant.

config: don't set source when no source is specified - don't overwrite with '' This prevents ui.fixconfig from overwriting the source of paths and it will thus show up in showconfig --debug.

config: give a useful hint of source for the most common command line settings 'hg showconfig --debug' will instead of: none: ui.verbose=False give the better hint: --verbose: ui.verbose=False

config: backout 77f1f206e135 - 743daa601445 removed the only use of overlay

mq: omit ".hgsubstate" from qnew/qrefresh target list for consistent node hash Before this patch, even if specified file patterns and -I/-X options cause listing ".hgsubstate" up in the target list, qnew/qrefresh put ".hgsubstate" into the target list individually and forcibly. This changes how many times ".hgsubstate" appear in the target list according to run-time conditions, and causes inconsistent node hash, even though revision content is same, because node hash calculation uses the specified target list directly (without duplication check or so). This patch always omits ".hgsubstate" from qnew/qrefresh target list for consistent node hash. This omitting doesn't miss including ".hgsubstate" changes, because: - "localrepository.commit()" puts ".hgsubstate" into the target list for "commitctx()" forcibly if needed - "mq.putsubstate2changes()" puts ".hgsubstate" into the target list for "patch.diff()" if it is not yet listed up

qnew: omit meaningless and harmful putting subrepositories into target list Before this patch, qnew puts updated subrepositories into target list forcibly, if any of -I, -X or patterns are specified. But this is meaningless and harmful, because: - putting subrepositories into target list doesn't affect the result of "localrepository.status()" "dirstate.status()" invoked via "localrepository.status()" always omits subrepositories from the result of it - any -I/-X opts and empty "pats" causes unexpected failure when any -I/-X opts are specified, "inclsubs" are always added to "pats", even if "pats" is empty. but this changes meaning of "pats" from "including all to be included" to "including only listed subrepositories" this may exclude ".hgsub" and cause unexpected exception raising ("can't commit subrepos without .hgsub" ). - qnew at other than repository root (with -I, -X or any patterns) causes unexpected failure "scmutil.match()" treats pattern without syntax type as 'relpath' type (= one rooted at cwd). but qnew puts subrepository paths rooted at the repository root, and it causes unexpected exception raising ("SUBREPO not under root ROOT" in "pathutil.canonpath()"), if "hg qnew" is executed at other than repository root with -I, -X or any patterns. This patch omits meaningless and harmful putting subrepositories into target list. This omitting doesn't miss including updated subrepositories, because subrepositories are specified to "scmutil.matchfiles()" directly, to get "match" object for "localrepository.commit()".

convert: more clear documentation of the 'include' default of a 'include .' At first glance it can be confusing that adding a superfluous include directive will exclude more files.

config: mention edit options and config topic in help

config: add --global and --local flags These start an editor on the system-wide or repository-level config files.

revrange: pass repo to revset parser This allows handling of hyphenated symbols for command-line revsets.

revset: try to handle hyphenated symbols if lookup callback is available Formerly an expression like "2.4-rc::" was tokenized as 2.4|-|rc|::. This allows dashes in symbols iff the whole symbol-like string can be looked up. Otherwise, it's tokenized as a series of symbols and operators. No attempt is made to accept dashed symbols inside larger symbol-like string, for instance foo-bar or bar-baz inside foo-bar-baz.

revset: pass a lookup function to the tokenizer

parser: allow passing a lookup function to a tokenizer This will allow us to dynamically handle hyphenated symbols in revsets.

benchmark-revset: add full version of benchmarked revset All revsets added to benchmark so far are aimed to show an improvement of performance from laziness. We had more wider version to track impact of laziness on them.

localrepo: rename capability set to lower case. This is not C and they are not even constant.

wireproto: move wireproto capabilities computation in a subfunction It will help people that need to add capabilities (in a more subtle was that just adding some to the list) in multiple way: 1. This function returns a list, not a string. Making it easier to look at, extend or alter the content. 2. The original capabilities function will be store in the dictionary of wire protocol command. So extension that wrap this function also need to update the dictionary entry. Both wrapping and update of the dictionary entry are needed because the `hello` wire protocol use the function itself. This is specifically sneaky for extension writer as ssh use the `hello` command while http use the `capabilities` command. With this new `_capabilities` function there is one and only one obvious place to wrap when needed.

wireproto: extract capabilities list in outside the wireproto function Before that it was over complicated to add capabilities from an extension. This mimic was is done for capabilities in localrepo.

merge with stable

commit: propagate --secret option to subrepos (issue4182) Before this patch, `hg commit --secret` was not getting propagated correctly, and subrepos were not getting the commit in the secret phase. The problem is that subrepos get their ui from the base repo's baseui object and ignore the ui object passed on to them. This sets and restores both ui objects with the appropriate option.

amend: save commit message into ".hg/last-message.txt" Before this patch, commit message (may be manually edited) for "commit --amend" is never saved into ".hg/last-message.txt", because it uses "localrepository.commitctx()" instead of "localrepository.commit()": saving into ".hg/last-message.txt" is executed only in the latter. This patch saves commit message for "commit --amend" into ".hg/last-message.txt" just after user editing. This is the simplest implementation to fix on stable. Editing and saving commit message for memctx should be centralized into the framework like "localrepository.commit()" with "editor" argument or so in the future.

histedit: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "fold" command in histedit-ing is never saved into ".hg/last-message.txt", because it uses "localrepository.commitctx()" instead of "localrepository.commit()": saving into ".hg/last-message.txt" is executed only in the latter. This patch saves manually edited commit message for "fold" command in histedit-ing into ".hg/last-message.txt" just after user editing. This is the simplest implementation to fix on stable. Editing and saving commit message for memctx should be centralized into the framework like "localrepository.commit()" with "editor" argument or so in the future.

qfold: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg qfold -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg qfold -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m/-l options as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future. This patch uses repository wrapping class for exception raising before saving commit message in "localrepository.commit()" easily and certainly, because such exception requires corner case condition.

qnew: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg qnew -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg qnew -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m/-l options as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future. This patch uses repository wrapping class for exception raising before saving commit message in "localrepository.commit()" easily and certainly, because such exception requires corner case condition.

tag: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg tag -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg tag -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m option (-l is not supported for "hg tag") as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future.

rebase: use "commitforceeditor" instead of "ui.edit()" for "--collapse" Before this patch, "rebase --collapse --edit" without "--message" and "--logfile" invokes editor twice unexpectedly: 1. explicit "ui.edit()" invocation in rebase extension itself 2. indirect invocation in "localrepository.commit()" with "editor = commitforceeditor" assigned by "--edit" option This patch uses indirect "commitforceeditor" invocation instead of "ui.edit()" for "--collapse" without "--message" and "--logfile" to: - suppress redundant the former invocation - ensure editor invocation even when "--edit" is not specified

localrepo: save manually edited commit message as soon as possible Before this patch, "localrepository.commit()" invokes specified "editor" to edit commit message manually, and saves it after checking sub-repositories. This may lose manually edited commit message, if unexpected exception is raised while checking (or commiting recursively) sub-repositories. This patch saves manually edited commit message as soon as possible.

hgk: enable selected patch text on Windows Port a patch from gitk. Original description: On windows, mouse input follows the keyboard focus, so to allow selecting text from the patch canvas we must not shift focus back to the top level. This change has no negative impact on X, so we don't explicitly test for Win32 on this change. This provides similar selection capability as already available using X-Windows. Signed-off-by: Mark Levedahl <mdl123@verizon.net> Signed-off-by: Paul Mackerras <paulus@samba.org>

hgk: ignore ctrl-z as EOF on windows Port a patch from gitk. Original description: Cygwin's Tcl is configured to honor any occurence of ctrl-z as an end-of-file marker, while some commits in the git repository and possibly elsewhere include that character in the commit comment. This causes gitk ignore commit history following such a comment and incorrect graphs. This change affects only Windows as Tcl on other platforms already has eofchar == {}. This fixes problems noted by me and by Ray Lehtiniemi, and the fix was suggested by Shawn Pierce. Signed-off-by: Mark Levedahl <mdl123@verizon.net> Signed-off-by: Paul Mackerras <paulus@samba.org>

graphmod: changed code in dagwalker to use lazy implementations Used lazy methods when possible.

webcommands: changed code to use lazy classes when calling dagwalker This needs to be changed to use a baseset since dagwalker now expects to receive a smartset. This is basically wrapping revs into a baseset to be compatible with smartset implementations.

cmdutil: changed max method for lazy call Used the lazy max call instead of the python max implementation to be able to use lazysets for graphlog.

getgraphlogrevs: return an empty baseset instead of a empty list We aims at returning smartset only so that function higher in the stack can use smartset feature.

getgraphlogrevs: do not convert smartset to baseset We are now sure that revs is a smartset. We remove the baseset call that would defeat any lazyness.

cmdutil: changed revset for spanset Instead of using baseset(repo.changelog) changed it for spanset(repo) which is much faster.

cmdutil: changed code in _makegraphlogrevset not to use getitem __getitem__ is a method that is not implemented lazily on many of the new classes and it can be easily replaced with a structure that takes advantage of the new lazy implementations instead.

cmdutil: changed code in getgraphlogrevs not to use getitem __getitem__ is a method that is not implemented lazily on many of the new classes and it can be easily replaced with a structure that takes advantage of the new lazy implementations instead.