bundle2: very first version of a bundle2 bundler This changeset is the very first of a long series. It create a new bundle2 module and add a simple class that generate and empty bundle2 container. The module is documented with the current state of the implementation. For information about the final goal you may want to consult the mercurial wiki page: http://mercurial.selenic.com/wiki/BundleFormat2 The documentation of the module will be updated with later patches adding more and more feature to the format. This patches also introduce a test case. This test case build and use its own small extension that use the new bundle2 module. Since the new format is unable to do anything right now, we could not use real mercurial code to test it. Moreover, some advanced feature of the bundle2 spec will not be used by core mercurial at all. So we need to have them tested.

clone: abort if default destination has no meaningful name (BC) If source URL has no path, default destination is resolved as '.'. It is surprising than useful, and perhaps an unexpected behavior. This change does not solve issue3880, but can avoid to clone into current directory by accident.

clone: add doctest for default destination

scmutil: fix revrange when multiple revs are specified revrange was trying to add a list to a revset class, but revset classes only support adding with other revset classes. So wrap the lists in basesets.

parsers: fix compiler errors on MSVC 2008 This broke in 3681de20b0a7.

check-code: check for argument passing py2.6ism

merge with stable

sshpeer: only print out 'running ssh' messages in debug mode (BC) Previously, if another command was run with --verbose, and for whatever reason that invoked sshpeer, we'd get a 'running ssh' message from sshpeer. This extra line would interfere with that command's output and cause dumb parsers to break. For example, hg annotate can be run with --verbose to get full usernames. This, combined with the third-party remotefilelog extension which can cause ssh connections to be created, leads to an extra 'running ssh' line that breaks most parsers. This patch is (BC) because hg pull --verbose will no longer print out exactly what ssh command it is running. No tests are affected by this change.

tests: small refactoring of run-tests' handling of tests list

merge: mark mergestate as dirty when resolve changes _state Correctness - no visible difference so far.

backout: improve confusing 'cannot backout change on a different branch' abort These days 'branch' usually refer to a named branch. Instead, abort with 'cannot backout change that not is an ancestor'.

config: set a 'source' in most cases where config don't come from file but code Some extensions set configuration settings that showed up in 'hg showconfig --debug' with 'none' as source. That was confusing. Instead, they will now tell which extension they come from. This change tries to be consistent and specify a source everywhere - also where it perhaps is less relevant.

config: don't set source when no source is specified - don't overwrite with '' This prevents ui.fixconfig from overwriting the source of paths and it will thus show up in showconfig --debug.

config: give a useful hint of source for the most common command line settings 'hg showconfig --debug' will instead of: none: ui.verbose=False give the better hint: --verbose: ui.verbose=False

config: backout 77f1f206e135 - 743daa601445 removed the only use of overlay

mq: omit ".hgsubstate" from qnew/qrefresh target list for consistent node hash Before this patch, even if specified file patterns and -I/-X options cause listing ".hgsubstate" up in the target list, qnew/qrefresh put ".hgsubstate" into the target list individually and forcibly. This changes how many times ".hgsubstate" appear in the target list according to run-time conditions, and causes inconsistent node hash, even though revision content is same, because node hash calculation uses the specified target list directly (without duplication check or so). This patch always omits ".hgsubstate" from qnew/qrefresh target list for consistent node hash. This omitting doesn't miss including ".hgsubstate" changes, because: - "localrepository.commit()" puts ".hgsubstate" into the target list for "commitctx()" forcibly if needed - "mq.putsubstate2changes()" puts ".hgsubstate" into the target list for "patch.diff()" if it is not yet listed up

qnew: omit meaningless and harmful putting subrepositories into target list Before this patch, qnew puts updated subrepositories into target list forcibly, if any of -I, -X or patterns are specified. But this is meaningless and harmful, because: - putting subrepositories into target list doesn't affect the result of "localrepository.status()" "dirstate.status()" invoked via "localrepository.status()" always omits subrepositories from the result of it - any -I/-X opts and empty "pats" causes unexpected failure when any -I/-X opts are specified, "inclsubs" are always added to "pats", even if "pats" is empty. but this changes meaning of "pats" from "including all to be included" to "including only listed subrepositories" this may exclude ".hgsub" and cause unexpected exception raising ("can't commit subrepos without .hgsub" ). - qnew at other than repository root (with -I, -X or any patterns) causes unexpected failure "scmutil.match()" treats pattern without syntax type as 'relpath' type (= one rooted at cwd). but qnew puts subrepository paths rooted at the repository root, and it causes unexpected exception raising ("SUBREPO not under root ROOT" in "pathutil.canonpath()"), if "hg qnew" is executed at other than repository root with -I, -X or any patterns. This patch omits meaningless and harmful putting subrepositories into target list. This omitting doesn't miss including updated subrepositories, because subrepositories are specified to "scmutil.matchfiles()" directly, to get "match" object for "localrepository.commit()".

convert: more clear documentation of the 'include' default of a 'include .' At first glance it can be confusing that adding a superfluous include directive will exclude more files.

config: mention edit options and config topic in help

config: add --global and --local flags These start an editor on the system-wide or repository-level config files.

revrange: pass repo to revset parser This allows handling of hyphenated symbols for command-line revsets.

revset: try to handle hyphenated symbols if lookup callback is available Formerly an expression like "2.4-rc::" was tokenized as 2.4|-|rc|::. This allows dashes in symbols iff the whole symbol-like string can be looked up. Otherwise, it's tokenized as a series of symbols and operators. No attempt is made to accept dashed symbols inside larger symbol-like string, for instance foo-bar or bar-baz inside foo-bar-baz.

revset: pass a lookup function to the tokenizer

parser: allow passing a lookup function to a tokenizer This will allow us to dynamically handle hyphenated symbols in revsets.

benchmark-revset: add full version of benchmarked revset All revsets added to benchmark so far are aimed to show an improvement of performance from laziness. We had more wider version to track impact of laziness on them.

localrepo: rename capability set to lower case. This is not C and they are not even constant.

wireproto: move wireproto capabilities computation in a subfunction It will help people that need to add capabilities (in a more subtle was that just adding some to the list) in multiple way: 1. This function returns a list, not a string. Making it easier to look at, extend or alter the content. 2. The original capabilities function will be store in the dictionary of wire protocol command. So extension that wrap this function also need to update the dictionary entry. Both wrapping and update of the dictionary entry are needed because the `hello` wire protocol use the function itself. This is specifically sneaky for extension writer as ssh use the `hello` command while http use the `capabilities` command. With this new `_capabilities` function there is one and only one obvious place to wrap when needed.

wireproto: extract capabilities list in outside the wireproto function Before that it was over complicated to add capabilities from an extension. This mimic was is done for capabilities in localrepo.

merge with stable

commit: propagate --secret option to subrepos (issue4182) Before this patch, `hg commit --secret` was not getting propagated correctly, and subrepos were not getting the commit in the secret phase. The problem is that subrepos get their ui from the base repo's baseui object and ignore the ui object passed on to them. This sets and restores both ui objects with the appropriate option.

amend: save commit message into ".hg/last-message.txt" Before this patch, commit message (may be manually edited) for "commit --amend" is never saved into ".hg/last-message.txt", because it uses "localrepository.commitctx()" instead of "localrepository.commit()": saving into ".hg/last-message.txt" is executed only in the latter. This patch saves commit message for "commit --amend" into ".hg/last-message.txt" just after user editing. This is the simplest implementation to fix on stable. Editing and saving commit message for memctx should be centralized into the framework like "localrepository.commit()" with "editor" argument or so in the future.

histedit: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "fold" command in histedit-ing is never saved into ".hg/last-message.txt", because it uses "localrepository.commitctx()" instead of "localrepository.commit()": saving into ".hg/last-message.txt" is executed only in the latter. This patch saves manually edited commit message for "fold" command in histedit-ing into ".hg/last-message.txt" just after user editing. This is the simplest implementation to fix on stable. Editing and saving commit message for memctx should be centralized into the framework like "localrepository.commit()" with "editor" argument or so in the future.

qfold: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg qfold -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg qfold -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m/-l options as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future. This patch uses repository wrapping class for exception raising before saving commit message in "localrepository.commit()" easily and certainly, because such exception requires corner case condition.

qnew: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg qnew -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg qnew -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m/-l options as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future. This patch uses repository wrapping class for exception raising before saving commit message in "localrepository.commit()" easily and certainly, because such exception requires corner case condition.

tag: save manually edited commit message into ".hg/last-message.txt" Before this patch, manually edited commit message for "hg tag -e" isn't saved into ".hg/last-message.txt" until it is saved by "localrepository.savecommitmessage()" in "localrepository.commit()". This may lose such commit message, if unexpected exception is raised. This patch saves manually edited commit message for "hg tag -e" into ".hg/last-message.txt" just after user editing. This patch doesn't save the message specified by -m option (-l is not supported for "hg tag") as same as other commands. This is the simplest implementation to fix on stable. Editing and saving commit message should be centralized into the framework of "localrepository.commit()" with "editor" argument in the future.

rebase: use "commitforceeditor" instead of "ui.edit()" for "--collapse" Before this patch, "rebase --collapse --edit" without "--message" and "--logfile" invokes editor twice unexpectedly: 1. explicit "ui.edit()" invocation in rebase extension itself 2. indirect invocation in "localrepository.commit()" with "editor = commitforceeditor" assigned by "--edit" option This patch uses indirect "commitforceeditor" invocation instead of "ui.edit()" for "--collapse" without "--message" and "--logfile" to: - suppress redundant the former invocation - ensure editor invocation even when "--edit" is not specified

localrepo: save manually edited commit message as soon as possible Before this patch, "localrepository.commit()" invokes specified "editor" to edit commit message manually, and saves it after checking sub-repositories. This may lose manually edited commit message, if unexpected exception is raised while checking (or commiting recursively) sub-repositories. This patch saves manually edited commit message as soon as possible.

hgk: enable selected patch text on Windows Port a patch from gitk. Original description: On windows, mouse input follows the keyboard focus, so to allow selecting text from the patch canvas we must not shift focus back to the top level. This change has no negative impact on X, so we don't explicitly test for Win32 on this change. This provides similar selection capability as already available using X-Windows. Signed-off-by: Mark Levedahl <mdl123@verizon.net> Signed-off-by: Paul Mackerras <paulus@samba.org>

hgk: ignore ctrl-z as EOF on windows Port a patch from gitk. Original description: Cygwin's Tcl is configured to honor any occurence of ctrl-z as an end-of-file marker, while some commits in the git repository and possibly elsewhere include that character in the commit comment. This causes gitk ignore commit history following such a comment and incorrect graphs. This change affects only Windows as Tcl on other platforms already has eofchar == {}. This fixes problems noted by me and by Ray Lehtiniemi, and the fix was suggested by Shawn Pierce. Signed-off-by: Mark Levedahl <mdl123@verizon.net> Signed-off-by: Paul Mackerras <paulus@samba.org>

graphmod: changed code in dagwalker to use lazy implementations Used lazy methods when possible.

webcommands: changed code to use lazy classes when calling dagwalker This needs to be changed to use a baseset since dagwalker now expects to receive a smartset. This is basically wrapping revs into a baseset to be compatible with smartset implementations.

cmdutil: changed max method for lazy call Used the lazy max call instead of the python max implementation to be able to use lazysets for graphlog.

getgraphlogrevs: return an empty baseset instead of a empty list We aims at returning smartset only so that function higher in the stack can use smartset feature.

getgraphlogrevs: do not convert smartset to baseset We are now sure that revs is a smartset. We remove the baseset call that would defeat any lazyness.

cmdutil: changed revset for spanset Instead of using baseset(repo.changelog) changed it for spanset(repo) which is much faster.

cmdutil: changed code in _makegraphlogrevset not to use getitem __getitem__ is a method that is not implemented lazily on many of the new classes and it can be easily replaced with a structure that takes advantage of the new lazy implementations instead.

cmdutil: changed code in getgraphlogrevs not to use getitem __getitem__ is a method that is not implemented lazily on many of the new classes and it can be easily replaced with a structure that takes advantage of the new lazy implementations instead.

revset: changed minrev and maxrev implementations to use ordered sets Performance Benchmarking: 0) max(tip:0) 1) min(0:tip) 2) min(0::) b96cb15ec9e0 (2.9.1 release) 0) ! wall 0.005699 comb 0.000000 user 0.000000 sys 0.000000 (best of 450) 1) ! wall 0.005414 comb 0.010000 user 0.010000 sys 0.000000 (best of 493) 2) ! wall 0.025951 comb 0.030000 user 0.030000 sys 0.000000 (best of 107) 05267e6e94dd (public tip at submission time) 0) ! wall 0.015177 comb 0.020000 user 0.020000 sys 0.000000 (best of 175) 1) ! wall 0.014779 comb 0.010000 user 0.010000 sys 0.000000 (best of 189) 2) ! wall 12.345179 comb 12.350000 user 12.350000 sys 0.000000 (best of 3) Current patches: 0) ! wall 0.001911 comb 0.000000 user 0.000000 sys 0.000000 (best of 1357) 1) ! wall 0.001943 comb 0.010000 user 0.010000 sys 0.000000 (best of 1406) 2) ! wall 0.000405 comb 0.000000 user 0.000000 sys 0.000000 (best of 6761)

revset: changed addset to extend _orderedsetmixin Now _addset can use the lazy min and max implementation.

revset: add a default argument for baseset.__init__ We are now able to create empty baseset using `baseset()` as we are able to create empty list with `list()`.

revset: changed orderedlazyset to also extend _orderedsetmixin Now orderedlazyset can use the lazy min and max implementation.

revset: changed spanset to extend _orderedsetmixin Now spanset can use the lazy min and max methods implementation.

revset: added _orderedsetmixin class This class has utility methods for any ordered class to get the min and the max values.

revset: added min and max methods to baseset and lazyset This classes have no particular order so they rely on python min() and max() implementation. This methods will be implemented in every smartset class in future patches. For other classes there are lazy implementations that can be made for this methods.

contrib: make revset benchmark script able to read from stdin This help fine control of what we want to benchmark

contrib: have the revset benchmark test script take a revset The script now selection revision to run benchmark against using a revset query instead of a revision range. It is expected that people benchmarking revset have some knowledge of revset.

contrib: added revset performance benchmarking script This script takes two arguments (starting revision, ending revision) and tests for each revision in between the entire list of revsets in the script using perfrevset.

contrib: added revset examples for benchmarking performance Added list of revsets used for benchmarking revset performance so far.

help: filter out deprecated options with untranslated descriptions When using a different language than English, deprecated options were only removed from the output of `hg help anycmd` when "DEPRECATED" in the options description was translated.

parsers: fail fast if Python has wrong minor version (issue4110) This change causes an informative ImportError to be raised when importing the parsers extension module if the minor version of the currently-running Python interpreter doesn't match that of the Python used when compiling the extension module. This change also exposes a parsers.versionerrortext constant in the C implementation of the module. Its presence can be used to determine whether this behavior is present in a version of the module. The value of the constant is the leading text of the ImportError raised and is set to "Python minor version mismatch". Here is an example of what the new error looks like: Traceback (most recent call last): File "test.py", line 1, in <module> import mercurial.parsers ImportError: Python minor version mismatch: The Mercurial extension modules were compiled with Python 2.7.6, but Mercurial is currently using Python with sys.hexversion=33883888: Python 2.5.6 (r256:88840, Nov 18 2012, 05:37:10) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] at: /opt/local/Library/Frameworks/Python.framework/Versions/2.5/Resources/ Python.app/Contents/MacOS/Python The reason for raising an error in this scenario is that Python's C API is known not to be compatible from minor version to minor version, even if sys.api_version is the same. See for example this Python bug report about incompatibilities between 2.5 and 2.6+: http://bugs.python.org/issue8118 These incompatibilities can cause Mercurial to break in mysterious, unforeseen ways. For example, when Mercurial compiled with Python 2.7 was run with 2.5, the following crash occurred when running "hg status": http://bz.selenic.com/show_bug.cgi?id=4110 After this crash was fixed, running with Python 2.5 no longer crashes, but the following puzzling behavior still occurs: $ hg status ... File ".../mercurial/changelog.py", line 123, in __init__ revlog.revlog.__init__(self, opener, "00changelog.i") File ".../mercurial/revlog.py", line 251, in __init__ d = self._io.parseindex(i, self._inline) File ".../mercurial/revlog.py", line 158, in parseindex index, cache = parsers.parse_index2(data, inline) TypeError: data is not a string which can be reproduced more simply with: import mercurial.parsers as parsers parsers.parse_index2("", True) Both the crash and the TypeError occurred because the Python C API's PyString_Check() returns the wrong value when the C header files from Python 2.7 are run with Python 2.5. This is an example of an incompatibility of the sort mentioned in the Python bug report above. Failing fast with an informative error message results in a better user experience in cases like the above. The information in the ImportError also simplifies troubleshooting for those on Mercurial mailing lists, the bug tracker, etc. This patch only adds the version check to parsers.c, which is sufficient to affect command-line commands like "hg status" and "hg summary". An idea for a future improvement is to move the version-checking C code to a more central location, and have it run when importing all Mercurial extension modules and not just parsers.c.

debuginstall: change showing to checking for consistency and future checking

debuginstall: add Python information to debuginstall output (issue4128) This change adds to the output of "hg debuginstall" information about the Python being used by Mercurial. It adds both the path to the Python executable (i.e. the value of sys.executable) and the version of Python (specifically the major, minor, and micro versions). Below is an example of what the output looks like after this change. The marked lines are the new output lines: $ hg debuginstall checking encoding (UTF-8)... -->showing Python executable (/Users/chris/.virtualenvs/default/bin/python) -->showing Python version (2.7.6) checking Python lib (/Users/chris/.virtualenvs/default/lib/python2.7)... checking installed modules (/Users/chris/mercurial)... checking templates (/Users/chris/mercurial/templates)... checking commit editor... checking username... no problems detected Note that we use the word "showing" without an ellipsis for the new lines because, unlike the other lines (except for "Python lib" which will be adjusted in a subsequent commit), no check follows the display of this information.

revset: add documentation and comment for _generatorset (clean up some old irrelevant comment in the process)

revset: add some documentation for lazyset

revset: added documentation and comment for spanset class

revset: extend sorting tests

merge with crew

revset: changed smartset methods to return ordered addsets Now when adding two structures that are ordered, they are wrapped into an _addset and they get added lazily while keeping the order.

revset: added isascending and isdescending methods to _addset This methods are intended to duck-type baseset, so we will still have _addset as a private class but now we can return it without wrapping it into an orderedlazyset or a lazyset. These were the last methods to add for smartset compatibility.

revset: added __add__ method to _addset This method is intended to duck-type baseset, so we will still have _addset as a private class but we will be able to return it without wrapping it into an orderedlazyset or a lazyset.

revset: added __sub__ mehtod to _addset This method is intended to duck-type baseset, so we will still have _addset as a private class but now will be able to return it without wrapping it into an orderedlazyset or a lazyset.

revset: added __and__ method to _addset This method is intended to duck-type baseset, so we will still have _addset as a private class but we will be able to return it without wrapping it into an orderedlazyset or a lazyset.

revset: added ascending and descending methods to _addset This methods are intended to duck-type baseset, so we will still have _addset as a private class but will be able return it without wrapping it into an orderedlazyset or a lazyset.

revset: added filter method to _addset This method is intended to duck-type baseset, so we will still have _addset as a private class but we will be able return it without wrapping it into an orderedlazyset or a lazyset.

revset: added comments to all methods needed to duck-type from baseset All this methods are required to duck-type for any class that works as a smart set.

revset: use more explicit argument names for baseset methods Use other instead of x and condition instead of l

revset: added isascending and isdescending methods to smartset classes This methods state if the class is sorted in an ascending or descending order We need this to implement methods based on order on smartset classes in order to be able to create new objects with a given order. We cannot just rely on a simple boolean since unordered set are neither ascending nor descending.

revset: added sort method in addset We need this method to duck-type generatorset since this class is not going to be used outside revset.py and we don't need to duck-type baseset. This sort method will only do something when the addset is not already sorted or is not sorted in the way we want it to be.

revset: added reverse method to addset This method is needed to duck type generatorset.

revset: changed _iterator() method on addset to work with a given order If the two collections are in ascending order, yield their values in an ordered way by iterating both at the same time and picking the values to yield.

revset: changed _iterator() in addset to use the generated list when available Now when all the elements have been generated, the iterator will just use the generated list instead of going through all the elements again.

revset: added cached generated list to addset This way when all the values have been generated the list can be sorted without having to generate them all again.

revset: changed sort method to use native sort implementation of smartsets When sort is done by revision or reversed revision number it can just call sort on the set and doesn't have to iterate it all over again.

revset: fixed sorting issue with spanset When a spanset was being sorted it didn't take into account it's current state (ascending or descending) and it reversed itself everytime the reverse parameter was True. This is not yet used but it will be as soon as the sort revset is changed to directly use the structures sort method.

tests: added tests to test sort revset This tests are intended to test sort in many different cases where it could fail when using the new structures

revset: added __nonzero__ method to spanset class Implemented it in a lazy way, just look for the first non-filtered revision and return True if there's any revision at all.

require: provide a link to a wiki page in addition of suggesting upgrade The wiki page is intended to describe several solution to the requirement issue. Some of those solutions does not involve upgrading mercurial. That is very useful for people that can't easily upgrade they Mercurial in some place.

revset: optimized sort method in lazyset class We are taking advantage of the smartset classes sort method when it exists and converting the set to a baseset otherwise.

revset: improve head revset performance Previously the head() revset would iterate over every item in the subset and check if it was a head. Since the subset is often the entire repo, this was slow on large repos. Now we iterate over each item in the head list and check if it's in the subset, which results in much less work. hg log -r 'head()' on a large repo: Before: 0.95s After: 0.28s

revset: added ascending attribute to addset class In case both collections are in an ascending/descending order then we will be able to iterate them lazily while keeping the order.

revset: added set method to addset to duck type generatorset Since this class is only going to be used inside revset.py (it does not duck type baseset) it needs to duck type only a few more methods for the next patches.

merge with default

revsets: backout d04aac468bf4 due to performance regressions

revset: made addset a private class This class is not supposed to be used outside revset.py since it only wraps content that is used by baseset typed classes. It only gets created by revset operations or private methods.

revset: made descgeneratorset a private class This class is not supposed to be used outside revset.py since it only wraps content that is used by baseset typed classes. It only gets created by revset operations or private methods.

revset: made ascgeneratorset a private class This class is not supposed to be used outside revset.py since it only wraps content that is used by baseset typed classes. It only gets created by revset operations or private methods.