revset: fix O(n**2) behaviour of bisect() (issue3381)

convert/svn: do not try converting empty head revisions (issue3347) Subversion conversion works by picking trunk and branches heads, computing a revision graph from them and converting the selected commits. By design we fail to convert empty revisions so we have to be careful when discovering the revision graph. In this particular issue, the source svn repository was a partial mirror made by svnsync. The funny part is svnsync preserves all revisions including empty ones. Also, we trusted ra.stat(path, stop).created_rev to give us the latest revision with changes in path history up to stop. This assumption broke at least when path is '', that is the repository root, which always returned 'stop' revision despited being empty. The workaround is to first trust ra.stat() but if the returned revision appear empty, search the whole path history from stop to r1 until some changes are found.

convert/svn: refactor svn_source.latest() with a nested function We will call it more than once for reasons detailed later.

convert/svn: clarify svn_source.latest() stop arg default value stop=0 could pass for a valid default value at first sight.

factotum: rename mount and path configuration entries The factotum extension used mount and path entries which were too generic. These have been replaced by mountpoint and executable (respectively) to match existing conventions.

Added signature for changeset d9e2f09d5488

Added tag 2.2-rc for changeset d9e2f09d5488

Makefile: be more careful when cleaning up pure/ components The recent introduction of pure/__init__.py causes mercurial/__init__.py to get clobbered by make clean.

merge default into stable for 2.2 code freeze

commit: add option to amend the working dir parent The --amend flag can be used to amend the parent of the working directory with a new commit that contains the changes in the parent in addition to those currently reported by "hg status", if there are any. The old commit is stored in a backup bundle in ".hg/strip-backup"(see "hg help bundle" and "hg help unbundle" on how to restore it). Message, user and date are taken from the amended commit unless specified. When a message isn't specified on the command line, the editor will open with the message of the amended commit. It is not possible to amend public changesets (see "hg help phases") or changesets that have children. Behind the scenes, first commit the update (if there is one) as a regular child of the current parent. Then create a new commit on the parent's parent with the updated contents. Then change the working copy parent to this new combined changeset. Finally, strip the amended commit and update commit created in the beginning. An alternative (cleaner?) approach of doing this is suggested here: http://selenic.com/pipermail/mercurial-devel/2012-March/038540.html It is currently not possible to amend merge commits or recursively, this can be added at a later time.

transplant: remove extraneous whitespace

journal: use tryread helper to backup files (issue3375)

opener: introduce tryread helper This makes it easier to follow the common pattern "read a file or give an empty string if it's missing".

tests: add test for fileset 'subrepo' keyword

revset: make matching() work on python 2.4 tuple.index() was apparently added to python 2.6: http://bugs.python.org/issue1696444 Also remove a trailing comma to make check-code.py happy.

revset: use list instead of tuple for compatibility with python before 2.6 'string elements'.split() instead of explicitly typing a list of strings is used. This is done in other parts of Mercurial code, too.

merge with stable

archive: make it work with svn subrepos (issue3308) - _svncommand() in files() returns a tuple since 0ae98cd2a83f not a string. - _svncommand() in filedata() returns a tuple not a string. - "svn list" returns files but also directories. - "svn list" is not recursive by default. I have no idea what happens to svn:externals possibly embedded in the svn subrepository.

largefiles: hide .hglf/ prefix for largefiles in hgweb This makes the manifest view in hgweb match what you see in the working copy and what you get when you download an archive in hgweb.

hgweb: add hook for remapping repository path into virtual paths Extensions such as largefiles can use this to remap files so they appear in the same location as they do in the user's working copy.

tests: add tests for matching keyword This adds a couple of tests for the revset "matching" keyword: 1. Test that the 2nd parameter is optional 2. Test that all the 1st argument can be a revset and that all the supported fields of the 2nd argument work.

revset: speedup matching() by first matching fields that take less time to match This patch sorts the fields that are passed to the matching function so that it always starts by matching those fields that take less time to match. Not all fields take the same amount of time to match. I've done several measurements running the following command: hg --time log -r "matching(1, field)" on the mercurial repository, and where 'field' was each one of the fields accepted by match. In order to avoid the print overhead (which could be different for different fields, given the different number of matches) I used a modified version of the matching() function which always returns no matches. These tests showed that different fields take wildly different amounts of time to match. Particulary the substate field takes up to 25 seconds to match on my machine, compared to the 0.3 seconds that takes to match the phase field or the 2 seconds (approx) that takes to match most fields. With this patch, matching both the phase and the substate of a revision takes the same amount of time as matching the phase. The field match order introduced by this patch is as follows: phase, parents, user, date, branch, summary, files, description, substate An extra nice thing about this patch is that it makes the match time stable.

revset: speedup matching() by stopping the match early if a field does not match Rather than getting all the fields that are being matches from every revision and then comparing them to those of the target revision, compare each field one by one and stop the match as soon as there is a match failure. This can greatly reduce the match time when matching multiple fields. The impact on match time when matching a single field seems negligible (according to my measurements).

revset: make matching keyword not match summary when matching for description

fileset: add "subrepo" fileset symbol This new fileset symbol returns a list of subrepos whose paths match a given pattern. If the argument has no pattern type set, an exact match is performed. If no argument is passed, return a list of all subrepos.

plan9: finalize documentation for next release

localrepo: do not complain about missing journal files

repair: fix missing import introduced by: b5c0c7d0f83fe56240ce7a919581f1cbff4f5ee5

largefiles: fix cat for largefiles (issue3352) This is a fix to largefiles so that 'hg cat' will work correctly when a largefile is specified. As per discussion on Issue 3352: 1) The file will be printed regardless if it is binary or large. 2) The file is downloaded if it is not readily available (not found in the system cache), so that it can be printed. If the download fails, then we abort.

pure: quiesce build warnings The following patch fixes the following warning when building pure: package init file 'mercurial/pure/__init__.py' not found (or not a regular file)

util.h: replace ntohl/htonl with get/putbe32

scmutil: fix systemrcpath regression introduced in f5dd179bfa4a Changeset f5dd179bfa4a introduced a 'root' path component to look for hgrc files, which is used both as an absolute path and a path relative to the <install-root>. The latter one was broken since 'root' was set to an absolute location and the subsequent os.path.join discarded the <install-root> path prefix.

revlog: fix partial revision() docstring (from d7d64b89a65c)

graphlog: fix --follow FILE and relative paths The situation is complicated because filelog() revset uses a match object in relpath mode while follow() revset interprets the filename as a manifest entry.

graphlog: reduce duplication in --follow code

graphlog: add all log options to glog command glog and log -G should be equivalent.

graphlog: implement --hidden

revert: add support for reverting subrepos without --no-backup and/or --all When a subrepo is reverted but --no-backup is not set, call revert on the subrepo that is being reverted prior to updating it to the revision specified in the parent repo's .hgsubstate file. The --all flag is passed down to the subrepo when it is being reverted. If the --all flag is not set, all files that are modified on the subrepo will be reverted.

revert: add support for reverting subrepos Reverting a subrepo is done by updating it to the revision that is selected on the parent repo .hgsubstate file. * ISSUES/TODO: - reverting added and removed subrepos is not supported yet. - reverting subrepos is only supported if the --no-backup flag is used (this limitation will be removed on another patch). - The behavior of the --all flag has been changed. It now reverts subrepos as well. Note that this may lead to data loss if the user has a dirty subrepo.

merge with stable

commands: move bundle type validation earlier Checking the bundle type late in the command's execution can mean that we do work for a long time before complaining about incorrect user input and aborting. Guess how I discovered this.

changegroupsubset: optimize function lookup in prune

changegroup: optimize gennodelist

revlog: drop some unneeded rev.node calls in revdiff

revlog: avoid an expensive string copy This showed up in a statprof profile of "hg svn rebuildmeta", which is read-intensive on the changelog. This two-line patch improved the performance of that command by 10%.

changegroupsubset: avoid setdefault in inner loop

changegroupsubset: drop repeated len() calls

changegroup: reduce progress overhead

changegroupsubset: renest inner loop The mdata list should generally be shorter than changedfiles

revlog: increase readahead size

revset: avoid demandimport bug Apparently the "import x as xy" doesn't manage to update xy in the current scope's dictionary after load, which causes nodemod.nullrev to do a huge amount of demandload magic in the inner loop.

mq: replace hasattr() with util.safehasattr(), update check-code.py

merge with stable

parsers: use base-16 trie for faster node->rev mapping This greatly speeds up node->rev lookups, with results that are often user-perceptible: for instance, "hg --time log" of the node associated with rev 1000 on a linux-2.6 repo improves from 0.3 seconds to 0.03. I have not found any instances of slowdowns. The new perfnodelookup command in contrib/perf.py demonstrates the speedup more dramatically, since it performs no I/O. For a single lookup, the new code is about 40x faster. These changes also prepare the ground for the possibility of further improving the performance of prefix-based node lookups.

check-code: avoid false-positive on ++

graphlog: cleanup before code move - Avoid revset module aliasing - Mark makefilematcher() private

graphlog: correctly handle calls in subdirectories

context: revert workingctx.ancestors() followfirst option It was introduced by mistake in 352053e6cd8e.

graphlog: fix --follow-first --rev combinations This solves a similar problem than the previous --follow/--rev patch. This time we need changelog.ancestors()/descendants() filtering on first parent. Duplicating the code looked better than introducing keyword arguments. Besides, the ancestors() version was already implemented in follow() revset.

graphlog: fix --follow --rev combinations The previous behaviour of --follow was really a subset of what is really happening in log command: - If --rev is not passed, default to '.:0' - Resolve --rev into a revision list "revs" - Set the starting revision to revs[0] - If revs[1] > revs[0] keep descendants(revs[0]) in revs, otherwise keep ancestors.

graphlog: support changeset identifiers in --branch

graphlog: pass changesets to revset.match() in changelog order Running: $ time hg debugrevspec 'user(mpm)' | wc on Mercurial repository takes 1.0s with a regular version and 1.8s if commands.debugrevspec() is patched to pass revisions to revset.match() from tip to 0. Depending on what we expect from the revset API and caller wisdom, we might want to push this change in revset.match() later.

graphlog: refactor revset() to return revisions When --follow and --rev are passed, --follow actual behaviour depends on the input revision sequence defined by --rev. If --rev is not passed, the default revision sequence depends on the presence of --follow. It means the revision sequence generation is part of log logic and must be wrapped. The issue described above is fixed in following patches.

store: speed up read and write of large fncache files In my tests of an fncache containing 300,000 entries, this improves read time from 567ms to 307, and write time from 1328ms to 533. These numbers aren't so great, since the fncache file is only 17MB in size, but they're an improvement.

perf: time fncache read and write performance

revset: add "matching" keyword This keyword can be used to find revisions that "match" one or more fields of a given set of revisions. A revision matches another if all the selected fields (description, author, branch, date, files, phase, parents, substate, user, summary and/or metadata) match the corresponding values of those fields on the source revision. By default this keyword looks for revisions that whose metadata match (description, author and date) making it ideal to look for duplicate revisions. matching takes 2 arguments (the second being optional): 1.- rev: a revset represeting a _single_ revision (e.g. tip, ., p1(.), etc) 2.- [field(s) to match]: an optional string containing the field or fields (separated by spaces) to match. Valid fields are most regular context fields and some special fields: * regular fields: - description, author, branch, date, files, phase, parents, substate, user. Note that author and user are synonyms. * special fields: summary, metadata. - summary: matches the first line of the description. - metatadata: It is equivalent to matching 'description user date' (i.e. it matches the main metadata fields). Examples: 1.- Look for revisions with the same metadata (author, description and date) as the 11th revision: hg log -r "matching(11)" 2.- Look for revisions with the same description as the 11th revision: hg log -r "matching(11, description)" 3.- Look for revisions with the same 'summary' (i.e. same first line on their description) as the 11th revision: hg log -r "matching(11, summary)" 4.- Look for revisions with the same author as the current revision: hg log -r "matching(., author)" You could use 'user' rather than 'author' to get the same result. 5.- Look for revisions with the same description _AND_ author as the tip of the repository: hg log -r "matching(tip, 'author description')" 6.- Look for revisions touching the same files as the parent of the tip of the repository hg log -r "matching(p1(tip), files)" 7.- Look for revisions whose subrepos are on the same state as the tip of the repository or its parent hg log -r "matching(p1(tip):tip, substate)" 8.- Look for revisions whose author and subrepo states both match those of any of the revisions on the stable branch: hg log -r "matching(branch(stable), 'author substate')"

plan9: add execute permissions to 9diff

transplant: permit merge changesets via --parent This change permits the transplant extension to operate on merge changesets by way of --parent. This is particularly useful for workflows which cherrypick branch merges rather than each commit within a branch.

help: fix indentation on cacert (issue3350) The parser doesn't really handle nesting, so reorder so the nested bit is last.

stream_in: avoid debug calls when debug is disabled

util: create bytecount array just once This avoids tons of gettext calls on workloads that call bytecount a lot.

revset: avoid set duplication in roots()

revset: retrieve a bit less parents in roots()

revset: do not ignore input revisions in roots() 0329d3b12d8e is also partially reverted to use the 'narrow' parameter again and make less changesets parents lookups.

util.h: more Python 2.4 fixes

dispatch: add support for statprof as a profiler This can be selected using the config variable profiling.type or the environment variable HGPROF ("ls" for the default, "stat" for statprof). The only tuneable is the frequency, profiling.freq, which defaults to 1000 Hz. If statprof is not available, a warning is printed.

ui: optionally quiesce ssl verification warnings on python 2.5 Some platforms, notably Plan 9 from Bell Labs are stuck on older releases of Python. Due to restrictions in the platform, it is not possible to backport the SSL library to the existing Python port. This patch permits the UI to quiesce SSL verification warnings by adding a configuration entry named reportoldssl to ui.

scmutil: speed up revrange This improves the performance of "hg log -l1" from 0.21 seconds to 0.07 on a Linux kernel tree. Ideally we could use xrange instead of range on the most common path, and thus avoid a ton of allocation, but xrange doesn't support slice-based indexing.

graft: add --dry-run support (issue3362)

repair: allow giving strip backup a different name So the user can differentiate amend backups from the rest.

encoding: add fast-path for ASCII lowercase

perf: add case collision auditor perf

util.h: unify some common platform tweaks

util.h: move Py_ssize_t bits from mpatch.c

plan9: initial support for plan 9 from bell labs This patch contains support for Plan 9 from Bell Labs. A README is provided in contrib/plan9 which describes the port in greater detail. A new extension is also provided named factotum which permits the factotum(4) authentication agent to provide credentials for HTTP repositories. This extension is also applicable to other POSIX platforms which make use of Plan 9 from User Space (aka plan9ports).

util.h: add a typedef for Py_ssize_t with Python 2.4

log: bypass file scan part of fastpath when no files This avoids loading dirstate parents, looking up p1 rev, and loading p1 manifest to match against an empty matcher.

cmdutil: use context instead of lookup

scmutil: use context instead of lookup

localrepo: lookup now goes through context

context: use rev for changelog lookup Faster when we're doing numeric scanning

context: internalize lookup logic This allows us to avoid doing rev->node->rev lookups on silly instances like "0", which end up caching the whole nodemap.

revlog: allow retrieving contents by revision number

revlog: add hasnode helper method

merge with stable

merge with crew

tags: defer tag validation until repo.tags() is called Before, we were validating all tags for any tag operation, which meant building a (nearly) full node->tag lookup tree for most operations.

parsers: fix a memleak, and add a clearcaches method to the index This change also fixes a nasty memory leak: previously, self->caches was not being freed. The new clearcaches method lets us benchmark with finer granularity, as it lets us separate the cost of loading a revlog index from those of populating and accessing the cache data structures.

hghave: remove symlink test made useless by ac0da5caebec

tests: disable progress estimate in test-debugbuilddag.t If the command takes long enough, including 'estimate' in the format will add some more data to the progress bar output, and make the test fail. See e.g. https://buildd.debian.org/status/fetch.php?pkg=mercurial&arch=kfreebsd-amd64&ver=2.1.2-2&stamp=1333493711

ui: swallow EBADF on stderr ui.write_err already swallows EPIPE and EIO if a write to stderr fails. On Mac OS X at least, a write to a closed file descriptor results in EBADF. Before this patch, hg would exit with status 1 if a write to stderr failed during startup (e.g. while trying to print a warning about not finding an extension): $ ./hg --config extensions.foo= version 2>&-; echo $? 1 With this patch, it correctly swallows stderr and continues to run the command: $ ./hg --config extensions.foo= version 2>&- Mercurial Distributed SCM (version 2.1) ...

mq: use list of already known target files instead of matching object for diff 'hg qnew' passes matching object to 'patch.diff()' to specify target filenames, and it causes 'dirstate.walk()' via 'repo.status()' in 'patch.diff()'. but target files are already known before 'patch.diff()' invocation. to avoid useless 'dirstate.walk()' invocation, this patch uses 'changes' argument to pass already known target files to 'patch.diff()' instead of 'match' argument. 'changes' argument of 'patch.diff()' should have lists for modified, added and removed files separately, so this patch saves status of '.hgsubstate' before commit, and put it into appropriate list in 'changes'.

mq: use exact matching in the second dirstate walking for efficiency of 'qnew' 'hg qnew' with pattern/-I/-X creates matching object with them, and uses it twice for 'dirstate.walk()': via 'repo.status()' and 'repo.commit()'. this may cause full manifest scan in the second 'dirstate.walk()', even though mq already knows complete target filenames at the first 'dirstate.walk()'. this patch creates exact matching object also in this case, and use it at 'repo.commit()' invocation to avoid full manifest scan in the second 'dirstate.walk()'. even though 'inclsubs' is added to 'pats' for original matching object, it is also passed to exact matching object, because subrepositories are deleted from result of 'dirstate.walk()' at the end of it.

tests: avoid test-hup hanging on AIX test-hup hangs on AIX. Under ksh89 on AIX (the default shell), echo Hello; while [ ! -s not-there ]; do true; done produces no output while the loop executes. Replacing 'true' with 'sleep 0' fixes, as does using a less broken shell. ksh93 is fine. Update check-code.py to look for this, and make same change in test-serve.t. In fact test-serve works fine, probably because of additional commands between echo and the loop, but that's a subtlety not easy to test for.

parsers: incrementally parse the revlog index in C We only parse entries in a revlog index file when they are actually needed, and cache them when first requested. This makes a huge difference to performance on large revlogs when accessing the tip revision or performing a handful of numeric lookups (very common cases). For instance, "hg --time tip --template {node}" on a tree with 300,000 revs takes 0.15 before, 0.02 after. Even for revlog-intensive operations (e.g. running "hg log" to completion), the lazy approach is about 1% faster than the eager parse_index2.

mdiff: fix diff header generation for files with spaces (issue3357) diff ---/+++ should end filenames with a TAB when they contain spaces. Current code failed to do so when only the +++ file had spaces. This only happened with git renames from a name without space to one with space.

protocol: Add the stream-preferred capability This makes the client use the uncompressed protocol.

templates/filters: extracting the user portion of an email address Currently, the 'user' filter is using util.shortuser(text) (which clearly doesn't extract only the user portion of an email address, even though the help text says it does). The new 'emailuser' filter uses the new util.emailuser(text) function which, instead, does exactly that. The help text on the 'user' filter has been modified accordingly.

context: make changectx.mutable() benefit from .phase() logic This fixes "hg qimport -r null". Previous versions used to: - Traceback because null revision mutability was not defined - Add an empty -1.diff patch to the series The error message: abort: revision -1 is not mutable is symptomatic of a deeper problem in phase command revision handling. It could be fixed easily in the command itself but I feel a better fix must be done in phase API which raises the issue of phase updates atomicity: aborting in phases.advanceboundary/retractboundary requires a better rollback behaviour to avoid partial changes.

patch: remove useless variable assignment

export: catch exporting empty revsets (issue3353) Additionally add tests for empty revsets and unknown revisions.

merge with stable

test-merge-types: allow different output with existing hgmerge (issue3346) Because hgmerge was meant to be adjusted to personal needs, there may be many remaining copies in people's $PATH.

inotify: catch SignalInterrupt during shutdown (issue3351) When inotify.repowatcher.shutdown() is called, mercurial.error.SignalInterrupt exception is thrown by mercurial.dispatch._runcatch.catchterm(), therefore socketlistener.shutdown() is not called. Catching this allows cleanup action (removing the socket file) to proceed.

merge with stable

clone: always close source repository (issue2491) This is especially needed for cloning from bundles as a temporary bundlerepository is created which needs to be deleted after clone has finished.

merge with stable

tests: make tests work if directory contains special characters With this quoting tests will work e.g. in "/tmp/foo bar/mercurial/".

test-gpg: replace 825565136235 by md5sum check The call to 'hg identify' would have needed '--cwd "$TESTDIR' to make it work anyway, but by using a checksum this test can work outside a repository.

config: discard UTF-8 BOM if found