hghave: allow adding customized features at runtime Before this patch, there is no way to add customized features to `hghave` without changing `hghave` and `hghave.py` themselves. This decreases reusability of `run-tests.py` framework for third party tools, because they may want to examine custom features at runtime (e.g. existence of some external tools). To allow adding customized features at runtime, this patch makes `hghave` import `hghaveaddon` module, only when `hghaveaddon.py` file can be found in directories below: - `TESTDIR` for invocation via `run-tests.py` - `.` for invocation via command line The path to the directory where `hghaveaddon.py` should be placed is added to `sys.path` only while importing `hghaveaddon`, because: - `.` may not be added to `PYTHONPATH` - adding additional path to `sys.path` may change behavior of subsequent `import` for other features `hghave` is terminated with exit code '2' at failure of `import hghaveaddon`, because exit code '2' terminates `run-tests.py` immediately. This is a one of preparations for issue4677.

import-checker.py: exit with code 0 if no error is detected Before this patch, `import-checker.py` exits with non-0 code, if no error is detected. This is unusual as Unix command. This change may be a one of preparations for issue4677, because this can avoid extra explanation about unusual exit code of `import-checker.py` for third party tool developers.

run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR Before this patch, `RUNTESTDIR` is added to `PATH`, but `TESTDIR` isn't. This doesn't cause any problems, if `run-tests.py` runs in `tests` directory of Mercurial source tree. In this case, `RUNTESTDIR` should be equal to `TESTDIR`. On the other hand, if `run-tests.py` runs in `tests` of third party tools, commands in that directory should be executed with explicit `$TESTDIR/` prefix in `*.t` test scripts. This isn't suitable for the policy "drop explicit $TESTDIR from executables" of Mercurial itself (see 4d2b9b304ad0). BTW, 4d2b9b304ad0 describes that "$TESTDIR is added to the path" even though `TESTDIR` isn't added to `PATH` exactly speaking, because `TESTDIR` and `RUNTESTDIR` weren't yet distinguished from each other at that time. This is a one of preparations for issue4677.

run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial Before this patch, there is no way to refer files under `tests` or so of Mercurial source tree, when `run-tests.py` runs in `tests` of third party tools. In this case, `TESTDIR` refers the latter `tests`. This prevents third party tools from using useful tools in Mercurial source tree (e.g. `contrib/check-code.py`). This patch adds `RUNTESTDIR` environment variable to refer `tests` of Mercurial source tree, in which `run-tests.py` now running is placed. For example, tests of third party tools can refer `contrib/check-code.py` in Mercurial source tree as `$RUNTESTDIR/../contrib/check-code.py`. BTW, for similarity with `TESTDIR` referring `test*s*` directory, newly added environment variable isn't named as `RUNTEST*S*DIR`. In addition to it, the corresponded local variable is also named as `runtestdir`. This is a one of preparations for issue4677.

run-tests.py: execute hghave by the path relative to run-tests.py Before this patch, `run-tests.py` executes `hghave` by the path relative to `TESTDIR` (= cwd of `run-tests.py` running). This prevents third party tools for Mercurial from running `run-tests.py`, which is placed in `tests` of Mercurial source tree, in `tests` of own source tree. In such cases, `TESTDIR` refers the latter `tests`, and `hghave` doesn't exist in it. This is a one of preparations for issue4677.

templatekw: make {latesttag} a hybrid list This maintains the previous behavior of expanding {latesttag} to a string containing all of the tags, joined by ':'. But now it also allows list type operations. I'm unsure if the plural handling is correct (i.e. it seems like it is usually "{foos % '{foo}'}"), but I guess we are stuck with this because the singular form previously existed.

templatekw: allow the caller of showlist() to specify the join() separator The keyword {latesttag} currently manually joins the list of tags using ':', which prevents a transparent switch over to a hybrid list.

archive: use {changessincelatesttag} to build the metadata file This isolates the current magic when dealing with wdir() and only().

templatekw: introduce the changessincelatesttag keyword Archive is putting a value with the same name in the metadata file, to count all of the changes not covered by the latest tag, instead of just along the longest path. It seems that this would be useful to have on the command line as well. It might be nice for the name to start with 'latesttag' so that it is grouped with the other tag keywords, but I can't think of a better name. The initial version of this counted a clean wdir() and '.' as the same value, and a dirty wdir() as the same value after it is committed. Yuya objected on the grounds of consistency [1]. Since revsets can be used to conditionally select a dirty wdir() or '.' when clean, I can build the version string I need and will defer to him on this. [1] https://www.selenic.com/pipermail/mercurial-devel/2015-June/071588.html

help: support 'hg help template.somekeyword' Previously the output was simply 'abort: help section not found'.

test-convert-git: use a relative gitmodule url The absolute URL was causing this error with 1.9.5 on Windows, which had a cascading effect: @@ -466,22 +466,24 @@ > url = $TESTTMP/git-repo5 > EOF $ git commit -a -m "weird white space submodule" - [master *] weird white space submodule (glob) - Author: nottest <test@example.org> - 1 file changed, 3 insertions(+) + fatal: bad config file line 6 in $TESTTMP/git-repo6/.gitmodules + [128] $ cd .. $ hg convert git-repo6 hg-repo6 initializing destination hg-repo6 repository scanning source... For reasons unknown, there is still this delta on Windows: @@ -490,7 +490,6 @@ $ git commit -q -m "missing .gitmodules" $ cd .. $ hg convert git-repo6 hg-repo6 --traceback - fatal: Path '.gitmodules' does not exist in '*' (glob) initializing destination hg-repo6 repository scanning source... sorting...

test-convert-git: stablize for git 1.7.7.6 The output has apparently changed slightly since this version. Since they are just commits without any obvious importance to the test, and I can't figure out how to glob them away, silence them. Sample diffs were like this: @@ -468,7 +468,7 @@ $ git commit -a -m "weird white space submodule" [master *] weird white space submodule (glob) Author: nottest <test@example.org> - 1 file changed, 3 insertions(+) + 1 files changed, 3 insertions(+), 0 deletions(-) $ cd .. $ hg convert git-repo6 hg-repo6 initializing destination hg-repo6 repository

hgweb: also monitor change to bookmarks This makes changes to bookmarks visible to hgweb through the official way. There is no change to tests because there is currently another hack in place to ensure the same behavior.

hgweb: also refresh the repo on changes to the obsstore Before this change, hgweb could miss update to the obsolescence markers store if that was the only change between two commands.

hgweb: use an extensible list of files to check for refresh The refresh feature was explicitly testing if '00changelog.i' and 'phaseroots' changed. This is overlooking other important information like bookmarks and obsstore (bookmark have their own hack to work around it). We move to a more extensible system with a list of files of interest that will be used to build the repo state. The system should probably move into a more central place so that the command server and other systems are able to use it. Extension writers will also be able to add entries to ensure that changes to extension data are properly detected. Also the current key (mtime, size) is notably weak for bookmarks and phases whose files can easily change content without effect on their size. Still, this patch seems like a valuable minimal first step.

hgweb: drop the default argument for get_stat This default argument is used twice and is making things confusing. Making it explicit helps to clarify coming changesets

revset: prefetch method in "parents" As already demonstrated, saving attribute lookup gains us some minor but noticeable performance improvements. revset #0: parents(all()) before) 0.024169 after ) 0.022756 94%

convert: handle deleted files when converting from Perforce (issue4743)

forget: add a note to the command help about remove

shelve: only keep the latest N shelve backups This will keep the backup directory from growing indefinitely. The number of backups to keep can be set using the shelve.maxbackups config option (defaults to 10 backups).

shelve: always backup shelves instead of deleting them Instead of being deleted, shelve files are now moved into the .hg/shelve-backup directory. This is designed similarly to how strip saves backups into .ht/strip-backup. The goal is to prevent data loss especially when using unshelve. There are cases in which a user can complete an unshelve but lose some of the data that was shelved by, for example, resolving merge conflicts incorrectly. Storing backups will allow the user to recover the data that was shelved, at the expense of using more disk space over time.

wireproto: remove a debug print This looks like someone forgot something here.

amend: move obsmarkers creation in the "new changeset" conditional We already check if we created a new changesets right above this piece of code, so we can just drop the condition and indent the markers creation.

amend: move createmarkers evaluation earlier The value is used at multiple points in the function. Retrieving the value in the middle of the transaction scope gives the false impression that it has a single user. We move it at the start of the function to clarify this.

wireproto: correctly escape batched args and responses (issue4739) This issue appears to be as old as wireproto batching itself: I can reproduce the failure as far back as 08ef6b5f3715 trivially by rebasing the test changes in this patch, which was back in the 1.9 era. I didn't test before that change, because prior to that the testfile has a different name and I'm lazy. Note that the test thought it was checking this case, but it actually wasn't: it put a literal ; in the arg and response for its greet command, but the mangle/unmangle step defined in the test meant that instead of "Fo, =;o" going over the wire, "Gp-!><p" went instead, which doesn't contain any special characters (those being [.=;]) and thus not exercising the escaping. The test has been updated to use pre-unmangled special characters, so the request is now "Fo+<:o", which mangles to "Gp,=;p". I have confirmed that the test fails without the adjustment to the escaping rules in wireproto.py. No existing clients of RPC batching were depending on the old behavior in any way. The only *actual* users of batchable RPCs in core were: 1) largefiles, wherein it batches up many statlfile calls. It sends hexlified hashes over the wire and gets a 0, 1, or 2 back as a response. No risk of special characters. 2) setdiscovery, which was using heads() and known(), both of which communicate via hexlified nodes. Again, no risk of special characters. Since the escaping functionality has been completely broken since it was introduced, we know that it has no users. As such, we can change the escaping mechanism without having to worry about backwards compatibility issues. For the curious, this was detected by chance: it happens that the lz4-compressed text of a test file for remotefilelog compressed to something containing a ;, which then caused the failure when I moved remotefilelog to using batching for file content fetching.

merge with stable

revset: port extra() to support keyword arguments This is an example to show how keyword arguments are processed.

revset: add function to build dict of positional and keyword arguments Keyword arguments will be convenient for functions that will take more than one optional or boolean flags. For example, file(pattern[, subrepos=false]) subrepo([[pattern], status]) Because I don't think all functions should accept key=value syntax, getkwargs() does not support variadic functions such as 'ancestor(*changeset)'. The core logic is placed in the parser module because keyword arguments will be more useful in the templater, where functions take more options. Test cases will be added by the next patch.

revset: add parsing rule for key=value pair It will be used as an keyword argument. Note that our "=" operator is left-associative. In general, the assignment operator is right-associative, but we don't care because it isn't allowed to chain "=" operations.

import-checker: establish modern import convention We introduce a new convention for declaring imports and enforce it via the import checker script. The new convention is only active when absolute imports are used, which is currently nowhere. Keying off "from __future__ import absolute_import" to engage the new import convention seems like the easiest solution. It is also beneficial for Mercurial to use this mode because it means less work and ambiguity for the importer and potentially better performance due to fewer stat() system calls because the importer won't look for modules in relative paths unless explicitly asked. Once all files are converted to use absolute import, we can refactor this code to again only have a single import convention and we can require use of absolute import in the style checker. The rules for the new convention are documented in the docstring of the added function. Tests have been added to test-module-imports.t. Some tests are sensitive to newlines and source column position, which makes docstring testing difficult and/or impossible.

import-checker: establish new function for verifying import conventions A future patch will formalize the modern import convention. In preparation for that, introduce a new wrapper function that will invoke the proper function.

import-checker: resolve relative imports "from . import X" will produce an ImportFrom ast node with .module = None. This resulted in a run-time error from attempting to concatenate None with a str. Another problem with relative imports is that the prefix may be dynamic based on the "level" attribute of the import. e.g. "from ." has level 1 and "from .." has level 2. We teach the "fromlocal" function how to cope with relative imports. Where appropriate, the consumer passes in the level so relative module names may be resolved properly.

templatekw: use a list of tags in getlatesttags() instead of joining them This will be used in the next patch. It also points out that the documentation for '{latesttag}' is not quite accurate, since it says "most recent global tag" (singular). I assume it is too radical of a change to convert it to a list of strings. At least ':' is currently a reserved character in tag names.

convert: improve support for unusual .gitmodules Previously convert would throw an exception if it encountered a git commit with a .gitmodules file that was malformed (i.e. was missing, but had submodule files, or was malformed). Instead of breaking the convert entirely, let's print error messages and move on.

convert: handle .gitmodules with non-tab whitespaces The old implementation assumed .gitmodules file lines always began with tabs. It can be any whitespace, so lets trim the lines appropriately.

convert: fix bug with converting the same commit twice Convert had a bug where it relied on repo.tip() to be the newly committed commit. This was not the case if the commit already existed in the repository (since repo.commitctx() did nothing, the tip() referenced some random other commit and the revmap got corrupted). This fixes it by using the node returned by repo.commitctx().

templater: remove workaround for escaped quoted string in quoted template This patch backs out 554d6fcc3c84 which should no longer be needed. The test for '{\"invalid\"}' is removed because the parser is permissive for \"...\" literal.

merge with stable

amend: stop updating the bookmarks twice There was code to move the bookmarks around both in the 'cmdutil' help and in the main 'commit' function. We kill the 'commit' version as it is performed outside the transaction. The debug note is moved into cmdutil.

amend: collaborate with the transaction when moving bookmarks We have code moving bookmarks from the old changeset to the new one within the transaction scope. Yet this code was still writing to disk instead of handing the change to the transaction. This changeset fixes this.

sshserver: drop ancient do_{lock,unlock,addchangegroup} methods These were marked as deprecated and dangerous way back in e8c4f3d3df8c, which was first included in Mercurial 0.9.1. While it's possible that clients from that long ago are still around somewhere, they're risky for servers in that they want to lock the repo, and then might leave it locked if they died before finishing their transaction. Given that it's been 9 years, let's go ahead and cut this last lingering tie with a basically-untested protocol.

wireproto: add config knob for http header length limit Well-behaved Mercurial clients will respect the httpheader capability by not sending http headers longer than the given limit in bytes. The limit is currently hard-coded at 1024 bytes, a safe value for any web server. Since parsing headers is a notable factor in web server performance, tuning header size can nontrivially improve performance for request-heavy operations (eg. obsolete marker negotiation). Exposing the maximum header length limit as a configuration setting is a simple way to enable such tuning.

archive: fix changesincelatesttag with wdir()

revset: fix a crash in parents() when 'wdir()' is in the set The crash was "TypeError: expected string or Unicode object, NoneType found" down in revlog.parentrevs(). This fixes heads() too (which is where I found it.)

workingctx: don't report the tags for its parents This fixes the bad distance calculation for '{latesttagdistance}' mentioned in the previous patch.

hgweb: don't try to show repo on repo-not-found page in monoblue

hgweb: fix help pages title in gitweb and monoblue Help pages in gitweb and monoblue were setting a completely inappropriate title: "Branches". Let's have a proper title (copy-pasted from paper style).

hgweb: don't show summary link as active on error pages in monoblue These two error pages in monoblue think they are the summary page, when they clearly aren't.

identify: build the tag list directly instead of using wctx.tags() The current implementation of workingctx.tags() returns the tags of the parents. This causes the calculation of {lastesttagdistance} from wdir() to be wrong. The value when updated to a tag is 0, but updated to the tag's child is 2, the child of that 3, and so on. This prepares for workingctx.tags() to not report the parent tags.

identify: avoid a crash when given '-r wdir()' The crash was 'NoneType is not subscriptable' in hexfunc(ctx.node()), because the node for wdir() is None. This can be avoided simply by detecting 'wdir()' and taking the existing path for no given revision.

tests: clean up duplicated output in test-subrepo-recursion progress We have fixed a bug where two progress instance were created and competed with each other (in 95f490136e75). But we did not updated the non-hardlink section of 'test-subrepo-recursion.t'. This patch fixes it.

Added signature for changeset 540cd0ddac49

Added tag 3.4.2 for changeset 540cd0ddac49

transplant: only pull the transplanted revision (issue4692) For some reason, transplant was pulling all remote revisions when transplanting from a remote repository (unless --branch was specified).

transplant: update test to use hash for remote transplant Transplant is apparently allowing using revision numbers when transplanting through http. I call this nonsense and update the test to use an explicit hash. This "feature" will break in the next change fixing another bug.

changegroup: properly compute common base in changeggroupsubset (issue4736) The computation of roots was buggy, any ancestor of a bundled merge which was also a descendant of the parents of a bundled revision were included as part of the bundle. We fix it and add a test for strip (which revealed the problem). Check the test for a practical usecase.

templater: parse \"...\" as string for 2.9.2-3.4 compatibility (issue4733) As of Mercurial 3.4, there were several syntax rules to process nested template strings. Unfortunately, they were inconsistent and conflicted each other. a. buildmap() rule - template string is _parsed_ as string, and parsed as template - <\"> is not allowed in nested template: {xs % "{f(\"{x}\")}"} -> parse error - template escaping <\{> is handled consistently: {xs % "\{x}"} -> escaped b. _evalifliteral() rule - template string is _interpreted_ as string, and parsed as template in crafted environment to avoid double processing of escape sequences - <\"> is allowed in nested template: {if(x, "{f(\"{x}\")}")} - <\{> and escape sequences in string literal in nested template are not handled well c. pad() rule - template string is first interpreted as string, and parsed as template, which means escape sequences are processed twice - <\"> is allowed in nested template: {pad("{xs % \"{x}\"}', 10)} Because of the issue of template escaping, issue4714, 7298da81f5a9 (in stable) unified the rule (b) to (a). Then, 576d6c74784b (in default) unified the rule (c) to (b) = (a). But they disabled the following syntax that was somewhat considered valid. {if(rev, "{if(rev, \"{rev}\")}")} {pad("{files % \"{file}\"}", 10)} So, this patch introduces \"...\" literal to work around the escaped-quoted nested template strings. Because this parsing rule exists only for the backward compatibility, it is designed to copy the behavior of old _evalifliteral() as possible. Future patches will introduce a better parsing rule similar to a command substitution of POSIX shells or a string interpolation of Ruby, where extra escapes won't be necessary at all. {pad("{files % "{file}"}", 10)} ~~~~~~~~~~~~~~~~~~ parsed as a template, not as a string Because <\> character wasn't allowed in a template fragment, this patch won't introduce more breakages. But the syntax of nested templates are interpreted differently by people, there might be unknown issues. So if we want, we could instead remove db7463aa080f, 890845af1ac2 and 7298da81f5a9 from the stable branch as the bug fixed by these patches existed for longer periods. 554d6fcc3c8, "strip single backslash before quotation mark in quoted template", should be superseded by this patch. I'll remove it later.

merge with stable

demandimport: support importing builtins for Python 3 __builtin__ was renamed to builtins in Python 3. If importing __builtin__ fails, fall back to importing builtins in order to support Python 3.

demandimport: alias __builtin__ as builtins Python 3 renamed the __builtin__ module to builtins. In preparation for supporting Python 3, alias the imported module as "builtins."

bufferedinputpipe: remove N^2 computation of buffer length (issue4735) The assumption that dynamically computing the length of the buffer was N^2, but negligible because fast was False. So we drop the dynamic computation and manually keep track of the buffer length.

bufferedinputpipe: remove an outdate comment This comment is the remains of a intermediate implementation using self._buffer += data This implementation never made it to the repository and we can safely drop the comment.

statichttprepo: kill off sopener

obsolete: replace references to 'sopener' with 'svfs'

exchange: replace references to 'sopener' with 'svfs'

localrepo: kill off sopener (API) sopener is deprecated since 7034365089bf (Mercurial 2.3). It's annoying for extension authors to have to deal with both. Let's just kill it off.

share: replace reference to 'sopener' with 'svfs' sopener is deprecated. It's annoying for extension authors to have to deal with both. Let's just kill it off.

archive: don't assume '.' is being archived for changessincelatesttag Hardcoding '.' is wrong, and yielded strange results when archiving old revisions. For example, when archiving the cset that adds the signature to 3.4 (c48850339988), the resulting value was previously 51 (the number of commits on stable between 3.4 and today), even though it was a direct descendant of a tag, with a {latesttagdistance} of 2. This still includes all other _ancestor_ paths not included in {latesttag}. Note that archiving wdir() currently blows up several lines above this when building the 'base' variable. Since wdir() isn't documented, ignore that it needs work to handle wdir() here for now.

hgweb: use css for stripey background in coal Since "b8ecc3830c89 or 25dae11bb044::c229a5e7511e" paper style used css for stripes in background for browsing files, for listing branches/tags/bookmarks, and so on. Since coal borrows many paper templates (e.g. shortlogentry.tmpl), it actually tried to do the same, but it didn't have the needed css classes. You can compare https://selenic.com/hg?style=coal with https://selenic.com/hg?style=paper and see how log view in coal style has plain white background, unlike the one in paper style. This wasn't intended. Let's copy css classes directly from style-paper.css and remove parity classes from elements that don't need them anymore. This makes plain white background have stripes again and makes coal/map even more similar to paper/map (which can ease porting changes or %including paper/map in future).

templatekw: correct typo in activebookmark documentation

merge with stable

check-code: detect legacy exception syntax We just rewrote all files to use modern exception syntax. Ban the old form. This will detect the "except type, instance" and "except (type1, type2), instance" forms.

global: mass rewrite to use modern exception syntax Python 2.6 introduced the "except type as instance" syntax, replacing the "except type, instance" syntax that came before. Python 3 dropped support for the latter syntax. Since we no longer support Python 2.4 or 2.5, we have no need to continue supporting the "except type, instance". This patch mass rewrites the exception syntax to be Python 2.6+ and Python 3 compatible. This patch was produced by running `2to3 -f except -w -n .`.

check-code: detect legacy octal syntax Now that we have mass rewriting all files to use the modern octal syntax, detect and ban the legacy syntax, which is no longer supported in Python 3.

global: mass rewrite to use modern octal syntax Python 2.6 introduced a new octal syntax: "0oXXX", replacing "0XXX". The old syntax is not recognized in Python 3 and will result in a parse error. Mass rewrite all instances of the old octal syntax to the new syntax. This patch was generated by `2to3 -f numliterals -w -n .` and the diff was selectively recorded to exclude changes to "<N>l" syntax conversion, which will be handled separately.

revert: change the direction of revert -i After the discussion on the list about hg revert -i, it seems like we are satisfied with what we called proposition 2. It shows the changes to revert in the same direction as hg diff. This patch makes it the default option. It changes all the + in - and vice versa in the tests for revert -i.

revert: change a test to make the change of direction of revert -i easier Currently we are handling editing of newly-added files with the interactive interface. We are not handling editing of deleted files. In the test for revert, we were editing a newly-added file. Since we want to change the direction of revert -i, this editing of a newly-added file will become editing of a deleted file. Since we don't support that, this patch changes the test to make the rest of the series cleaner.

parser: update documentation about tokenizer and elements

parser: accept iterator of tokens instead of tokenizer function and program This can simplify the interface of parse() function. Our tokenizer tends to have optional arguments other than the message to be parsed. Before this patch, the "lookup" argument existed only for the revset, and the templater had to pack [program, start, end] to be passed to its tokenizer.

verify: print hint to run debugrebuildfncache Corrupt fncache is now a recoverable operation. Inform the user how to recover from this warning.

repair: add functionality to rebuild fncache Currently, there is no way to recover from a missing or corrupt fncache file in place (a clone is required). For certain use cases such as servers and with large repositories, an in-place repair may be desirable. This patch adds functionality for in-place repair of the fncache. The `hg debugrebuildfncache` command is introduced. It ensures the fncache is up to date by reconstructing the fncache from all seen files encountered during a brute force traversal of the repository's entire history. The command will add missing entries and will prune excess ones. Currently, the command no-ops unless the repository has the fncache requirement. The command could later grow the ability to "upgrade" an existing repository to be fncache enabled, if desired. When testing this patch on a local clone of the Firefox repository, it removed a bunch of entries. Investigation revealed that removed entries belonged to empty (0 byte size) .i filelogs. The functionality for pruning fncache of stripped revlogs was introduced in f49d60fa40a5, so the presence of these entries likely predates this feature.

import: cross-reference patch.fuzz option from `hg help import`

import: cross-reference ui.patch option from `hg help import` This is a fairly obscure patch option. It seems reasonable to expose it a bit more from the help text for `hg import`.

doc: document the ui.patch option This option has been undocumented since e56c7e05c7e6 (July 2007).

revset: rework 'filteredset.last' 'isascending' and 'isdescending' are methods, not attributes. This led 'last()' to misbehave on some non-ascending filtered sets.

revset: improves time complexity of 'roots(xxx)' The canonical way of doing 'roots(X)' is 'X - children(X)'. This is what the implementation used to be. However, computing children is expensive because it is unbounded. Any changesets in the repository may be a children of '0' so you have to look at all changesets in the repository to compute children(0). Moreover the current revsets implementation for children is not lazy, leading to bad performance when fetching the first result. There is a more restricted algorithm to compute roots: roots(X) = [r for r in X if not parents(r) & X] This achieve the same result while only looking for parent/children relation in the X set itself, making the algorithm 'O(len(X))' membership operation. Another advantages is that it turns the check into a simple filter, preserving all laziness property of the underlying revsets. The speed is very significant and some laziness is restored. -) revset without 'roots(...)' to compare to base line 0) before this change 1) after this change revset #0: roots((tip~100::) - (tip~100::tip)) plain min last -) 0.001082 0.000993 0.000790 0) 0.001366 0.001385 0.001339 1) 0.001257 92% 0.001028 74% 0.000821 61% revset #1: roots((0::) - (0::tip)) plain min last -) 0.134551 0.144682 0.068453 0) 0.161822 0.171786 0.157683 1) 0.137583 85% 0.146204 85% 0.070012 44% revset #2: roots(tip~100:) plain min first last -) 0.000219 0.000225 0.000231 0.000229 0) 0.000513 0.000529 0.000507 0.000539 1) 0.000463 90% 0.000269 50% 0.000267 52% 0.000463 85% revset #3: roots(:42) plain min first last -) 0.000119 0.000146 0.000146 0.000146 0) 0.000231 0.000254 0.000253 0.000260 1) 0.000216 93% 0.000186 73% 0.000184 72% 0.000244 93% revset #4: roots(not public()) plain min first -) 0.000478 0.000502 0.000504 0) 0.000611 0.000639 0.000634 1) 0.000604 0.000560 87% 0.000558 revset #5: roots((0:tip)::) plain min max first last -) 0.057795 0.004905 0.058260 0.004908 0.038812 0) 0.132845 0.118931 0.130306 0.114280 0.127742 1) 0.111659 84% 0.005023 4% 0.111658 85% 0.005022 4% 0.092490 72% revset #6: roots(0::tip) plain min max first last -) 0.032971 0.033947 0.033460 0.032350 0.033125 0) 0.083671 0.081953 0.084074 0.080364 0.086069 1) 0.074720 89% 0.035547 43% 0.077025 91% 0.033729 41% 0.083197 revset #7: 42:68 and roots(42:tip) plain min max first last -) 0.006827 0.000251 0.006830 0.000254 0.006771 0) 0.000337 0.000353 0.000366 0.000350 0.000366 1) 0.000318 94% 0.000297 84% 0.000353 0.000293 83% 0.000351 revset #8: roots(0:tip) plain min max first last -) 0.002119 0.000145 0.000147 0.000147 0.000147 0) 0.047441 0.040660 0.045662 0.040284 0.043435 1) 0.038057 80% 0.000187 0% 0.034919 76% 0.000186 0% 0.035097 80% revset #0: roots(:42 + tip~42:) plain min max first last sort -) 0.000321 0.000317 0.000319 0.000308 0.000369 0.000343 0) 0.000772 0.000751 0.000811 0.000750 0.000802 0.000783 1) 0.000632 81% 0.000369 49% 0.000617 76% 0.000358 47% 0.000601 74% 0.000642 81%

revsetbenchmark: do not abort on failure to run a revset Instead of aborting the whole process, we just skip entry for revset that failed to run.

osutil: remove Python 2.4 errno conversion workaround

patch: remove email import workaround for Python 2.4 Python 2.6 provides access to the sub-modules just fine. This workaround is no longer needed since we no longer support Python 2.4.

check-commit: catch both patterns of double empty lines

revsetbenchmarks: ignore empty lines Before this change, empty lines were seen as an entry and the benchmark tried to run benchmark for "".

bundle2: reword debug message for invalid .hgtags data marmoute requested a follow-up to make the verb the first word.

bundle2.getunbundler: rename "header" to "magicstring" This is more consistent with the name used in the bundler class. Thanks goes to Martin von Zweigbergk for pointing this out.

ancestors: prefetch method outside of the loop 15412bba5a68 is yet another example where this is worthwhile when it comes to performance, we blindly do it for all 'lazyancestors' methods.

templater: fix handling of \-escapes in raw string literals The backslash character should start escape sequences no matter if a string is prefixed with 'r'. They are just not interpreted as escape sequences in raw strings. revset.tokenize() handles them correctly, but templater didn't. https://docs.python.org/2/reference/lexical_analysis.html#string-literals

templater: evaluate "query" argument passed to revset() revset() had the same issue as 9452112c8eb0. It crashed by passing non-string expression.

match: let 'path:.' and 'path:' match everything (issue4687) Previously, both queries exited with code 1, printing nothing. The pattern in the latter query is normalized to '.', so it is really the same case.

changelog: document the 'readpending' method I happen to have spent some time understanding this logic, so I'm leaving documentation for the next poor fellow.

revsets: use '&' instead of '.filter' in head More high level operations are more likely to be optimised.

fileset: don't suggest private or undocumented queries Unlike revsets, it looks like all of the filesets are documented, so there's really nothing to test. This is aimed more at parity with revsets and future-proofing.

revset: don't suggest private or undocumented queries I noticed when I mistyped 'matching', that it suggested '_matchfiles' as well. Rather than simply exclude names that start with '_', this excludes anything without a docstring. That way, if it isn't in the help text, it isn't suggested, such as 'wdir()'.

patch: add fuzz config flag (issue4697)

devel-warn: issue a warning for old style revsets We have move to smartset class more than a year ago, we now have the tool to aggressively nudge developer into upgrading their extensions.

devel-warn: move the develwarn function as a method of the ui object We are going to use this feature in more and more place. Having to import scmutil makes it an import cycle hell.

templates: fix missing closing brace in map-cmdline.status Currently the parser does not validate the last closing brace, but future patches will make it less permissive for this kind of syntax errors.

verify: clarify misleading fncache message This is a message about cache corruption, not repository corruption or actually missing files. Fix message and reduce to a warning.

phase: add a pointer to 'hg help phases' in the 'phase' help The command and the general topic have very similar name. The topic is pointing to the command, but the command was not pointing to the topic.

publishing: unconditionally trust publishing flag Unfortunately, there are currently no tests for this behavior because we lack a good way of testing trust.

publishing: use new helper method

publishing: add helper method to localrepo

revset: make use of natively-computed set for 'draft()' and 'secret()' If the computation of a set for each phase (done in C) is available, we use it directly instead of applying a simple filter. This give a massive speed-up in the vast majority of cases. On my mercurial repo with about 15000 out of 40000 draft changesets: revset: draft() plain min first last 0) 0.011201 0.019950 0.009844 0.000074 1) 0.000284 2% 0.000312 1% 0.000314 3% 0.000315 x4.3 Bad performance for "last" come from the handling of the 15000 elements set (memory allocation, filtering hidden changesets (99% of it) etc. compared to applying the filter only on a handfuld of revisions (the first draft changesets being close of tip). This is not seen as an issue since: * Timing is still pretty good and in line with all the other one, * Current user of Vanilla Mercurial will not have 1/3 of their repo draft, This bad effect disappears when phase's set is smaller. (about 200 secrets): revset: secret() plain min first last 0) 0.011181 0.022228 0.010851 0.000452 1) 0.000058 0% 0.000084 0% 0.000087 0% 0.000087 19%

revset: refactor the non-public phase code Code for draft and secret are the same. We'll make it more complex to take advantages of the set recomputed in C, so we first refactor the code to only have one place to update (and make sure all behave properly). We do not refactor the 'public()' code because it does not have a natively computed set.

revset: translate node directly with changelog in 'head' Using 'repo[X]' is much slower because it creates a 'changectx' object and goes though multiple layers of code to do so. It is also error prone if there is tags, bookmarks, branch or other names that could map to a node hash and take precedence (user are wicked). This provides a significant performance boost on repository with a lot of heads. Benchmark result for a repo with 1181 heads. revset: head() plain min last reverse 0) 0.014853 0.014371 0.014350 0.015161 1) 0.001402 9% 0.000975 6% 0.000874 6% 0.001415 9% revset: head() - public() plain min last reverse 0) 0.015121 0.014420 0.014560 0.015028 1) 0.001674 11% 0.001109 7% 0.000980 6% 0.001693 11% revset: draft() and head() plain min last reverse 0) 0.015976 0.014490 0.014214 0.015892 1) 0.002335 14% 0.001018 7% 0.000887 6% 0.002340 14% The speed up is visible even when other more costly revset are in use revset: head() and author("mpm") plain min last reverse 0) 0.105419 0.090046 0.017169 0.108180 1) 0.090721 86% 0.077602 86% 0.003556 20% 0.093324 86%

revset: use a baseset in _notpublic() The '_notpublic()' internal revset was "returning" a set. That was wrong. We now return a 'baseset' as appropriate. This has no effect on performance in most case, because we do the exact same operation than what the combination with a 'fullreposet' was doing. This as a small effect on some operation when combined with other set, because we now apply the filtering in all cases. I think the correctness is worth the impact on some corner cases. The optimizer should take care of these corner cases anyway. revset #0: not public() plain min max first last reverse 0) 0.000465 0.000491 0.000495 0.000500 0.000494 0.000479 1) 0.000484 0.000503 0.000498 0.000505 0.000504 0.000491 revset #1: (tip~1000::) - public() plain min max first last reverse 0) 0.002765 0.001742 0.002767 0.001730 0.002761 0.002782 1) 0.002847 0.001777 0.002776 0.001741 0.002764 0.002858 revset #2: not public() and branch("default") plain min max first last reverse 0) 0.012104 0.011138 0.011189 0.011138 0.011166 0.011578 1) 0.011387 94% 0.011738 105% 0.014220 127% 0.011223 0.011184 0.012077 revset #3: (not public() - obsolete()) plain min max first last reverse 0) 0.000583 0.000556 0.000552 0.000555 0.000552 0.000610 1) 0.000613 105% 0.000559 0.000557 0.000573 0.000558 0.000613 revset #4: head() - public() plain min max first last reverse 0) 0.010869 0.010800 0.011547 0.010843 0.010891 0.010891 1) 0.011031 0.011497 106% 0.011087 0.011100 0.011100 0.011085

contrib: clean up all-revsets.txt file I forgot to cleanup a handful of them when I originally created the file.

hgweb: link to revision by node hash in paper & coal Unlike other styles, paper and coal had only one link to current revision: in the sidebar. Since those links now use symbolic revisions after 3bb6f5f478a7, it's nice to have a link that allows going from /rev/tip to /rev/<tip hash>, for instance. Let's make the node hash in the page header that new link.

hgweb: link to revision by node hash in gitweb & monoblue This allows going from /rev/tip to /rev/<tip hash> with ease.

archive: report the node as "{p1node}+" when archiving a dirty wdir() This is more useful than reporting all 'f's, allowing the archive to be diffed against a specific revision to see what changed.

phase: remove a 'for x in "foo bar".split()' idiom in phasecache.replace I find this idiom fairly horrible.

phase: document the replace method This is a minor documentation update to answer a co-worker question.

revset: ensure we have loaded phases data in '_notpublic()' If we are the very first rev access (or if the phase cache just got invalidated) the phasesets will be None even if we support the native computation. So we explicitly trigger a computation if needed. This was not an issue before because requesting any phase information would have triggered such computation.

phase: rename getphaserevs to loadphaserevs This function is: - already loading the data in place, - used once in the code. So we drop the return value and change the name to make this obvious. We keep the function public because we'll have to use it in revset.

test-subrepo-recursion: more aggressively regex 'unzip -l' output for 10.10 The output on 10.10 looks like this: Length Date Time Name -------- ---- ---- ---- 172 01-01-80 00:00 .hg_archival.txt 10 01-01-80 00:00 .hgsub 45 01-01-80 00:00 .hgsubstate 3 01-01-80 00:00 x.txt 10 01-01-80 00:00 foo/.hgsub 45 01-01-80 00:00 foo/.hgsubstate 9 01-01-80 00:00 foo/y.txt 9 01-01-80 00:00 foo/bar/z.txt -------- ------- 303 8 files (2 digit year, shorter -- separators and closer columns). We don't care about any of that, so ignore it.

contrib: introduce an all-revsets.txt file This file should gather all revsets ever thought interesting by anyone. That way one can check the impact of a change when touching something revset-ish. See inline comments for details. This file have been refilled with all the entry I could automatically find from changeset descriptions. I assume we missed some not using 'revsetbenchmarks.py' output.

contrib: rename revsetbenchmarks.txt to 'base-revsets.txt' We rename the file and document its purpose. We'll be introducing another file gathering revsets useful for benchmark of the predicate themsleves in a coming changesets.

revsetbenchmarks: add main documention for the script This allow us to document the fact we can use comment in the file listing revsets.

hgweb: don't dereference symbolic revision in paper & coal style (issue2296) Let's make paper (and coal, since it borrows so much from paper) templates use symbolic revision in navigation links. The majority of links (log, filelog, annotate, etc) still use node hashes. Some pages don't have permanent links to current node hash (so it's not very easy to go from /rev/tip to /rev/<tip hash>), this will be addressed in future patches.

hgweb: don't dereference symbolic revision in gitweb style Let's make gitweb templates use symbolic revision in navigation links. The majority of links (log, filelog, annotate, etc) still use node hashes. Some pages don't have permanent links to current node hash (so it's not very easy to go from /rev/tip to /rev/<tip hash>), this will be addressed in future patches.