help: rewrite template examples to use integer literals where appropriate

templater: update error message of invalid number passed to word() function "word(3, desc)" is valid now.

templater: tokenize decimal integer literal (issue4638) (BC) Before this patch, we had to quote integer literals to pass to template functions. It was error-prone, so we should allow "word(0, x)" syntax. Currently only decimal integers are allowed. It's easy to support 0x, 0b and 0 prefixes, but I don't think they are useful. This patch assumes that template keywords and names defined in map files do not start with digits, except for positional variables seen in the schemes extension.

templater: switch methods table on compileexp() of func args and inner expr The next patch will introduce integer literals, but the schemes extension expects that '{1}', '{2}', ... are interpreted as keywords. This patch allows us to process '{foo(1)}' as 'func(integer)', whereas '{1}' as 'symbol'.

templatefilters: don't stringify None into "None" A few template keywords can in fact return None, such as {bisect}. In some contexts, these get stringified into None instead of "". This is leaking Python details into the UI.

hgweb: clearly outline <tr> block in gitweb and spartan styles It's easy to miss the closing </tr> tag when it's not on its own line. Some neighboring blocks also already use this clearer style. (In 03f692eee31d I didn't even think grepping for this, but this patch should now cover every occurrence).

rebase: use dirstateguard instead of dirstate.invalidate Before this patch, "rebase.concludenode()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook). After this patch, the changed dirstate becomes visible to external "precommit" hooks during "hg rebase" in "test-largefiles-misc.t", because "dirstateguard()" writes it out. But this content isn't yet correct, because: - "normal3" should be marked as "A"(dded) at committing It is newly added in the changeset being rebased. - but it is marked as "M"(odified) The result of "repo.setparents()" after "dirstateguard()" isn't yet written out before "precommit". So, merging is still in progress for "hg status" in it. This causes marking the file newly added on "other" branch as "A". This will be fixed by subsequent patch.

mq: use dirstateguard instead of dirstate.invalidate (qrefresh) Before this patch, "mq.queue.refresh()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

mq: use dirstateguard instead of dirstate.invalidate (qpush) Before this patch, "mq.queue.apply()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during afailure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

tryimportone: use dirstateguard instead of beginparentchange/endparentchange To fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook), a subsequent patch makes "localrepository.commit()" invoke "dirstate.write()" in it. This change will make "beginparentchange()" and "endparentchange()" on dirstate in "cmdutil.tryimportone()" meaningless, because: - "dirstate.write()" writes changed data into ".hg/dirstate", but - aborting between "beginparentchange()" and "endparentchange()" doesn't cause any restoring ".hg/dirstate" it just discards changes in memory. This patch uses "dirstateguard" instead of "beginparentchange()" and "endparentchange()" in "cmdutil.tryimportone()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch uses "lockmod.release(dsguard)" instead of "dsguard.release()", because processing may be aborted before assignment to "dsguard" , and the "if dsguard" examination for safety is redundant.

import: use dirstateguard instead of dirstate.invalidate Before this patch, "commands.import()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless, too. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

amend: use dirstateguard instead of dirstate.invalidate Before this patch, "cmdutil.amend()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

localrepo: use changelog.hasnode instead of self.__contains__ Before this patch, releasing the store lock implies the actions below, when the transaction is aborted: 1. "commithook()" scheduled in "localrepository.commit()" is invoked 2. "changectx.__init__()" is invoked via "self.__contains__()" 3. specified ID is examined against "repo.dirstate.p1()" 4. validation function is invoked in "dirstate.p1()" In subsequent patches, "dirstate.invalidate()" invocations for discarding changes are replaced with "dirstateguard", but discarding changes by "dirstateguard" is executed after releasing the store lock: resources are acquired in "wlock => dirstateguard => store lock" order, and are released in reverse order. This may cause that "dirstate.p1()" still refers to the changeset to be rolled-back at (4) above: pushing multiple patches by "hg qpush" is a typical case. When releasing the store lock, such changesets are: - not contained in "repo.changelog", if it is reloaded from ".hg/00changelog.i", as that file was already truncated by "transaction.abort()" - still contained in it, otherwise (this "dirty read" problem is discussed in "Transaction Plan" http://mercurial.selenic.com/wiki/TransactionPlan) Validation function shows "unknown working parent" warning in the former case, but reloading "repo.changelog" depends on the timestamp of ".hg/00changelog.i". This causes occasional test failures. In the case of scheduled "commithook()", it just wants to examine whether "node ID" of committed changeset is still valid or not. Other examinations implied in "changectx.__init__()" are meaningless. To avoid showing the "unknown working parent" warning irregularly, this patch uses "changelog.hasnode()" instead of "node in self" to examine existence of committed changeset.

cmdutil: add class to restore dirstate during unexpected failure Before this patch, after "dirstate.write()" execution, there was no way to restore dirstate to the original status before "dirstate.write()". In some code paths, "dirstate.invalidate()" is used as a kind of "restore .hg/dirstate to the original status", but it just avoids writing changes in memory out, and doesn't actually restore the ".hg/dirstate" file. To fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hooks), "dirstate.write()" should be invoked before invocation of external processes. But at the same time, ".hg/dirstate" should be restored to its content before "dirstate.write()" during an unexpected failure in some cases. This patch adds the class "dirstateguard" to easily restore ".hg/dirstate" during unexpected failures. Typical usecase of it is: # (1) build dirstate up .... # (2) write dirstate out, and backup ".hg/dirstate" dsguard = dirstateguard(repo, 'scopename') try: # (3) execute somethig to do: # this may imply making some additional changes on dirstate .... # (4) unlink backed-up dirstate file at the end of dsguard scope dsguard.close() finally: # (5) if execution is aborted before "dsguard.close()", # ".hg/dirstate" is restored from the backup dsguard.release() For this kind of issue, an "extending transaction" approach (in https://titanpad.com/mercurial32-sprint) seems to not be suitable, because: - transaction nesting occurs in some cases (e.g. "shelve => rebase"), and - "dirstate" may be already modified since the beginning of OUTER transaction scope, then - dirstate should be backed up into the file other than "dirstate.journal" at the beginning of INNER transaction scope, but - such alternative backup files are useless for transaction itself, and increases complication of its implementation "transaction" and "dirstateguard" differ from each other also in "what it should do for .hg/dirstate" in cases other than success. ============== ======= ======== ============= type success fail "hg rollback" ============== ======= ======== ============= transaction keep keep restore dirstateguard keep restore (not implied) ============== ======= ======== ============= Some collaboration between transaction and dirstate will probably be introduced in the future. But this layer is needed in all cases.

merge with stable

wix: add new json templates folder to MSI installers

templater: rename parsestring() to unquotestring() (API) Since db7463aa080f, it doesn't parse string escapes.

templater: remove noop calls of parsestring(s, quoted=False) (API) Since db7463aa080f, parsestring(s, quoted=False) just returns s.

bookmarks: simplify iscurrent to isactivewdirparent (API) Previously this function accepted two optional parameters that were unused by any callers and complicated the function. Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

tests: add a new commit to test-commandline-template This commit modifies, adds, renames, removes files all at once. It will be mostly interesting for an upcoming test relating to the status log template.

run-tests: spread and document the content of time tuple The spreading will make future modification clearer. The documentation improve code readability.

run-tests: when building json, get time data in the same order as elsewhere The json code was changing the order of the time tuple for unclear reasons. We now use the same order as everywhere else.

run-tests: stop explicit expansion of time data We are about to record more complex time-related data, which will require repeated changes of every loop touching times data. That will also extend such lines to a point where things become too long. Instead, we iterate on each entry and expand values in the loops. We keep intermediate variables in most cases to preserve readability. The loop producing json data does a strange inversion for no obvious reason. We preserved that for now and will fix it in another changeset.

run-tests: refactor json entry creation logic We are about to add more timing related information to each entry. Having a single place where the json entry is created is going to be much simpler.

run-tests: when building json, use result.failures instead of result.faildata It is unclear to me why 'faildata' was used. Let's use the same kind of attribute as for the other groups.

run-test: add a test for json output when -i is used The -i can apparently confused some internal data structure, we want to make sure we do not regress.

changegroup: drop _changelog and _manifest properties We already have a _repo property on the packer, and we only access the changelog and manifest revlog in one place, so it's just as easy to get them from self._repo.

changegroup: document the cases where reordering complicates linkrevs

changegroup: extract condition for linkrev fastpath The condition for taking the fastpath (or not) is used in two places. By extracting it, we also provide a place to document what it's about.

lazymanifest: drop SP before some labels These seem to be the only occurrences in our code base, so let's make it consistent.

lazymanifest: avoid 'bail' label when used on success path 3d485727e45e (lazymanifest: extract function for iterating to next line, 2015-03-11) introduced a large memory leak because I stopped running the 'bail' block in the successful case. Let's rename 'bail' to 'done', since it has to run not only in the error case.

dockerdeb: rules to build a debian package using docker Currently only supports jessie (current stable), but other version should be trivial.

packaging: extract packagelib for common code from builddeb and buildrpm

builddeb: new script for building a deb package Future work will allow us to use docker to build debs. Right now this doesn't install any config files. I plan to do that as a followup, but getting something basic and working checked in seems like more of a priority than getting everything done in one big step. This also does not create a source deb yet. I haven't looked into that process. Note that this declares incompatibility with the `mercurial-common` package. It's typical for debian packages to be split between architecture-independent bits and native bits, meaning the python bits downstream live in mercurial-common and the c extension bits live in mercurial. We don't do that because we want to (ideally) give users a single deb file to install.

dockerlib: fix initcontainer for boot2docker users This allows me to build rpm packages using boot2docker on my Mac. It's probably a very fragile hack, but it seems to work well enough for now that I felt it was worth sharing.

dockerlib: extract initcontainer() method This helps contain all the logic around creating containers.

dockerlib: start extracting common functions for setting up docker I'm about to start interacting with docker for Debian packaging too, so it's time to centralize this so that any bugfixes I figure out apply to both codepaths.

run-test: ensure the test ports are available before launching test I'm running into a systematic issue because there is always some port taken in the 1500-wide range of ports used by the test (3 for each test file).

templater: strip single backslash before quotation mark in quoted template db7463aa080f fixed the issue of double escapes, but it made the following template fail with syntax error because of <\">. Strictly speaking, <\"> appears to be invalid in non-string part, but we are likely to escape <"> if surrounded by quotes, and we are used to write such templates by trial and error. [templates] sl = "{tags % \"{ifeq(tag,'tip','',label('log.tag', ' {tag}'))}\"}" So, for backward compatibility between 2.8.1 and 3.4, a single backslash before quotation mark is stripped only in quoted template. We don't care for <\"> in string literal in quoted template, which never worked as expected before. template result --------- ------------------------ {\"\"} parse error "{""}" {""} -> <> "{\"\"}" {""} -> <> {"\""} {"\""} -> <"> '{"\""}' {"\""} -> <"> "{"\""}" parse error (don't care)

run-tests: implement Test._testMethodName This methods is needed internally by utilities like __repr__. I do not see any harm in having it declared. Any actual attempt to use it to call a test would crash, but at least random debug print in the test runner will no longer crash. Another approach would have been to redefine the __repr__ function, but I think it a good thing to have some respect for the base class API.

test-subrepo-git: ignore pax_global_header when listing archive 5115d03440f4 is adding a new tar call but forgot to filter out the pax_global_header file that some tar versions write. This is similar to what happened in 5dda6c708138.

test-subrepo-recursion: glob out all the date listed by unzip -l This date output is unstable (01-01-1980 vs 1980-01-01). As the test is not about the modification date, we glob the data out. Caught by buildbot.

bookmarks: remove unused updatecurrentbookmark function (API) This function was not used anywhere in core and there is no indication that it is used elsewhere either.

changelog: drop unnecessary override of "hasnode" revlog.hasnode() calls self.rev(node) that takes filtering into account.

test-run-test: unset run-test specific environment variables Otherwise variable set for the real test run interfere with the test runner tests.

histedit: fix --continue when rules are finished If histedit failed after all the rules were complete (for instance, if there was an exception in the cleanup phase), you couldn't --continue because it was unable to pop a rule. Let's just skip the rule execution phase of --continue if there are no more rules.

histedit: fix serializing of None backupfile If the histedit backupfile was None (like if evolve is enabled) it would get serialized as 'None' into the state file. Later if the histedit was aborted and the top most commit was unreachable (ex: if it was obsolete or stripped), histedit would try to unbundle the backupfile and try to read .hg/None. This fixes it to not serialize None. Since it only happens with evolve, I'm not sure how to add test coverage here.

merge with stable