run-tests: use bytes when constructing shell script

run-tests: use bytes explicitly for tmpdir and hgrc construction We now get slightly further on Python 3. Python 2.6 still works.

run-tests: work around the rename of xrange to range

heredoctest: 2to3 -w -f numliterals -f except -f print tests/heredoctest.py

python3: update killdaemons and run-tests print and exception syntax test-run-tests.t still passes fine on Python 2.6. run-tests.py --local no longer fails with syntax errors, and now fails looking for xrange. Most changes done with 2to3 -w -f numliterals -f except -f print tests/run-tests.py tests/killdaemons.py after which one import was fixed in run-tests and a __future__ import was added.

check-code: allow with statements We dropped python 2.4 compatibility.

check-code: allow 'Except EClass as variable:' Python 2.4 compatibility has been dropped.

check-code: allow print and exec as a function This is required to move forward on python3 compatibility.

pathencode: for long paths, strip first 5 chars, not first dir When encoding long paths, the pure Python code strips the first directory from the path, while the native code currently strips the first 5 characters. This discrepancy has not been a problem so far, since we have not stored anything in directories other than data/. However, we will soon be storing submanifest revlogs in metadata/, so the discrepancy will have to go [1]. Since file collisions are avoided by the hashing alone (which is done on the full unencoded path), it doesn't really matter whether we drop the first dir, the first 5 characters, or special-case non-data/. To avoid touching the C code, let's always strip the first 5 characters. [1] Or maybe elsewhere, but the discrepancy is ugly either way.

util.h: kill no longer needed definitions for Python < 2.5 see e1fb276d4619

rebase: add short -k option for --keep histedit and strip already have a short option for keep, so this makes the interface more consistent

revset: test current behavior of addset class The addset class isn't simple and it has a hidden bug that will be fixed by future patches. So let's test the current behavior.

revset: remove duplicated definition of choice() from addset._iterordered() choice() is already defined before val1 = None. Perhaps there was merge or rebase error.

canonpath: fix infinite recursion

commit: no longer allow empty commit with the 'force' argument (API) The new way to allow empty commit is to temporarily set the 'ui.allowemptycommit' config option. allowemptyback = repo.ui.backupconfig('ui', 'allowemptycommit') try: repo.ui.setconfig('ui', 'allowemptycommit', True) repo.commit(...) finally: repo.ui.restoreconfig(allowemptyback) All known uses of force for allowing empty commits have been removed, so let's remove it from the allowemptycommits condition.

import: use ui.allowemptycommit to allow empty commits Previously import used force=partial to allow empty commits to be made. Let's switch it to using the new ui.allowemptycommit option. Tests says we can drop the 'force' argument in the processs.

mq: use ui.allowemptycommit to allow empty commits Previously, mq used the force flag to allow empty commits. Now that we have ui.allowemptycommit let's switch to that instead. We can't completely remove the force flag since it is used for a bunch of other behavior in localrepo.commit.

commit: add ui.allowemptycommit config option This adds a config flag that enables a user to make empty commits. This is useful in a number of cases. For instance, automation that creates release branches via bookmarks may want to make empty commits to that release bookmark so that it can't be fast-forwarded and so it can record information about the release bookmark's creation. This is already possible with named branches, so making it possible for bookmarks makes sense. Another case we've wanted it is for mirroring repositories into Mercurial. We have automation that syncs commits into hg by running things from the command line. The ability to produce empty commits is useful for syncing unusual commits from other VCS's. In general, allowing the user to create the DAG as they see fit seems useful, and when I mentioned this in IRC more than one person piped up and said they were already hacking around this limitation by using mq, import, and commit-dummy-change-then-amend-the-content-away style solutions.

commit: move empty commit condition to a new line The empty commit condition was a messy if condition. Let's move it to a new line and change it to 'or' statements so it's cleaner and more readable. A future commit will add additional logic to this line.

dirs: speed up by storing number of direct children per dir The Python version of the dirs type stores only the number of direct children associated with each directory. That means that while adding a directory, it only has to walk backwards until it runs into a directory that is already in its map. The C version walks all the way to the top-most directory. By copying the Python version's clever trick to the C code, we can speed it up quite a bit. On the Firefox repo, perfdirs now runs in 0.031390, from 0.056518 before the undoing Sid's optimization in the previous change, and 0.061835 before previous his optimization. More practically, it speeds up 'hg status nonexistent' on the Firefox repo from 0.176s to 0.155s. It's unclear why the C version did not have the same cleverness implemented from the start, especially given that they were both written by the same person (Bryan O'Sullivan) very close in time: 856960173630 (scmutil: add a dirs class, 2013-04-10) 02ee846b246a (scmutil: rewrite dirs in C, use if available, 2013-04-10)

dirs: back out forward-searching in finddirs() This backs out the changes below. The next patch will implement a faster algorithm based on backward-walking in finddirs(). 67241ee427cf (dirs._addpath: reinstate use of Py_CLEAR, 2015-04-07) 6f0e6fa9fdd7 (dirs._addpath: don't mutate Python strings after exposing them (issue4589), 2015-04-06) 1a9efc312700 (dirs.addpath: rework algorithm to search forward, 2015-03-27)

templatekw: replace currentbookmark with activebookmark keyword Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

templatekw: introduce activebookmark keyword Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

templatekw: rename variable current to active Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

pathutil: hint if a path is root relative instead of cwd relative (issue4663) Given that this path is going to abort, it seems OK to spend the time to do an alternate lookup to better inform the user. The path returned by util.pathto() ends with '/' at least in some cases, so os.path.relpath() is used instead, which requires python 2.6.

check-code: drop the python 2.5 warning for os.path.relpath() There's plenty of other cleanup to do in here, but this specific one is used in the next patch.

setup.py: drop compatibility with Python 2.4 and 2.5 (BC) The last blocker for dropping Python 2.4 was Centos 5. We now provide our own Mercurial package for Centos 5 with a bundled Python2.7. I'm therefore happy to officially drop compatibility with Python 2.4 (and Python 2.5 that nobody really cares about). This open the season for code cleanup. It is war's prize to take all vantage.

rpm.spec: bump python dependency to 2.6 We are about to drop 2.4 requirement in Mercurial's setup.py, we bump rpm dependency first for the sake of smaller changeset. Clean up of the spec file can come after the dependency is actually dropped.

map-cmdline.bisect: rewrite to just %include the default template This also adds labels to the output, as the tests now show. I took the liberty to give two labels to the bisection status, just like revisions get the log.changeset and changeset.phase statuses.

log: add a status template It appears that git users like having a --name-status option on `git log`. There exist at least three questions on StackOverflow on how to do this for hg. The desired output is not difficult to build with templates, and since this is something that svn users might also want, it seems desirable to have this as another standard template. This mimics the output of `hg status` and adds it to the log output. It also follows status's convention of displaying copies with a -C option. Brief example: $ hg log -T status -C changeset: 24883:7d1b9b7ac9fd bookmark: statustemplate tag: tip parent: 24864:c560d8c68791 user: Jordi Gutiérrez Hermoso <jordigh@octave.org> date: Wed Apr 22 14:05:04 2015 -0400 summary: log: add a status template files: A added A copied original M modified R removed Of course, everything is also coloured correctly, and there are tests to prove it.

tests: fix hidden repeatability problem with age filter test This test was corrupting later cases in mysterious ways because the hash of the changeset was changing every day at midnight. This made it hard to figure out why tests were consistently breaking after they were consistently working.

help: rewrite template examples to use integer literals where appropriate

templater: update error message of invalid number passed to word() function "word(3, desc)" is valid now.

templater: tokenize decimal integer literal (issue4638) (BC) Before this patch, we had to quote integer literals to pass to template functions. It was error-prone, so we should allow "word(0, x)" syntax. Currently only decimal integers are allowed. It's easy to support 0x, 0b and 0 prefixes, but I don't think they are useful. This patch assumes that template keywords and names defined in map files do not start with digits, except for positional variables seen in the schemes extension.

templater: switch methods table on compileexp() of func args and inner expr The next patch will introduce integer literals, but the schemes extension expects that '{1}', '{2}', ... are interpreted as keywords. This patch allows us to process '{foo(1)}' as 'func(integer)', whereas '{1}' as 'symbol'.

templatefilters: don't stringify None into "None" A few template keywords can in fact return None, such as {bisect}. In some contexts, these get stringified into None instead of "". This is leaking Python details into the UI.

hgweb: clearly outline <tr> block in gitweb and spartan styles It's easy to miss the closing </tr> tag when it's not on its own line. Some neighboring blocks also already use this clearer style. (In 03f692eee31d I didn't even think grepping for this, but this patch should now cover every occurrence).

rebase: use dirstateguard instead of dirstate.invalidate Before this patch, "rebase.concludenode()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook). After this patch, the changed dirstate becomes visible to external "precommit" hooks during "hg rebase" in "test-largefiles-misc.t", because "dirstateguard()" writes it out. But this content isn't yet correct, because: - "normal3" should be marked as "A"(dded) at committing It is newly added in the changeset being rebased. - but it is marked as "M"(odified) The result of "repo.setparents()" after "dirstateguard()" isn't yet written out before "precommit". So, merging is still in progress for "hg status" in it. This causes marking the file newly added on "other" branch as "A". This will be fixed by subsequent patch.

mq: use dirstateguard instead of dirstate.invalidate (qrefresh) Before this patch, "mq.queue.refresh()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

mq: use dirstateguard instead of dirstate.invalidate (qpush) Before this patch, "mq.queue.apply()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during afailure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

tryimportone: use dirstateguard instead of beginparentchange/endparentchange To fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook), a subsequent patch makes "localrepository.commit()" invoke "dirstate.write()" in it. This change will make "beginparentchange()" and "endparentchange()" on dirstate in "cmdutil.tryimportone()" meaningless, because: - "dirstate.write()" writes changed data into ".hg/dirstate", but - aborting between "beginparentchange()" and "endparentchange()" doesn't cause any restoring ".hg/dirstate" it just discards changes in memory. This patch uses "dirstateguard" instead of "beginparentchange()" and "endparentchange()" in "cmdutil.tryimportone()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch uses "lockmod.release(dsguard)" instead of "dsguard.release()", because processing may be aborted before assignment to "dsguard" , and the "if dsguard" examination for safety is redundant.

import: use dirstateguard instead of dirstate.invalidate Before this patch, "commands.import()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless, too. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

amend: use dirstateguard instead of dirstate.invalidate Before this patch, "cmdutil.amend()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

localrepo: use changelog.hasnode instead of self.__contains__ Before this patch, releasing the store lock implies the actions below, when the transaction is aborted: 1. "commithook()" scheduled in "localrepository.commit()" is invoked 2. "changectx.__init__()" is invoked via "self.__contains__()" 3. specified ID is examined against "repo.dirstate.p1()" 4. validation function is invoked in "dirstate.p1()" In subsequent patches, "dirstate.invalidate()" invocations for discarding changes are replaced with "dirstateguard", but discarding changes by "dirstateguard" is executed after releasing the store lock: resources are acquired in "wlock => dirstateguard => store lock" order, and are released in reverse order. This may cause that "dirstate.p1()" still refers to the changeset to be rolled-back at (4) above: pushing multiple patches by "hg qpush" is a typical case. When releasing the store lock, such changesets are: - not contained in "repo.changelog", if it is reloaded from ".hg/00changelog.i", as that file was already truncated by "transaction.abort()" - still contained in it, otherwise (this "dirty read" problem is discussed in "Transaction Plan" http://mercurial.selenic.com/wiki/TransactionPlan) Validation function shows "unknown working parent" warning in the former case, but reloading "repo.changelog" depends on the timestamp of ".hg/00changelog.i". This causes occasional test failures. In the case of scheduled "commithook()", it just wants to examine whether "node ID" of committed changeset is still valid or not. Other examinations implied in "changectx.__init__()" are meaningless. To avoid showing the "unknown working parent" warning irregularly, this patch uses "changelog.hasnode()" instead of "node in self" to examine existence of committed changeset.

cmdutil: add class to restore dirstate during unexpected failure Before this patch, after "dirstate.write()" execution, there was no way to restore dirstate to the original status before "dirstate.write()". In some code paths, "dirstate.invalidate()" is used as a kind of "restore .hg/dirstate to the original status", but it just avoids writing changes in memory out, and doesn't actually restore the ".hg/dirstate" file. To fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hooks), "dirstate.write()" should be invoked before invocation of external processes. But at the same time, ".hg/dirstate" should be restored to its content before "dirstate.write()" during an unexpected failure in some cases. This patch adds the class "dirstateguard" to easily restore ".hg/dirstate" during unexpected failures. Typical usecase of it is: # (1) build dirstate up .... # (2) write dirstate out, and backup ".hg/dirstate" dsguard = dirstateguard(repo, 'scopename') try: # (3) execute somethig to do: # this may imply making some additional changes on dirstate .... # (4) unlink backed-up dirstate file at the end of dsguard scope dsguard.close() finally: # (5) if execution is aborted before "dsguard.close()", # ".hg/dirstate" is restored from the backup dsguard.release() For this kind of issue, an "extending transaction" approach (in https://titanpad.com/mercurial32-sprint) seems to not be suitable, because: - transaction nesting occurs in some cases (e.g. "shelve => rebase"), and - "dirstate" may be already modified since the beginning of OUTER transaction scope, then - dirstate should be backed up into the file other than "dirstate.journal" at the beginning of INNER transaction scope, but - such alternative backup files are useless for transaction itself, and increases complication of its implementation "transaction" and "dirstateguard" differ from each other also in "what it should do for .hg/dirstate" in cases other than success. ============== ======= ======== ============= type success fail "hg rollback" ============== ======= ======== ============= transaction keep keep restore dirstateguard keep restore (not implied) ============== ======= ======== ============= Some collaboration between transaction and dirstate will probably be introduced in the future. But this layer is needed in all cases.

merge with stable

wix: add new json templates folder to MSI installers

templater: rename parsestring() to unquotestring() (API) Since db7463aa080f, it doesn't parse string escapes.