revset: use 'next()' to detect end of iteration in 'limit' The 'next()' built-in can return a default value, allow to get rid of the confusing try/except code flow.

_revancestors: use 'next' to remove the verbose try except clauses The 'next()' built-in can return a default value, making the final iteration case simpler and clearer.

check-code: lift the ban on 'next()' 'next' is supported by Python2.6 the new lowest version supported by Mercurial.

check-code: drop the 'isdisjoint' ban 'isdisjoint' is available in Python 2.6. The new lowest supported version.

check-code: fix the error message about 'class foo():' Using 'classs foo():' result in old style object instead of new style object. We do not want old style object so this check is unrelated to 2.4.

check-code: remove the check for os.path.relpath This is available in Python2.6 the new default supported release.

patch: remove duplicated import of cStringIO It is already imported a couple of line above.

check-commit: print limit when user has a too-long summary

paper: show branch/tags/bookmarks when blaming (issue3559)

paper: show branch/tags/bookmarks when comparing (issue3559)

paper: show branch/tags/bookmarks when diffing (issue3559)

paper: show branch/tags/bookmarks when viewing (issue3559)

paper: show branch/tags/bookmarks when browsing (issue3559) Browse (or manifest) action allows browsing the directory structure at some specified revision. In gitweb and monoblue styles, the revision header already has branch/tag/bookmark information for the revision, but in paper style this header was only showing tags. This patch adds branches and bookmarks. Branch name needs to be obtained in this special way to be consistent with regular changeset page, where in paper style default branch is never shown.

revset: extract addset._iterordered to free function It never uses self, so let's make it less dependent on variables.

revset: use fastasc/fastdesc switch consistently in addset.__iter__

revset: drop redundant filteredset from right-hand side set of "or" operation Since 5548f558db3d, it should no longer be necessary because the addset can remove duplicates correctly.

getbundle: sort bundlecaps before exchanging then over the wire The 'bundlecaps' argument is built as a set, we need to stabilise the order before exchanging them. Otherwise, in the test, http logs are unstable when the 'bundlecaps' contains something (eg: using bundle2).

hg-ssh: reject push earlier (on pretxnopen) We now have a lock triggered for any transaction. We use it to ensure no-read are made in read-only mode. We need more that just "no changegroup is added", since bundle2 allows for more than just changegroup to be exchanged. We still protect pushkey as it may write data without opening a transaction.

test: test both bundle1 and bundle2 paths in 'test-push-hook-lock.t' The locking scheme is a bit different in each cases. So we explicitly test each case.

progress: get the extremely verbose output out of default debug When the progress extension is not enabled, each call to 'ui.progress' used to issue a debug message. This results is a very verbose output and often redundant in tests. Dropping it makes tests less volatile to factor they do not meant to test. We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all output from a certain point and hidding the progress output remove its anchor. So we anchor on something else.

hgweb: show changeset branches/tags/bookmarks in file log (style=monoblue) As for the gitweb style, this line for filelogentry template is copied from shortlogentry. No change to python code is needed. Tests are unaffected.

hgweb: show changeset branches/tags/bookmarks in file log (style=gitweb) This line for filelogentry template is copied from shortlogentry. The information is already displayed in paper style, so no change to python code is needed. Tests are unaffected.

match: resolve filesets in subrepos for commands given the '-S' argument This will work for any command that creates its matcher via scmutil.match(), but only the files command is tested here (both workingctx and basectx based tests). The previous behavior was to completely ignore the files in the subrepo, even though -S was given. My first attempt was to teach context.walk() to optionally recurse, but once that was in place and the complete file list was built up, the predicate test would fail with 'path in nested repo' when a file in a subrepo was accessed through the parent context. There are two slightly surprising behaviors with this functionality. First, any path provided inside the fileset isn't narrowed when it is passed to the subrepo. I dont see any clean way to do that in the matcher. Fortunately, the 'subrepo()' fileset is the only one to take a path. The second surprise is that status predicates are resolved against the subrepo, not the parent like 'hg status -S' is. I don't see any way to fix that either, given the path auditor error mentioned above.

subrepo: introduce getfileset() This will be used in the next patch to help matchers resolve filesets in subrepos. The default implementation returns an empty set (for git and svn).

phase: default to current revision if no rev is provided (issue4666)

treemanifest: add --dir option to debug{revlog,data,index} It should be possible to debug the submanifest revlogs without having to know where they are stored (in .hg/store/meta/), so let's add a --dir option for this purpose.

obsolete: sort obsmarkers during exchange Because bundle2 allows a more precise exchange of obsmarkers during pull, it sends them in a different order (previously unstable because of sets.) As a result, they are added to the repository in a different order. To stabilize the order and ensure tests are unchanged when moving from bundle1 to bundle2 we sort markers when exchanging them. In the long run, the obsstore will probably not use a linear storage.

test-discovery-legacy: disable bundle2 for legacy test We need to disable the "bundle2" capability to ensure we stay on legacy capability.

test-discovery: disable bundle2 for legacy test (issue4610) We need to disable the "bundle2" capability to ensure we stay on legacy capability.

revset: fix iteration over ordered addset composed of non-ordered operands Before this change, doing ordered iteration over an 'addset' object composed of operands without fastasc or fastdesc method could result in duplicated entries. This was the result of applying '_iterordered' on an unordered set. We fix it by ensuring we iterate over the set in a sorted order. Using the fast iterator when it exists on any operand. We kill the '_iterator' method in the process because it did not make a lot of sense independently. Thanks goes to Yuya Nishihara for reporting the issue and analysing the cause.

match: add match.ispartial() match.ispartial() will return the opposite of match.always() in core, but this function will be extensible by extensions to produce another result even if match.always() will be untouched. This will be useful for narrowhg, where ispartial() will return False even if the match won't always match. This would happen in the case where the only time the match function is False is when the path is outside of the narrow spec.

util: drop alias for collections.deque Now that util.deque is just an alias for collections.deque, let's just remove it.

util: kill Python 2.4 deque.remove hack

summary: add a phase line (draft, secret) to the output The number of draft and secret changesets are currently not summarized. This is an important information because the number of drafts give some rough idea of the number of outgoing changesets in typical workflows, without needing to probe a remote repository. And a non-zero number of secrets means that those changeset will not be pushed. If the repository is "dirty" - some draft or secret changesets exists - then summary will display a line like: phases: X draft, Y secret (public) The phase in parenthesis corresponds to the highest phase of the parents of the working directory, i.e. the current phase. By default, the line is not printed if the repository is "clean" - all changesets are public - but if verbose is activated, it will display: phases: (public) On the other hand, nothing will be printed if quiet is in action. A few tests have been added in test-phases.t to cover the -v and -q cases.

hgweb: also display changeset bookmarks in file log (style=paper) There already are branches and tags in file log, now let's add what's been missing: bookmarks. Also, since coal borrows this template from paper, this change is effective for coal as well.

tests: test branch/tag/bookmark display in test-hgweb-filelog.t Displaying branches, tags and bookmarks is an obviously important feature of hgweb and should be tested a bit more than not at all, so let's add a branch, a tag and a bookmark to the test. With this change it's evident that the default style (paper) doesn't show bookmarks in filelog. Future patch will fix this.

hgweb: split some long lines in paper, gitweb and monoblue styles This will ease future patches for the templates. As a result of this patch, paper style has one visual change in log/shortlog/file log view: the spacing between commit message and the first tag (or branch name, or bookmark) is now roughly who spaces wide instead of one space wide. This spacing is consistent with the one between branch names/tags/bookmarks themselves, so it looks better. In gitweb style, the change from non-breakable space to regular space is consistent with other elements. In monoblue the change is not noticeable.

run-tests: add a --profile-runner option While in the throes of a recent run-tests adventure, I found it useful to have profiler output for the testrunner itself. Adding it was simple enough and seems worth keeping around.

test-http: drop compatibility check on Python >= 2.4.3 This is obsolete now that Python < 2.6 is no more supported. This is thus a backout of c499fff76440.

revset: remove unused 'only' from methods table The infix 'only' operator is mapped to 'only()' function by optimize(), so it won't be looked up as a method. The test shows it.

shelve: allow --patch and --stat without --list for a single shelf It's annoying having to specify --list and --patch/--stat when all you really want to do is to dump a patch. This creates an explicit --patch/--stat command that is executed if --list is not specified. It ensures that 1) there is only one shelf name specified and 2) that the shelf exists. Then it redirects to the original listcmd code.

shelve: refactor allowables to specify sets of valid operations This will allow us to have --patch and --stat as standalone operations that don't require --list, as well as pairing them with --list.

merge with stable

rebase: check that the bookmark is still valid when restoring (issue4669) After a rebase --abort, we attempt to restore the previously active bookmark. We need to ensure that the bookmark still exists.

bookmarks: rename current to active in variables and comments Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

rebase: remove extraneous blank at the end of file

run-tests: include 'start' and 'end' in --time output This is useful information to understand what is taking time in tests. Both are included because I can see myself sorting this output using shell script. Having both data makes it much easier than extracting 'start'+'real'.

run-tests: track start and end time of tests We currently have information about how long each test took, but we have no data about their actual scheduling. So we now track when a test started and stopped (in the referential of the whole tests run) to expose this information. The data is currently in the json only because the json output is meant to be extensible. Later changeset will includes this data in the text output and we will be happy to bikeshed its formating there. Yes, "end" is actually just "start" + "time", but computing it an including it in the output is simple, cheap and convenient.

templater: look for mapfiles in template paths This will allow %include statements to search the default template paths in addition to the directory where the %including file is.

config: give it an includepaths option for looking for config files It is desirable to "derive" templates from the provided templates. A simple way to do this is e.g. %include map-cmdline.default in your own mapfile. Then you only have to redefine a few templates instead of copying over the whole thing. This %include mechanism already works for the built-in templates because by default it *only* looks for files that are in the same directory as the including mapfile. With this changeset, config grows an option to add more include paths for config files.

revset: map postfix '%' to only() to optimize operand recursively (issue4670) Instead of keeping 'onlypost' as a method, this patch rewrites it to 'only' function. This way, 'x%' always has the same weight as 'only(x)'.

dirs.c: pass C string, not Python string, to _finddir() The callers already have the C string, and although the PyString_AS_STRING() macro is probably free, this simplifies the code.

dirs.c: extract 'cpath' variable in _delpath() to match _addpath() The PyString_AS_STRING() macro is probably free, but this makes _delpath() more similar to _addpath() and simplifies the next patch.

treemanifest: store submanifest revlog per directory With this change, when tree manifests are enabled (in .hg/requires), commits will be written with one manifest revlog per directory. The manifest revlogs are stored in .hg/store/meta/$dir/00manifest.[id]. Flat manifests can still be read and interacted with as usual (they are also read into treemanifest instances). The functionality for writing treemanifest as a flat manifest to disk is still left in the code; tests still pass with '_treeinmem=True' hardcoded. Exchange is not yet implemented.

hghave: use try/except/finally

setup: use try/except/finally This will raise a syntax error for people who attempt to use Py2.4, but that's already going to fail and we have no way to keep other 2.6isms from creeping in since we've removed the check-code rules and the buildbot.

util: use try/except/finally

tags: use try/except/finally

repoview: use try/except/finally

httppeer: use try/except/finally

hooks: use try/except/finally

hgweb: use try/except/finally

commands: use try/except/finally

bookmarks: use try/except/finally

shelve: use try/except/finally

largefiles: use try/except/finally

factotum: use try/except/finally

check-code: drop try/except/finally check

util.h: kill no longer needed definitions for Python < 2.6 see e1fb276d4619

merge with stable

setup: integrate osutil C extension into extmodules initialization

setup: move osutil_ldflags logic to before extmodules definition

setup: kill pure hack for osutil on Windows for Python 2.4 see e1fb276d4619

windows: add comment in normcase() see also 3c5e818ac679

rebase: clear merge when aborting before any rebasing (issue4661) The check of the inrebase function was not correct, and it failed to consider the situation in which nothing has been rebased yet, *and* the working dir had been updated away from the initial revision. But this is easy to fix. Given the rebase state, we know exactly where we should be standing: on the first unrebased commit. We check that instead. I also took the liberty to rename the function, as "inrebase" doesn't really describe the situation: we could still be in a rebase state yet the user somehow forcibly updated to a different revision. We also check that we're in a merge state, since an interrupted merge is the only "safe" way to interrupt a rebase. If the rebase got interrupted by power loss or whatever (so there's no merge state), it's still safer to not blow away the working directory.

test-rebase-abort: add test from issue4009 The fix for issue4009, namely fe78eb7bcca0, introduced issue4661. Let's make sure that the fix for issue4661 will not reintroduce issue4009.

run-test.py: greatly increase the priority of 'check-code' tests As check-code is actually quite long to run for its file size, this prevent them to be scheduled too late during the test run. This reduces my typical test run from 107 seconds to 90 seconds

run-tests: allow different extra weight for slow tests The 'test-check-code-hg.t' file is not big enough to be prioritized properly. As a result my tests run often spend about 15 seconds running only it at the end of its tests run. We make the "slow" mechanism a bit smarter to adjust the extra weight of each category independently in a future patch.

ignore: refactor syntax concatenation This refactors the syntax+rule concatenation logic to be more separated. It determines the syntax and the rule separately and then puts them back together. This will help in a later patch when we want to process just the rule before it gets concatenated.

ignore: refactor file read into a function This refactors the ignore file reading code into a function so that in a future patch we can make it recursive.

import-checker: loop to get list of locally defined modules at first This is a preparation for subsequent patches, which expect that all locally defined (= mercurial specific) modules are already known before examinations. Looping twice for specified modules is a little redundant, but reasonable cost for improvement in subsequent patches.

import-checker: add xargs like mode Before this patch, "import-check.py" is invoked via "xargs" in "test-module-imports.t", but it doesn't ensure that "import-checker.py" is certainly invoked with all mercurial specific files at once. "xargs" may invoke specified command multiple times with part of arguments given from stdin: according to "xargs(1)" man page, this dividing arguments is system-dependent. This patch adds "xargs" like mode to "import-checker.py". This can ensure that "import-checker.py" is certainly invoked with all mercurial specific files at once in "test-module-imports.t". This is assumed by subsequent patches.

store: add comment in _hashencode with reason for skipping the first 5 chars

test-run-tests: fix for Python 3.5 This test now passes in both 3.5 and 2.6.

run-tests: make sure all script lines are bytes

run-tests: fix checking a line to see if it needs globbing

run-tests: unblock running python tests in python 3 This makes it clear we need to ban os.popen, but we'll do that in a later cleanup.

run-tests: os.popen returns a string, make it bytes again

run-tests: be more paranoid about os.system using bytes

run-tests: blacklist entries are bytes, use bname to check blacklisting

test-run-tests.t: work around file.write() returning an int In Python 3.5, file.write() returns the number of bytes it wrote instead of None.

run-tests: write bytes to the binary buffer on sys.{stdout,stderr}

run-tests: record faildata using bytes instead of str lines is already bytes, so this is just fixing a bug on python 3.

run-tests: do cdata escaping using bytes instead of str

run-tests: make sure keyword(s) are in bytes and not str

run-tests: use items instead of iteritems on dicts This works the same-enough on 2.6 and 3.5 to be fine.

run-tests: refer to test.bname when sniffing for keywords

run-tests: string-escape no longer exists in python 3, use unicode_escape

run-tests: switch all uses of iolock.acquire() to a context manager

run-tests: use difflib.diff_bytes on Python 3 This method was introduced in Python 3.5 to satisfy our diffing-strings-of-bytes needs.

run-tests: fix installation of hg by bytesifying more constants

setup: decode xcode version number on python3

run-tests: work around with_hg being bytes or string depending on provenance

run-tests: even more bytestring annotations for Python 3

run-tests: write bytes to sys.stdout.buffer in python 3

run-tests: be more judicious about bytes vs string on test attrs This introduces a bname attribute for when we need to use the test name in a path, and retains name for when we need it as a string. Sadly, we seem to need both of these.

run-tests: fix _findprogram to reliably return bytes

run-tests: introduce a name for os.environb On python 2 this is just os.environ, but on python 3 we need to get to os.environb in several places.

run-tests: work around chr() producing unicode in Python 3

run-tests: use bytes when constructing shell script

run-tests: use bytes explicitly for tmpdir and hgrc construction We now get slightly further on Python 3. Python 2.6 still works.

run-tests: work around the rename of xrange to range

heredoctest: 2to3 -w -f numliterals -f except -f print tests/heredoctest.py

python3: update killdaemons and run-tests print and exception syntax test-run-tests.t still passes fine on Python 2.6. run-tests.py --local no longer fails with syntax errors, and now fails looking for xrange. Most changes done with 2to3 -w -f numliterals -f except -f print tests/run-tests.py tests/killdaemons.py after which one import was fixed in run-tests and a __future__ import was added.

check-code: allow with statements We dropped python 2.4 compatibility.

check-code: allow 'Except EClass as variable:' Python 2.4 compatibility has been dropped.

check-code: allow print and exec as a function This is required to move forward on python3 compatibility.

pathencode: for long paths, strip first 5 chars, not first dir When encoding long paths, the pure Python code strips the first directory from the path, while the native code currently strips the first 5 characters. This discrepancy has not been a problem so far, since we have not stored anything in directories other than data/. However, we will soon be storing submanifest revlogs in metadata/, so the discrepancy will have to go [1]. Since file collisions are avoided by the hashing alone (which is done on the full unencoded path), it doesn't really matter whether we drop the first dir, the first 5 characters, or special-case non-data/. To avoid touching the C code, let's always strip the first 5 characters. [1] Or maybe elsewhere, but the discrepancy is ugly either way.

util.h: kill no longer needed definitions for Python < 2.5 see e1fb276d4619

rebase: add short -k option for --keep histedit and strip already have a short option for keep, so this makes the interface more consistent

revset: test current behavior of addset class The addset class isn't simple and it has a hidden bug that will be fixed by future patches. So let's test the current behavior.

revset: remove duplicated definition of choice() from addset._iterordered() choice() is already defined before val1 = None. Perhaps there was merge or rebase error.

canonpath: fix infinite recursion

commit: no longer allow empty commit with the 'force' argument (API) The new way to allow empty commit is to temporarily set the 'ui.allowemptycommit' config option. allowemptyback = repo.ui.backupconfig('ui', 'allowemptycommit') try: repo.ui.setconfig('ui', 'allowemptycommit', True) repo.commit(...) finally: repo.ui.restoreconfig(allowemptyback) All known uses of force for allowing empty commits have been removed, so let's remove it from the allowemptycommits condition.

import: use ui.allowemptycommit to allow empty commits Previously import used force=partial to allow empty commits to be made. Let's switch it to using the new ui.allowemptycommit option. Tests says we can drop the 'force' argument in the processs.

mq: use ui.allowemptycommit to allow empty commits Previously, mq used the force flag to allow empty commits. Now that we have ui.allowemptycommit let's switch to that instead. We can't completely remove the force flag since it is used for a bunch of other behavior in localrepo.commit.

commit: add ui.allowemptycommit config option This adds a config flag that enables a user to make empty commits. This is useful in a number of cases. For instance, automation that creates release branches via bookmarks may want to make empty commits to that release bookmark so that it can't be fast-forwarded and so it can record information about the release bookmark's creation. This is already possible with named branches, so making it possible for bookmarks makes sense. Another case we've wanted it is for mirroring repositories into Mercurial. We have automation that syncs commits into hg by running things from the command line. The ability to produce empty commits is useful for syncing unusual commits from other VCS's. In general, allowing the user to create the DAG as they see fit seems useful, and when I mentioned this in IRC more than one person piped up and said they were already hacking around this limitation by using mq, import, and commit-dummy-change-then-amend-the-content-away style solutions.

commit: move empty commit condition to a new line The empty commit condition was a messy if condition. Let's move it to a new line and change it to 'or' statements so it's cleaner and more readable. A future commit will add additional logic to this line.

dirs: speed up by storing number of direct children per dir The Python version of the dirs type stores only the number of direct children associated with each directory. That means that while adding a directory, it only has to walk backwards until it runs into a directory that is already in its map. The C version walks all the way to the top-most directory. By copying the Python version's clever trick to the C code, we can speed it up quite a bit. On the Firefox repo, perfdirs now runs in 0.031390, from 0.056518 before the undoing Sid's optimization in the previous change, and 0.061835 before previous his optimization. More practically, it speeds up 'hg status nonexistent' on the Firefox repo from 0.176s to 0.155s. It's unclear why the C version did not have the same cleverness implemented from the start, especially given that they were both written by the same person (Bryan O'Sullivan) very close in time: 856960173630 (scmutil: add a dirs class, 2013-04-10) 02ee846b246a (scmutil: rewrite dirs in C, use if available, 2013-04-10)

dirs: back out forward-searching in finddirs() This backs out the changes below. The next patch will implement a faster algorithm based on backward-walking in finddirs(). 67241ee427cf (dirs._addpath: reinstate use of Py_CLEAR, 2015-04-07) 6f0e6fa9fdd7 (dirs._addpath: don't mutate Python strings after exposing them (issue4589), 2015-04-06) 1a9efc312700 (dirs.addpath: rework algorithm to search forward, 2015-03-27)

templatekw: replace currentbookmark with activebookmark keyword Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

templatekw: introduce activebookmark keyword Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

templatekw: rename variable current to active Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

pathutil: hint if a path is root relative instead of cwd relative (issue4663) Given that this path is going to abort, it seems OK to spend the time to do an alternate lookup to better inform the user. The path returned by util.pathto() ends with '/' at least in some cases, so os.path.relpath() is used instead, which requires python 2.6.

check-code: drop the python 2.5 warning for os.path.relpath() There's plenty of other cleanup to do in here, but this specific one is used in the next patch.

setup.py: drop compatibility with Python 2.4 and 2.5 (BC) The last blocker for dropping Python 2.4 was Centos 5. We now provide our own Mercurial package for Centos 5 with a bundled Python2.7. I'm therefore happy to officially drop compatibility with Python 2.4 (and Python 2.5 that nobody really cares about). This open the season for code cleanup. It is war's prize to take all vantage.

rpm.spec: bump python dependency to 2.6 We are about to drop 2.4 requirement in Mercurial's setup.py, we bump rpm dependency first for the sake of smaller changeset. Clean up of the spec file can come after the dependency is actually dropped.

map-cmdline.bisect: rewrite to just %include the default template This also adds labels to the output, as the tests now show. I took the liberty to give two labels to the bisection status, just like revisions get the log.changeset and changeset.phase statuses.

log: add a status template It appears that git users like having a --name-status option on `git log`. There exist at least three questions on StackOverflow on how to do this for hg. The desired output is not difficult to build with templates, and since this is something that svn users might also want, it seems desirable to have this as another standard template. This mimics the output of `hg status` and adds it to the log output. It also follows status's convention of displaying copies with a -C option. Brief example: $ hg log -T status -C changeset: 24883:7d1b9b7ac9fd bookmark: statustemplate tag: tip parent: 24864:c560d8c68791 user: Jordi Gutiérrez Hermoso <jordigh@octave.org> date: Wed Apr 22 14:05:04 2015 -0400 summary: log: add a status template files: A added A copied original M modified R removed Of course, everything is also coloured correctly, and there are tests to prove it.

tests: fix hidden repeatability problem with age filter test This test was corrupting later cases in mysterious ways because the hash of the changeset was changing every day at midnight. This made it hard to figure out why tests were consistently breaking after they were consistently working.

help: rewrite template examples to use integer literals where appropriate

templater: update error message of invalid number passed to word() function "word(3, desc)" is valid now.

templater: tokenize decimal integer literal (issue4638) (BC) Before this patch, we had to quote integer literals to pass to template functions. It was error-prone, so we should allow "word(0, x)" syntax. Currently only decimal integers are allowed. It's easy to support 0x, 0b and 0 prefixes, but I don't think they are useful. This patch assumes that template keywords and names defined in map files do not start with digits, except for positional variables seen in the schemes extension.

templater: switch methods table on compileexp() of func args and inner expr The next patch will introduce integer literals, but the schemes extension expects that '{1}', '{2}', ... are interpreted as keywords. This patch allows us to process '{foo(1)}' as 'func(integer)', whereas '{1}' as 'symbol'.

templatefilters: don't stringify None into "None" A few template keywords can in fact return None, such as {bisect}. In some contexts, these get stringified into None instead of "". This is leaking Python details into the UI.

hgweb: clearly outline <tr> block in gitweb and spartan styles It's easy to miss the closing </tr> tag when it's not on its own line. Some neighboring blocks also already use this clearer style. (In 03f692eee31d I didn't even think grepping for this, but this patch should now cover every occurrence).

rebase: use dirstateguard instead of dirstate.invalidate Before this patch, "rebase.concludenode()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook). After this patch, the changed dirstate becomes visible to external "precommit" hooks during "hg rebase" in "test-largefiles-misc.t", because "dirstateguard()" writes it out. But this content isn't yet correct, because: - "normal3" should be marked as "A"(dded) at committing It is newly added in the changeset being rebased. - but it is marked as "M"(odified) The result of "repo.setparents()" after "dirstateguard()" isn't yet written out before "precommit". So, merging is still in progress for "hg status" in it. This causes marking the file newly added on "other" branch as "A". This will be fixed by subsequent patch.

mq: use dirstateguard instead of dirstate.invalidate (qrefresh) Before this patch, "mq.queue.refresh()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

mq: use dirstateguard instead of dirstate.invalidate (qpush) Before this patch, "mq.queue.apply()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during afailure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hook).

tryimportone: use dirstateguard instead of beginparentchange/endparentchange To fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook), a subsequent patch makes "localrepository.commit()" invoke "dirstate.write()" in it. This change will make "beginparentchange()" and "endparentchange()" on dirstate in "cmdutil.tryimportone()" meaningless, because: - "dirstate.write()" writes changed data into ".hg/dirstate", but - aborting between "beginparentchange()" and "endparentchange()" doesn't cause any restoring ".hg/dirstate" it just discards changes in memory. This patch uses "dirstateguard" instead of "beginparentchange()" and "endparentchange()" in "cmdutil.tryimportone()" to restore ".hg/dirstate" during a failure even if "dirstate.write()" is executed before a failure. This patch uses "lockmod.release(dsguard)" instead of "dsguard.release()", because processing may be aborted before assignment to "dsguard" , and the "if dsguard" examination for safety is redundant.

import: use dirstateguard instead of dirstate.invalidate Before this patch, "commands.import()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This patch also removes "beginparentchage()" and "endparentchange()", because "dirstateguard" makes them useless, too. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

amend: use dirstateguard instead of dirstate.invalidate Before this patch, "cmdutil.amend()" uses "dirstate.invalidate()" as a kind of "restore .hg/dirstate to the original status" during a failure. But it just discards changes in memory, and doesn't actually restore ".hg/dirstate". Then, it can't work as expected, if "dirstate.write()" is executed while processing. This patch uses "dirstateguard" instead of "dirstate.invalidate()" to restore ".hg/dirstate" at failure even if "dirstate.write()" is executed before failure. This is a part of preparations to fix the issue that the recent (in memory) dirstate isn't visible to external process (e.g. "precommit" hook).

localrepo: use changelog.hasnode instead of self.__contains__ Before this patch, releasing the store lock implies the actions below, when the transaction is aborted: 1. "commithook()" scheduled in "localrepository.commit()" is invoked 2. "changectx.__init__()" is invoked via "self.__contains__()" 3. specified ID is examined against "repo.dirstate.p1()" 4. validation function is invoked in "dirstate.p1()" In subsequent patches, "dirstate.invalidate()" invocations for discarding changes are replaced with "dirstateguard", but discarding changes by "dirstateguard" is executed after releasing the store lock: resources are acquired in "wlock => dirstateguard => store lock" order, and are released in reverse order. This may cause that "dirstate.p1()" still refers to the changeset to be rolled-back at (4) above: pushing multiple patches by "hg qpush" is a typical case. When releasing the store lock, such changesets are: - not contained in "repo.changelog", if it is reloaded from ".hg/00changelog.i", as that file was already truncated by "transaction.abort()" - still contained in it, otherwise (this "dirty read" problem is discussed in "Transaction Plan" http://mercurial.selenic.com/wiki/TransactionPlan) Validation function shows "unknown working parent" warning in the former case, but reloading "repo.changelog" depends on the timestamp of ".hg/00changelog.i". This causes occasional test failures. In the case of scheduled "commithook()", it just wants to examine whether "node ID" of committed changeset is still valid or not. Other examinations implied in "changectx.__init__()" are meaningless. To avoid showing the "unknown working parent" warning irregularly, this patch uses "changelog.hasnode()" instead of "node in self" to examine existence of committed changeset.

cmdutil: add class to restore dirstate during unexpected failure Before this patch, after "dirstate.write()" execution, there was no way to restore dirstate to the original status before "dirstate.write()". In some code paths, "dirstate.invalidate()" is used as a kind of "restore .hg/dirstate to the original status", but it just avoids writing changes in memory out, and doesn't actually restore the ".hg/dirstate" file. To fix the issue that the recent (in memory) dirstate isn't visible to external processes (e.g. "precommit" hooks), "dirstate.write()" should be invoked before invocation of external processes. But at the same time, ".hg/dirstate" should be restored to its content before "dirstate.write()" during an unexpected failure in some cases. This patch adds the class "dirstateguard" to easily restore ".hg/dirstate" during unexpected failures. Typical usecase of it is: # (1) build dirstate up .... # (2) write dirstate out, and backup ".hg/dirstate" dsguard = dirstateguard(repo, 'scopename') try: # (3) execute somethig to do: # this may imply making some additional changes on dirstate .... # (4) unlink backed-up dirstate file at the end of dsguard scope dsguard.close() finally: # (5) if execution is aborted before "dsguard.close()", # ".hg/dirstate" is restored from the backup dsguard.release() For this kind of issue, an "extending transaction" approach (in https://titanpad.com/mercurial32-sprint) seems to not be suitable, because: - transaction nesting occurs in some cases (e.g. "shelve => rebase"), and - "dirstate" may be already modified since the beginning of OUTER transaction scope, then - dirstate should be backed up into the file other than "dirstate.journal" at the beginning of INNER transaction scope, but - such alternative backup files are useless for transaction itself, and increases complication of its implementation "transaction" and "dirstateguard" differ from each other also in "what it should do for .hg/dirstate" in cases other than success. ============== ======= ======== ============= type success fail "hg rollback" ============== ======= ======== ============= transaction keep keep restore dirstateguard keep restore (not implied) ============== ======= ======== ============= Some collaboration between transaction and dirstate will probably be introduced in the future. But this layer is needed in all cases.

merge with stable

wix: add new json templates folder to MSI installers

templater: rename parsestring() to unquotestring() (API) Since db7463aa080f, it doesn't parse string escapes.

templater: remove noop calls of parsestring(s, quoted=False) (API) Since db7463aa080f, parsestring(s, quoted=False) just returns s.

bookmarks: simplify iscurrent to isactivewdirparent (API) Previously this function accepted two optional parameters that were unused by any callers and complicated the function. Today, the terms 'active' and 'current' are interchangeably used throughout the codebase in reference to the active bookmark (the bookmark that will be updated with the next commit). This leads to confusion among developers and users. This patch is part of a series to standardize the usage to 'active' throughout the mercurial codebase and user interface.

tests: add a new commit to test-commandline-template This commit modifies, adds, renames, removes files all at once. It will be mostly interesting for an upcoming test relating to the status log template.

run-tests: spread and document the content of time tuple The spreading will make future modification clearer. The documentation improve code readability.

run-tests: when building json, get time data in the same order as elsewhere The json code was changing the order of the time tuple for unclear reasons. We now use the same order as everywhere else.

run-tests: stop explicit expansion of time data We are about to record more complex time-related data, which will require repeated changes of every loop touching times data. That will also extend such lines to a point where things become too long. Instead, we iterate on each entry and expand values in the loops. We keep intermediate variables in most cases to preserve readability. The loop producing json data does a strange inversion for no obvious reason. We preserved that for now and will fix it in another changeset.

run-tests: refactor json entry creation logic We are about to add more timing related information to each entry. Having a single place where the json entry is created is going to be much simpler.

run-tests: when building json, use result.failures instead of result.faildata It is unclear to me why 'faildata' was used. Let's use the same kind of attribute as for the other groups.

run-test: add a test for json output when -i is used The -i can apparently confused some internal data structure, we want to make sure we do not regress.

changegroup: drop _changelog and _manifest properties We already have a _repo property on the packer, and we only access the changelog and manifest revlog in one place, so it's just as easy to get them from self._repo.

changegroup: document the cases where reordering complicates linkrevs

changegroup: extract condition for linkrev fastpath The condition for taking the fastpath (or not) is used in two places. By extracting it, we also provide a place to document what it's about.

lazymanifest: drop SP before some labels These seem to be the only occurrences in our code base, so let's make it consistent.

lazymanifest: avoid 'bail' label when used on success path 3d485727e45e (lazymanifest: extract function for iterating to next line, 2015-03-11) introduced a large memory leak because I stopped running the 'bail' block in the successful case. Let's rename 'bail' to 'done', since it has to run not only in the error case.

dockerdeb: rules to build a debian package using docker Currently only supports jessie (current stable), but other version should be trivial.

packaging: extract packagelib for common code from builddeb and buildrpm

builddeb: new script for building a deb package Future work will allow us to use docker to build debs. Right now this doesn't install any config files. I plan to do that as a followup, but getting something basic and working checked in seems like more of a priority than getting everything done in one big step. This also does not create a source deb yet. I haven't looked into that process. Note that this declares incompatibility with the `mercurial-common` package. It's typical for debian packages to be split between architecture-independent bits and native bits, meaning the python bits downstream live in mercurial-common and the c extension bits live in mercurial. We don't do that because we want to (ideally) give users a single deb file to install.

dockerlib: fix initcontainer for boot2docker users This allows me to build rpm packages using boot2docker on my Mac. It's probably a very fragile hack, but it seems to work well enough for now that I felt it was worth sharing.

dockerlib: extract initcontainer() method This helps contain all the logic around creating containers.

dockerlib: start extracting common functions for setting up docker I'm about to start interacting with docker for Debian packaging too, so it's time to centralize this so that any bugfixes I figure out apply to both codepaths.

run-test: ensure the test ports are available before launching test I'm running into a systematic issue because there is always some port taken in the 1500-wide range of ports used by the test (3 for each test file).

templater: strip single backslash before quotation mark in quoted template db7463aa080f fixed the issue of double escapes, but it made the following template fail with syntax error because of <\">. Strictly speaking, <\"> appears to be invalid in non-string part, but we are likely to escape <"> if surrounded by quotes, and we are used to write such templates by trial and error. [templates] sl = "{tags % \"{ifeq(tag,'tip','',label('log.tag', ' {tag}'))}\"}" So, for backward compatibility between 2.8.1 and 3.4, a single backslash before quotation mark is stripped only in quoted template. We don't care for <\"> in string literal in quoted template, which never worked as expected before. template result --------- ------------------------ {\"\"} parse error "{""}" {""} -> <> "{\"\"}" {""} -> <> {"\""} {"\""} -> <"> '{"\""}' {"\""} -> <"> "{"\""}" parse error (don't care)

run-tests: implement Test._testMethodName This methods is needed internally by utilities like __repr__. I do not see any harm in having it declared. Any actual attempt to use it to call a test would crash, but at least random debug print in the test runner will no longer crash. Another approach would have been to redefine the __repr__ function, but I think it a good thing to have some respect for the base class API.

test-subrepo-git: ignore pax_global_header when listing archive 5115d03440f4 is adding a new tar call but forgot to filter out the pax_global_header file that some tar versions write. This is similar to what happened in 5dda6c708138.

test-subrepo-recursion: glob out all the date listed by unzip -l This date output is unstable (01-01-1980 vs 1980-01-01). As the test is not about the modification date, we glob the data out. Caught by buildbot.

bookmarks: remove unused updatecurrentbookmark function (API) This function was not used anywhere in core and there is no indication that it is used elsewhere either.

changelog: drop unnecessary override of "hasnode" revlog.hasnode() calls self.rev(node) that takes filtering into account.

test-run-test: unset run-test specific environment variables Otherwise variable set for the real test run interfere with the test runner tests.

histedit: fix --continue when rules are finished If histedit failed after all the rules were complete (for instance, if there was an exception in the cleanup phase), you couldn't --continue because it was unable to pop a rule. Let's just skip the rule execution phase of --continue if there are no more rules.

histedit: fix serializing of None backupfile If the histedit backupfile was None (like if evolve is enabled) it would get serialized as 'None' into the state file. Later if the histedit was aborted and the top most commit was unreachable (ex: if it was obsolete or stripped), histedit would try to unbundle the backupfile and try to read .hg/None. This fixes it to not serialize None. Since it only happens with evolve, I'm not sure how to add test coverage here.

merge with stable

treemanifest: set requires at repo creation time, ignore config after The very next changeset will start writing one revlog per directory when tree manifests are enabled. That is backwards incompatible, so it requires .hg/requires to be updated. Just like with generaldelta, we want to update .hg/requires only when the repo is created. Updating ..hg/requires is bad for repos on shared disk. Instead, those who do want to upgrade a repo to using treemanifest (or manifestv2, etc) can run hg clone --config experimental.treemanifest repo clone which will create a new repo with the requirement set. Unlike the case of e.g. generaldelta, it will not rewrite the changesets, since tree manifests hash differently.

remove: use ctx.hasdir(f) instead of 'f in ctx.dirs()' This speeds up 'hg remove python/README' on the Firefox repo from 2.479s to 0.664s with lazily loaded treemanifests (which is not yet in core) and has no measurable effect on flat manifests.

largefiles: drop os.path.join() in subrepo archive override This is related to 41cd8171e58f, but not required for that to function correctly because 'prefix' always ends with '/', so os.path.join(prefix, _path) simply concatenates the strings. Take the opportunity to drop the os.path usage, and avoid confusion by using the same pattern as the overridden subrepo method.

archive: drop the leading '.' path component from the prefix (issue4634) Unix utilities like tar will happily prefix the files it packs with './', but annoyingly, Windows Explorer will not show these packed files when it opens the archive. Since there doesn't seem to be a point in including './' in the path names, just drop it. The default 'hg archive' prefix is the basename of the archive, so specifying '.' allows for that default to be disabled completely.