merge with stable

log: allow revset for --follow to be lazily evaluated It is unclear to me why evaluation was forced. For a repository with over 700,000 commits, 'hg log -f' drops from 1.2 seconds to 0.2 seconds.

parsers: remove unused getintat function Warning detected by clang.

revset: maintain ordering when subtracting from a baseset (issue4289)

cmdutil: separate building commit text from 'commitforceeditor' This separation makes it easier to extend/hook building commit text from the specified context. This patch uses 'committext' instead of 'edittext' for names of newly added variable and function, because the former is more purpose specific than the latter, even though 'edittext' in 'buildcommittext' is left as it is to reduce amount of diff.

convert: detect removal of ".gitmodules" at git source revisions correctly Before this patch, all operations applied on ".gitmodules" at git source revisions are treated as modification, even if they are actually removal of it. If removal of ".gitmodules" is treated as modification unexpectedly, "hg convert" is aborted by the exception raised in "retrievegitmodules()" for ".gitmodules" at the git source revision removing it, because that revision doesn't have any information of ".gitmodules". This patch detects removal of ".gitmodules" at git source revisions correctly. If ".gitmodules" is removed at the git source revision, this patch records "hex(nullid)" as the contents hash value for ".hgsub" and ".hgsubstate" at the destination revision. This patch makes "getfile()" raise IOError also for ".hgstatus" and ".hgsubstate" if the contents hash value is "hex(nullid)", and this tells removal of ".hgstatus" and ".hgsubstate" at the destination revision to "localrepository.commitctx()" correctly. For files other than ".hgstatus" and ".hgsubstate", checking the contents hash value in "getfile()" may be redundant, because "catfile()" for them also does so. But this patch chooses writing it only once at the beginning of "getfile()", to avoid writing same code twice both for ".hgsub" and ".hgsubstate" separately.

templates: escape NUL bytes in jsonescape (issue4303) It's currently possible for various fields to contain NUL bytes, which are disallowed in JSON.

localrepo: document localrepo.hook()

filemerge: use 'util.ellipsis' to trim custom conflict markers correctly Before this patch, filemerge slices byte sequence directly to trim conflict markers, but this may cause: - splitting at intermediate multi-byte sequence - incorrect calculation of column width (length of byte sequence is different from columns in display in many cases) This patch uses 'util.ellipsis' to trim custom conflict markers correctly, even if multi-byte characters are used in them.

filemerge: use only the first line of the generated conflict marker for safety Before this patch, with careless configuration (missing '|firstline' filtering for '{desc}' keyword, for example), '[ui] mergemarkertemplate' can make conflict markers multiple lines. For ordinary users, advantage of allowing '[ui] mergemarkertemplate' to generate multiple lines for customizing seems to be less than advantage of disallowing it for safety. This patch uses only the first line of the conflict marker generated from '[ui] mergemarkertemplate' configuration for safety.

progress: use 'encoding.colwidth' to get column width of items correctly Before this patch, 'progress' extension applies 'len' on byte sequence to get column width of it, but it causes incorrect result, when length of byte sequence and columns in display are different from each other in multi-byte characters. This patch uses 'encoding.colwidth' to get column width of items in output line correctly, even if it contains multi-byte characters.

progress: use 'encoding.trim' to trim items in output line correctly Before this patch, 'progress' extension trims items in output line by directly slicing byte sequence, but it may split at intermediate multi-byte sequence. This patch uses 'encoding.trim' to trim items in output line correctly, even if it contains multi-byte characters.

encoding: add 'leftside' argument into 'trim' to switch trimming side

progress: use 'encoding.colwidth' to get column width of output line correctly Before this patch, 'progress' extension applies 'len' on byte sequence to get column width of it, but it causes incorrect result, when length of byte sequence and columns in display are different from each other in multi-byte characters. This patch uses 'encoding.colwidth' to get column width of output line correctly, even if it contains multi-byte characters.

progress: use 'encoding.trim' to trim output line correctly Before this patch, 'progress' extension trims output line by directly slicing byte sequence, but it may split at intermediate multi-byte sequence. This patch uses 'encoding.trim' to trim output line correctly, even if it contains multi-byte characters. "rm -f loop.pyc" before changing "loop.py" in "test-progress.t" ensures that re-compilation of "loop.py", even if "loop.py" and "loop.pyc" have same timestamp in seconds.

histedit: use 'util.ellipsis' to trim description of each changesets Before this patch, trimming description of each changesets in histedit may split at intermediate multi-byte sequence. This patch uses 'util.ellipsis' to trim description of each changesets instead of directly slicing byte sequence. Even though 'util.ellipsis' adds '...' as ellipsis when specified string is trimmed (= this changes result of trimming), this patch uses it, because: - it can be used without any additional 'import', and - ellipsis seems to be better than just trimming, for usability

util: replace 'ellipsis' implementation by 'encoding.trim' Before this patch, 'util.ellipsis' tried to avoid splitting at intermediate multi-byte sequence, but its implementation was incorrect. Internal function '_ellipsis' trims specified unicode sequence not at most maxlength 'columns in display', but at most maxlength number of 'unicode characters'. def _ellipsis(text, maxlength): if len(text) <= maxlength: return text, False else: return "%s..." % (text[:maxlength - 3]), True In many encodings, number of unicode characters can be different from columns in display. This patch replaces 'ellipsis' implementation by 'encoding.trim', which can trim string at most maxlength columns in display correctly, even though specified string contains multi-byte characters. '_ellipsis' is removed in this patch, because it is referred only from 'ellipsis'.

encoding: add 'trim' to trim multi-byte characters at most specified columns Newly added 'trim' is used to trim multi-byte characters at most specified columns correctly: directly slicing byte sequence should be replaced with 'encoding.trim', because the former may split at intermediate multi-byte sequence. Slicing unicode sequence ('uslice') and concatenation with ellipsis ('concat') are defined as function, to make enhancement in subsequent patch easier.

bugzilla: stop trying to cache setup across hook invocation In the context of standalone Hg receiving a set of incoming changes, it makes sense for the Bugzilla module to cache basic setup to avoid reconnecting to Bugzilla for each change. After processing the changes, Hg will exit and so the connection is short-lived. But this doesn't work too well when used from a long-lived environment such as hgweb or Kallithea where, for example, the connection can time out. So take the simple approach, abandon the cache and do the basic setup on each call. This fixes current problems with Kallithea.

strip: drop -b/--backup option (BC) This option had very limited utility and counterintuitive behavior and collided unfortunately with the much later -B option. Normally we would no-op such a feature so as to avoid annoying existing scripts. However, we have to weigh that against the silent misbehavior that results when users mistakenly intended to use -B: because -b takes no arg, the bookmark gets interpreted as a normal revision, and gets stripped without removing the associated bookmark, while also not backing up the revision in question. A no-op behavior or warning would only remove the latter half of the misadventure. The only users I can find of this feature were using it in error and have since stopped. The few (if any) remaining users of this feature would be better served by --no-backup.

purge: add options for deleting only files or only directories

shelve: accept '--edit' like other commands creating new changeset After this patch, users can invoke editor for the commit message by '--edit' option regardless of '--message'.

shelve: refactor option combination check to easily add new ones Before this patch, the name of a newly added option had to be added into each string that was passed to the "checkopt()" internal function: these are white-space-separated list of un-acceptable option names (= "black list" for the specified "opt"). This new option had to be added into multiple strings because each option could belong to only one action of "create", "cleanup", "delete" or "list". In addition to this redundancy, each string passed to "checkopt()" was already too long to include a new one. This patch refactors option combination check to make it easier to add a new option in a subsequent patch. New "checkopt()" only takes one action ("cleanup", "delete" or "list"), and checks whether all explicitly activated options are allowed for it or not (if specified action is activated in "opts"). The "date" entry is listed in "allowables", but commented out, because: - "date" shouldn't be checked for test checking "date" causes unexpected failure of "test-shelve.t", because "run-test.py" puts "[default] shelve = --date '0 0'" into hgrc. - explicitly listing it can advertise that ignoring it is intentional This patch doesn't choose "white list" for the specified "opt", to avoid treating global options.

merge with stable

test-extension: add check for 'hg version -v' listing enabled extensions

version: show enabled extensions (issue4209) This code is based by hg-versions extension (GPLv2) by Markus Zapke-Gruendemann <info@keimlink.de> http://mercurial.selenic.com/wiki/VersionsExtension

strip: remove bookmarks after strip succeed (issue4295) In case we have revs to strip, delete the bookmark after the strip succeeds, not beforehand as we might still abort due to dirty working directory, etc.

templater: introduce word function This function allows returning only the nth "word" from a string. By default a string is split as by Python's split() function default, but an optional third parameter can also override what string the string is split by.

committablectx: move __contains__ into workingctx This was mistakenly moved from workingctx to committablectx in 9d56a3359011. Since the method is querying the dirstate, the only logical place is for it to reside is in workingctx.

memctx: explicitly set substate to None In d2743be1bb06, memctx was changed to inherit from committablectx, this in turn added the 'substate' property to memctx. It turns out that the newcommitphase method tested for this property: def newcommitphase(ui, ctx): commitphase = phases.newcommitphase(ui) substate = getattr(ctx, "substate", None) if not substate: return commitphase Currently, memctx isn't ready to handle substates, nor removed files, so we explicitly must set substate=None to get the old behavior back. In the future, we can decide how memctx should play with substate. For now, this fixes third-party extensions and some internal code dealing with subrepos.

bookmarks: avoid deleting primary bookmarks on rebase Prior to this, doing "hg rebase -s @foo -d @" would delete @, which is obviously wrong: a primary bookmark should never be automatically deleted. This change blocks the deletion, but doesn't yet properly clean up the divergence: @ should replace @foo.

bugzilla: correct config documentation error The config documentation specifies the wrong access method in the XMLRPC+email configuration.

Merge with crew.

Merge with stable.

setup: fixed for Pythons which don't have a CFLAGS Specifically this is observed to happen on the PyPy one gets from homebrew

util.h: declare dirstateTupleType variable instead of defining it The definition is already in parsers.c This patch avoids, at least on Mac OS X 10.6.8, build issue since e250b8300e6e

test-context: add test for performing a diff on a memctx We now see the first result of all that refactoring of memctx: we can now diff against a memctx.

test-context: add test for memctx status

memctx: add _manifest implementation that computes the filenode This is an initial implementation of having a manifest for memctx that computes the hash in the same way that filenodes are computed elsewhere.

basectx: pass raw context objects to patch.diff

patch: use ctx.node() instead of bare node variable Future patches will allow patch.diff to take a basectx so node1 (and node2) could make hexfunc error out. Instead, we'll call the node function on the context object directly.

merge with stable

Added signature for changeset 2d8cd3d0e83c

Added tag 3.0.2 for changeset 2d8cd3d0e83c

merge with i18n

i18n-pt_BR: synchronized with a4b67bf1f0a5

i18n-ja: synchronized with a4b67bf1f0a5

rebase: fix bug that caused transitive copy records to disappear (issue4192) The defect was that copies were always duplicated against the target revision, rather than the first parent of the revision being rebased. This produced nominally correct results if changes were rebased one at a time (or with --collapse), but was wrong if we rebased a sequence of changesets which contained a sequence of copies.

cmdutil: add copy-filtering support to duplicatecopies In order to fix issue 4192 we need to be able to skip some copies while doing duplicatecopies.

dispatch: handle empty IOError args We occasionally throw bare IOError objects (especially in convert), don't choke on them.

repoview: do not crash when localtags refers to non existing revisions This fixes a crash that may happen when using mercurial 3.0.x. The _gethiddenblockers function assumed that the output of tags.readlocaltags() was a dict mapping tags to of valid nodes. However this was not necessarily the case. When a repository had obsolete revisions and had local tag pointing to a non existing revision was found, many mercurial commands would crash. This revision fixes the problem by removing any tags from the output of tags.readlocaltags() which point to invalid nodes. We may want to add a warning when this happens (although it might be annoying to get that warning for every command, possibly even more than once per command). A test for this problem has been added to test-obsolete.t. Without this fix the test would output: $ hg tags abort: 00changelog.i@3816541e5485: no node! [255] Instead of: $ hg tags tiptag 2:3816541e5485 tip 2:3816541e5485 visible 0:193e9254ce7e

templater: add symbol to error This patch makes it easier to debug writing templater functions by telling the user exactly what was sent to the templater.

templater: introduce startswith function This function returns a string only if it starts with a given string. It is particularly useful when combined with splitlines and/or used with conditionals that fail when empty strings are passed in to take action based on the contents of a line.

templatefilter: add splitlines function This is useful for applying changes to each line, and it's especially powerful when used in conjunction with conditionals to modify lines based on content.

serve: tidy up indent level of repository not found message

serve: inline checkrepo() that is used only when --stdio is specified Since e811b93f2cb1, --cmdserver is allowed to start without repository, so checkrepo() function is meaningless.

serve: make sure to print "listening at" message immediately If stdout is piped, status message won't be flushed until client connects to the server and access log is written to stdout. It seems bad idea to queue start-up banner of server process.

merge with stable

match: make glob '**/' match the empty string Previously, a glob pattern of the form 'foo/**/bar' would match 'foo/a/bar' but not 'foo/bar'. That was because the '**' in 'foo/**/bar' would be translated to '.*', making the final regex pattern 'foo/.*/bar'. That pattern doesn't match the string 'foo/bar'. This is a bug because the '**/' glob matches the empty string in standard Unix shells like bash and zsh. Fix that by making the ending '/' optional if an empty string can be matched.

tags: read tag info into a sorted dict (rather than into a regular dict) This lets us preserve the original tag order (as specified on the .hgtags file that is being read). This will be useful to preserve the tag order when saving the result of a successful automated .hgtags merge (which will be introduced on a future patch). There shouldn't be much impact on performance because the sortdict that _readtags returns is then used to update the alltags regular dict (which remains a regular dict).

config: move config.sortdict class into util This makes it more natural to use the sortdict class from outside config.py.

hg: add support for HGUNICODEPEDANTRY environment variable This lets us easily verify that there are no implicit conversions between unicodes and bytes in Mercurial's codebase. Based on something mpm did by hand periodically, but it kept regressing, so just open the door to running it in a buildbot.

dirstate.status: assign members one by one instead of unpacking the tuple With this patch, hg status and hg diff regain their previous speed. The following tests are run against a working copy with over 270,000 files. Here, 'before' means without this or the previous patch applied. Note that in this case `hg perfstatus` isn't representative since it doesn't take dirstate parsing time into account. $ time hg status # best of 5 before: 2.03s user 1.25s system 99% cpu 3.290 total after: 2.01s user 1.25s system 99% cpu 3.261 total $ time hg diff # best of 5 before: 1.32s user 0.78s system 99% cpu 2.105 total after: 1.27s user 0.79s system 99% cpu 2.066 total

dirstate.status: assign members one by one instead of unpacking the tuple With this patch, hg status and hg diff regain their previous speed. The following tests are run against a working copy with over 270,000 files. Here, 'before' means without this or the previous patch applied. Note that in this case `hg perfstatus` isn't representative since it doesn't take dirstate parsing time into account. $ time hg status # best of 5 before: 2.03s user 1.25s system 99% cpu 3.290 total after: 2.01s user 1.25s system 99% cpu 3.261 total $ time hg diff # best of 5 before: 1.32s user 0.78s system 99% cpu 2.105 total after: 1.27s user 0.79s system 99% cpu 2.066 total

parsers: inline fields of dirstate values in C version Previously, while unpacking the dirstate we'd create 3-4 new CPython objects for most dirstate values: - the state is a single character string, which is pooled by CPython - the mode is a new object if it isn't 0 due to being in the lookup set - the size is a new object if it is greater than 255 - the mtime is a new object if it isn't -1 due to being in the lookup set - the tuple to contain them all In some cases such as regular hg status, we actually look at all the objects. In other cases like hg add, hg status for a subdirectory, or hg status with the third-party hgwatchman enabled, we look at almost none of the objects. This patch eliminates most object creation in these cases by defining a custom C struct that is exposed to Python with an interface similar to a tuple. Only when tuple elements are actually requested are the respective objects created. The gains, where they're expected, are significant. The following tests are run against a working copy with over 270,000 files. parse_dirstate becomes significantly faster: $ hg perfdirstate before: wall 0.186437 comb 0.180000 user 0.160000 sys 0.020000 (best of 35) after: wall 0.093158 comb 0.100000 user 0.090000 sys 0.010000 (best of 95) and as a result, several commands benefit: $ time hg status # with hgwatchman enabled before: 0.42s user 0.14s system 99% cpu 0.563 total after: 0.34s user 0.12s system 99% cpu 0.471 total $ time hg add new-file before: 0.85s user 0.18s system 99% cpu 1.033 total after: 0.76s user 0.17s system 99% cpu 0.931 total There is a slight regression in regular status performance, but this is fixed in an upcoming patch.

dirstate: add dirstatetuple to create dirstate values Upcoming patches will switch away from using Python tuples for dirstate values in compiled builds. Make that easier by introducing a variable called dirstatetuple, currently set to tuple. In upcoming patches, this will be set to an object from the parsers module.

parsers: remove no longer used dirstate_unset

pack_dirstate: in C version, for invalidation set dict to what we write to disk For files written out in the last second, Mercurial used to invalidate all the stat data (state, size, mode, mtime) while persisting to disk. This included invalidating the data in the dirstate dict as well. In commit 187bf2dde7c1, this was found to be unnecessary, and Mercurial switched to invalidating only the mtime. However, in the C version of pack_dirstate the value set in the dict was still the fully invalidated one. Switch to invalidating just the mtime in the dict as well.

record: update comment to match code Commit 205599e31870 changed how newfiles get passed to commitfunc, but did not change the corresponding comment that explains this. This commit also updates this comment.

hg: update newly added listdir function of vfs in clone This change invokes os.listdir() via newly added vfs function in clone.

hg: use vfs functions in clone This change applies vfs functions in clone. destvfs.exists will invoke os.path.lexists via lexists of vfs to check for broken symbolic links.

hg: update util.writefile method to use write with vfs in share This patch replaces util.writefile() to use destvfs.write().

hg: update to use vfs functions in shared destination repository This patch uses destvfs with base as .hg directory in shared destination repository to update filesystem function with vfs. Some methods are changed to use vfs functions with destvfs. branch 'default' HG: bookmark '@' HG: changed mercurial/hg.py

hg: use vfs functions in destination repository with share In this patch, dstwvfs is added to use vfs functions in working directory of destination shared repository. Existing filesystem operations are updated to use vfs functions through dstwvfs.

vfs: add listdir for os.listdir in vfs This patch adds listdir() in vfs to use os.listdir().

templater: restore use of callable() since it was readded in Python 3.2

hook: restore use of callable() since it was readded in Python 3.2

help: restore use of callable() since it was readded in Python 3.2

extensions: restore use of callable() since it was readded in Python 3.2

fancyopts: restore use of callable() since it was readded in Python 3.2

gendoc: restore use of callable() since it was readded in Python 3.2

check-seclevel: restore use of callable() since it was readded in Python 3.2

check-code: drop ban on callable() which was restored in Python 3.2 A followup will restore use of callable() in place of the awkward hasattr() construction we were using to be one step closer to Python 3.

diff: add nobinary config to suppress git-style binary diffs

branchmap: don't use ui.warn for debug message

branch: add debug message for branch cache write failure

record: define inferrepo in command decorator

mq: define inferrepo in command decorator

largefiles: define inferrepo in command decorator

keyword: define inferrepo in command decorator

hgk: define inferrepo in command decorator

graphlog: define inferrepo in command decorator

extdiff: define inferrepo in command decorator

children: define inferrepo in command decorator

churn: define inferrepo in command decorator

commands: define inferrepo in command decorator

cmdutil: support inferrepo in command decorator

keyword: define optionalrepo in command decorator

commands: define optionalrepo in command decorator

cmdutil: add optionalrepo argument to command decorator

tests: define norepo in command decorator

share: define norepo in command decorator

mq: define norepo in command decorator

largefiles: define norepo in command decorator

convert: define norepo in command decorator

commands: define norepo in command decorator

commands: add norepo argument to command decorator Since decorators are evaluated at module load time and since the @command decorator imports commands, the norepo variable (along with its friends) may not be declared yet. These variables are now declared before @command usage to ensure they are present.

cmdutil: better document command()

convert: update the transplant, rebase and graft references in 'extra' This change allows the origin() and destination() revsets to yield the same results in the new and old repos after a conversion. Previously, nothing would be listed for queries in the new repo. Like the SHA1 updates to the commit messages, this is only operational when the 'convert.hg.saverev=True' option is specified. If the old reference cannot be found, it is left as-is. It seems slightly better to leave stale evidence of the graft/transplant/rebase than to eliminate it entirely.

merge with stable

run-tests: hold iolock across diff/prompt when interactive Otherwise diff may get separated from the corresponding prompt by other threads. This required moving the interactive prompting from one helper method to another.

bookmarks: improve the bookmark help (issue4244)

push: restore contents of HG_URL for hooks (issue4268)

tests: ignore missing file with PYTHONDONTWRITEBYTECODE (issue4239)

hgweb: avoid initialization race (issue4280)

p4: accept changesets with no description (issue4282)

merge with stable

test-run-tests.t: test the --nodiff option

test-run-tests.t: test the --interactive option

run-tests: fixes the '--interactive' option error This patch fixes a regression recently introduced by a refactoring. Previously when failure occurs while testing with '--interactive' was enable, it didn't prompt user by asking whether he wants to accept this failure changes or not. This was happening beacuse of the 'if' condition if ret or not self._options.interactive or \ not os.path.exists(test.errpath): Everytime failure occurs, this condition gets true and returns back even when '--interactive' is enabled. This condition don't led the function to execute further, which consist the '--interactive' functionality. Now, on failure with '--interactive' enabled, it prompts user whether he wants to accepts failure changes or not. If yes then test gets passed and returns true, else test gets failed. On every failure, results gets stored in "self.failures.append((test, reason))" But if failure changes accepted by user then test must get "pop out" from failed test list.