Merge with stable

dirstate: ignore symlinks when fs cannot handle them (issue1888) When the filesystem cannot handle the executable bit, we currently ignore it completely when looking for modified files. Similarly, it is impossible to set or clear the bit when the filesystem ignores it. This patch makes Mercurial treat symbolic links the same way. Symlinks are a little different since they manifest themselves as small files containing a filename (the symlink target). On Windows, these files show up as regular files, and on Linux and Mac they show up as real symlinks. Issue1888 presents a case where the symlink files are better ignored from the Windows side. A Linux client creates symlinks in a working copy which is shared over a network between Linux and Windows clients. The Samba server is helpful and defererences the symlink when the Windows client looks at it. This means that Mercurial on the Windows side sees file content instead of a file name in the symlink, and hence flags the link as modified. Ignoring the change would be much more helpful, similarly to how Mercurial does not report any changes when executable bits are ignored in a checkout on Windows. An initial checkout of a symbolic link on a file system that cannot handle symbolic links will still result in a regular file containing the target file name as its content. Sharing such a checkout with a Linux client will not turn the file into a symlink automatically, but 'hg revert' can fix that. After the revert, the Windows client will see the correct file content (provided by the Samba server when it follows the link on the Linux side) and otherwise ignore the change. Running 'hg perfstatus' 10 times gives these results: Before: After: min: 0.544703 min: 0.546549 med: 0.547592 med: 0.548881 avg: 0.549146 avg: 0.548549 max: 0.564112 max: 0.551504 The median time is increased about 0.24%.

mq/qqueue: enable bash completion Return the list of available queues when completion is attempted on qqueue.

mq/qqueue: --list does not print (active) with --quiet For scripting purposes, it can be convenient to get a simple listing of available queues, without indication of the active one. --quiet documentation change removed by Patrick Mézard.

Merge with stable

templatefilters: make json filter handle multibyte characters correctly It aims to fix javascript error of hgweb's graph view in Japanese 'cp932' encoding. 'cp932' contains multibyte characters ending with '\x5c' (backslash), e.g. '\x94\x5c' for Japanese Kanji 'Noh'. Due to json filter escapes '\' to '\\', multibyte string ending with '\x5c' is translated to "xxx\", resulting javascript parse error on a web browser. This patch changes json() to pass unicode to jsonescape(). Unicode decoding error handler changed to 'replace' by Patrick Mézard.

check-code: added a check for calls to the builtin cmp function

manifest: removed usage of the global cmp function Py3k doesn't have a global cmp() function, making this call problematic in the py3k port. Also, calling cmp() here is not necessary, since we only want to know if the two values are equal. A check for equality perfect in this case and this patch does that.

tests: run check-code.py on working directory files If we insist on Mercurial sources to pass check-code.py, let automate the process and make it part of the tests. Objections?

minor style fix: hgext/rebase.py:157 -- line too long found by a run of check-code

zsh completions: add qpush --move option

Merge with stable

chunkbuffer: use += rather than cStringIO to reduce memory footprint This significantly refactors the read() loop to use a queue of chunks. The queue is alternately filled to at least 256k and then emptied by concatenating onto the output buffer. For very large read sizes, += uses less memory because it can resize the target string in place.

httprepo: decompress stream incrementally to reduce memory usage

verify: initialize rp variable in case we hit out of memory

merge: drop reference to file contents after write This reduces memory usage on large consecutive gets.

revlog: drop cache after use to save memory footprint If we reconstruct back to back large versions, we need to drop the cache first to avoid doubling memory usage.

verify: reduce memory footprint when unpacking files By never holding a reference to the unpacked string, we avoid holding two consecutive large files in memory.

verify: report exceptions that don't have a str() method

color: fix test-diff-color breakage

hgcia/color: remove star imports This plays nicer with demandimport and allows pyflakes to detect undefined names.

hgfixes: added a fixer that makes bytes to be formatted correctly This patch implement a fixer that replaces all calls to the '%' when bytes arguments are used to a call to bytesformatter(), a function that knows how to format byte strings. As one can't be sure if a formatting call is done when only variables are used in a '%' call, these calls are also translated. The bytesformatter, in runtime, makes sure to return the "raw" % operation if that's what was intended.

py3kcompat: added a "compatibility layer" for py3k This patch adds some ugly constructs. The first of them is bytesformatter, a function that formats strings like when '%' is called. The main motivation for this function is py3k's strange behavior: >>> 'foo %s' % b'bar' "foo b'bar'" >>> b'foo %s' % b'bar' Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for %: 'bytes' and 'bytes' >>> b'foo %s' % 'bar' Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for %: 'bytes' and 'str' In other words, if we can't format bytes with bytes, and recall that all mercurial strings will be converted by a fixer, then things will break badly if we don't take a similar approach. The other addition with this patch is that the os.environ dictionary is monkeypatched to have bytes items. Hopefully this won't be needed in the future, as python 3.2 might get a os.environb dictionary that holds bytes items.

hgfixes: add a fixer to convert plain strings to bytestrings This patch implements a 2to3 fixer that converts all plain strings in a python source file to byte strings syntax. Example: foo = 'Normal string' would become foo = b'Normal string' The motivation behind this fixer can be found in http://selenic.com/pipermail/mercurial-devel/2010-June/022363.html or, in other words: the current hg source assumes that _most_ strings are "meant" to be byte sequences, so it makes sense to make the convertion implemented by this patch. As mentioned above, not all mercurial modules want to use strings as bytes, examples include i18n (which uses unicode), and demandimport (in py3k, module names are normal strings, thus unicode, and there's no need for a convertion). Therefore, these modules are blacklisted in the fixer. There are also a few functions that can take only unicode arguments, thus the convertion shouldn't be done for those.

revlog: add shallow header flag REVLOGSHALLOW header flag to mark revlog as shallow. The _shallow attribute of the revlog is used to check if the header flag is set.

revlog: add punched revision flag index flag to identify a revision as punched, i.e. it contains no data. REVIDX_PUNCHED_FLAG = 2, is used to mark a revision as punched. REVIDX_KNOWN_FLAGS is the accumulation of all index flags.

tests: unify test-tags

tests: unify test-empty

tests: unify test-basic

tests: basic support for unified tests