Added tag 3.6.3 for changeset ea389970c084

merge with i18n

i18n-ja: synchronized with ca8ada499529

merge: while checking for unknown files don't follow symlinks (issue5027) Previously, we were using Python's native 'os.path.isfile' method which follows symlinks. In this case, since we're operating on repo contents, we don't want to follow symlinks. There's a behaviour change here, as shown by the second part of the added test. Consider a symlink 'f' pointing to a file containing 'abc'. If we try and replace it with a file with contents 'abc', previously we would have let it though. Now we don't. Although this breaks naive inspection with tools like 'cat' and 'diff', on balance I believe this is the right change.

merge with stable

tests: add test-check-execute.t Try to prevent people from adding files with incorrect execute bits

parents: correct help revset replacements Implementing `hg parents -r REV FILE` correctly is hard. The output can be 0, 1, or 2 revs. First, you can't use parents(), because it sorts its output... Consider: echo $a echo par@rev: `hg log -r "parents($a)" -q` echo p12@rev: `hg log -r "p1($a)+p2($a)" -q` echo parents: `hg parents -q -r $a` (Merge 1 into 0) 3 par@rev: 0:d9612eafe8ec 1:070fe4290d06 p12@rev: 0:d9612eafe8ec 1:070fe4290d06 parents: 0:d9612eafe8ec 1:070fe4290d06 (Merge 4 into 5) 6 par@rev: 4:db73392995c3 5:c26e7dd67644 p12@rev: 5:c26e7dd67644 4:db73392995c3 parents: 5:c26e7dd67644 4:db73392995c3 (Merge 7 into 8) 9 par@rev: 7:d84f47462f70 8:9597bcab36e0 p12@rev: 8:9597bcab36e0 7:d84f47462f70 parents: 8:9597bcab36e0 7:d84f47462f70 You also can't use parents or/p1/p2 alone with a set, as in: -r "parents(::REV and file(FILE))" -r "parents(::REV - REV and file(FILE))" ... because each will return all parents for each candidate revision, and the :: gives too many candidates. Thus, we need a max and a p1/p2. Also, anything of this form: max(::REV - REV and file(FILE)) ... is wrong, because max will return only one revision, and for a proper parents, you need to return two occasionally. Lastly, it doesn't help that `hg parents -r REV FILE` is buggy due to a quirk in filelogs. Here's a repository to consider when evaluating whether your revset is correct: $ hg log -G --template '{rev} {files}\n'; @ 10 a | o 9 a b |\ | o 8 a | | o | 7 a | | +---o 6 b | |/ | o 5 b | | o | 4 b | | +---o 3 | |/ +---o 2 | |/ | o 1 b | o 0 a revs 4 and 5 create a conflict. The conflict is resolved in the same way by both 6 and 9. You would hope that parents around 9/10 would point to 9, but `hg parents` will point to 6 due to the aforementioned bug. Here's the winning solution test script and its output. echo $a; echo rp12-max: `hg log -r "max(::p1($a) and file(b)) + max(::p2($a) and file(b))" -q` 2> /dev/null; echo expected: `hg parents -q -r $a b` 2> /dev/null; Note that for 10, the output differs, but again, this is because of the aforementioned bug. The rp12-max output is "correct", whereas "expected" is just an unfortunate bug. The abort output is due to something else. I'm not sure why someone thought it was important to abort to stdio instead of stderr, but that's really not my problem here. 10 rp12-max: 9:184ebefc2fce expected: 6:dd558142b03f 9 rp12-max: 5:c26e7dd67644 4:db73392995c3 expected: 5:c26e7dd67644 4:db73392995c3 8 rp12-max: 5:c26e7dd67644 expected: 5:c26e7dd67644 7 rp12-max: 4:db73392995c3 expected: 4:db73392995c3 6 rp12-max: 5:c26e7dd67644 4:db73392995c3 expected: 5:c26e7dd67644 4:db73392995c3 5 rp12-max: 1:070fe4290d06 expected: 1:070fe4290d06 4 rp12-max: abort: 'b' not found in manifest! expected: 3 rp12-max: 1:070fe4290d06 expected: 1:070fe4290d06 2 rp12-max: 1:070fe4290d06 expected: 1:070fe4290d06 1 rp12-max: abort: 'b' not found in manifest! expected: 0 rp12-max: abort: 'b' not found in manifest! expected:

run-tests: avoid double counting server fails

commandserver: reset state of progress bar per command A progress bar is normally disabled in command-server session, but chg can enable it. This patch makes sure the last-print time is measured per command. Otherwise, progress.delay could be ineffective if a progbar was loaded before forking worker process. This patch is corresponding to the following change. https://bitbucket.org/yuja/chg/commits/2dfe3e90b365

commandserver: do not set nontty flag if channel is replaced by a real file This prepares for porting the chg server. In chg, a server receives client's stdio over a UNIX domain socket to override server channels. This is because chg should behave as if it is a normal hg command attached to tty. "nontty" is not wanted. This patch is corresponding to the following change. This doesn't test the identity of "cin" object because the current version of chg reopens stdio to apply buffering mode. https://bitbucket.org/yuja/chg/commits/c48c7aed5fc0

run-tests: report missing feature for skipped tests

paths: do not process default-push as pushurl of default path (issue5000) It didn't work because "default-push" and "default" are independent named items. Without this patch, "hg push default" would push to "default-push" because paths["default"].pushloc was overwritten by "default-push". Also, we shouldn't ban a user from doing "hg push default-push" so long as "default-push" item is defined, not "default:pushurl". Otherwise, he would be confused by missing "default-push" path. Tests are included in a patch for stable branch.

push: specify default-push and default as fallback paths The next patch will remove the "default-push" hack from ui.paths so that ui.paths["default"].pushurl can be different from "default-push".

paths: make getpath() accept multiple defaults This is necessary to handle "default-push" and "default" as fallback items. We can't apply the same rule as "default:pushurl" because "default-push" is a valid named path. This series is for default branch. I have a simpler patch for stable.

check-code: improve test-check-code error diffs Whenever check-code finds something wrong, the diffs it generated were fairly hard to read. The problem is that check-code before this change would list files that were white listed using no- check- code but without a glob marker. Whereas, the test-check-code.t expected output has no-che?k-code (glob) in order to avoid having itself flagged as a file to skip. Thus, in addition to any lines relating to things you did wrong, all of the white-listed files are listed as changed. There is no reason for things to be this painful. This change makes the output from check-code.py match the expected output in test-check-code.t

destutil: use scmutil.revrange for desthistedit (issue5001) This allows user aliases to be expanded. It also prevents the user-provided revset from being treated as a revset expression.

pull: make a single call to obsstore.add (issue5006) Prior to this, a pull of 90k markers (already known locally!) was making about 2000 calls to obsstore.add, which was repeatedly building a full set of known markers (in addition to other transaction overhead). This quadratic behavior accounted for about 50 seconds of a 70 second no-op pull. After this change, we're down to 20 seconds. While it would seem simplest to just cache the known set for obsstore.add, this would also introduce issues of correct cache invalidation. The extra pointless transaction overhead would also remain.

tests: Solaris diff -U also emits "No differences encountered" This came up before, but the tests in check-code.py don't find -U (only -u) and they don't work when the diff is inside a shell function. This fixes the offending tests and beefs up check-code.py.

test-glog: avoid check-code violation after next patch Danek's patch will help us avoid spurious test breakages on Solaris. This test wasn't broken on Solaris because of the grep(1) use, but it will upset check-code. Use cmp to silence check-code since it doesn't matter.

push: restore old behavior of default-push (issue5000) This effectively backs out dceaef70e410 and 10917b062adf. We can't handle "default-push" just like "default:pushurl" because it is a stand-alone named path. Instead, I have two ideas to work around the issue: a. two defaults: getpath(dest, default=('default-push', 'default')) b. virtual path: getpath(dest, default=':default') (a) is conservative approach and will have less trouble, but callers have to specify they need "default-push" or "default". (b) generates hidden ":default" path from "default" and "default-push", and callers request ":default". This will require some tricks and won't work if there are conflicting sub-options valid for both "pull" and "push". I'll take (a) for default branch. This patch should NOT BE MERGED to default except for tests because it would break handling of "pushurl" sub-option.

hgweb: fixed invalid atom-log feed url in file log page currently "subscribe to atom feed" link in file log page is as follows. /atom-log/[revision]/[file] This is invalid, because we could not get newer commit feed than [revision]. To fix this, atom-log feed url should be the following style. atom-log/tip/[file]

repair: improves documentation of strip regarding locks This patch adds a comment making it clear that we should hold a lock before calling repair.strip. The wording is the same than what we have for obsolete.createmarkers

help: remove stray double spaces from config help

help: clarify that the config hook priority prefix includes a period

monoblue: correct feed links on /branches, /tags and /bookmarks

gitweb: describe feed type in links on /branches, /tags and /bookmarks

gitweb: link to the correct feeds from help pages

histedit: handle exceptions from node.bin in fromrule

histedit: limit cleanup of histedit-last-edit.txt to success

histedit: use parse-error exception for parsing

test-histedit-edit: test histedit with dirty repo

histedit: limit mentioning histedit-last-edit.txt Before histedit-last-edit.txt would be mentioned for any failure. After, it should only be mentioned for failures relating to user input.

histedit: check fold of public change during verify

histedit: pass previous action to verify

annotate: mention that -n is suppressed in help

test-bundle2-format: force gc so a GeneratorExit will be thrown PyPy has looser semantics than CPython for when a generator's close method will be called. Forcing the gc causes it to be called at the right moment.

test-bad-extension: account for PyPy/CPython error difference

demandimport: update obsolete comment

demandimport: add support for PyPy PyPy's implementation of __import__ differs subtly from that of CPython. If invoked without a name or fromlist, it throws an ImportError, whereas CPython returns a reference to the level-appropriate importing package. Here, we achieve the same behaviour by hand.

test-demandimport: ensure that relative imports are deferred This adds a test not just at our local "top level" (the mercurial package), but also one level deeper (mercurial.hgweb).

histedit: don't bother with cPickle, demand-load pickle We're unlikely to ever need the pickle module, so there's no good reason to force loading of its faster cousin.

merge with stable

crecord: stop raising error.Abort if curses is not found (issue5008) On some servers, python curses support is disabled. This patch not only fixes that but provides a fallback on other machines (e.g. Windows) when curses is not found. The previous code was actually flawed logic and relied on wcurses throwing an ImportError which demandimport wouldn't throw. So, this patch also fixes that problem.

cmdutil: use crecordmod.checkcurses Instead of blindly trusting the user's experimental.crecord, we use checkcurses to abstract that logic so that we can handle the case where python was not built with curses.

crecord: ensure that curses is False if not imported This provides no functional change but makes the next two patches easier to review.

crecord: add helper function to determine if we should use curses To fix issue5008 properly, we need a helper function to determine if curses is imported and also if the user has enabled the experimental flag.

crecord: use try/except for import of curses Not only does this improve fragility with 'if os.name == ...' it will help future patches enable the behavior to fallback to use plain record when curses is unavailable (e.g. python compiled without curses support).

histedit: only use pickle if not using the modern save format This avoids a case where PyPy's cPickle module throws a more confusing error than CPython's.

perf: close transaction in perffncachewrite This fixes a bug, and brings CPython behaviour on this test into line with PyPy.

hbisect: use tryreadlines to load state This closes the file handle after reading, which stops PyPy from leaking open file handles and thus failing test-bisect3.t.

eol: make output stable This eliminates a divergence in behaviour between PyPy and Python.

exchange: use absolute_import

localrepo: use absolute_import

httpconnection: use absolute_import

import-checker: force 'logging' to stdlib module Not sure why this isn't getting picked up. Perhaps we're not handling stdlib paths that have sub-modules? Something to investigate for another day...

test-install: embed wix namespace for Python 2.6 compatibility According to doc, the syntax is "{uri}tag". https://docs.python.org/2.7/library/xml.etree.elementtree.html#parsing-xml-with-namespaces

fileset: add hint for list error to use or

revset: add hint for list error to use or

error: make ParseError inherit from HintException

error: refactor OutOfBandError to inherit from HintException

phases: mention how to make secret commits in help

mq: use fallback patch name if no alpha-numeric in summary line (issue5025) Before this patch, "hg qimport -r REV" fails, if the summary line of description of REV doesn't contain any alpha-numeric bytes. In this case, all bytes in the summary line 'title' are dropped from 'namebase' by the code path below. namebase = re.sub('[\s\W_]+', '_', title.lower()).strip('_') 'makepatchname()' immediately returns this empty string as valid patch name, because patch name conflicting against empty string never exists. Then, "hg qimport -r REV" is aborted at creation of patch file with empty filename. This situation isn't so rare. For example, ordinary texts in Japanese often consist of non alpha-numeric bytes in UTF-8. This patch makes 'makepatchname()' use fallback patch name if the summary line of imported revision doesn't contain any alpha-numeric bytes.

osutil: c_ssize_t is only available in Python >= 2.7

check-seclevel: pass a ui to the extension loader Without this, if an import error occurs (as with pypy), the attempt to report it fails since ui is None.

check-seclevel: add a --debug option This will make it possible to get semi-meaningful tracebacks if an import error occurs. Why care? Trying to run this script under pypy currently fails, but the true error is obscured.

test-run-tests: glob away a --debug run difference on Windows This internal test is piped through 'grep -v pwd' to eliminate the pwd alias set when running with MSYS. Unfortunately, the '.' from the successful run of the prior internal test precedes the pwd alias for the next test on the same line, so grep filters out '.' too, except for the final test. It also looks like there may be a bug with --debug: the output of the internal test that had this diff says 2 ran, 0 failed (one test being test-failure.t), but if --debug is omitted from the internal test, then it says 2 ran, 1 failed. With this longstanding issue fixed, the test suite finally runs cleanly on Windows (except subrepo merge documented in issue 4988), with 88 skips. \o/

import-checker: force 'ctype.util' to stdlib module Not having this caused warnings on Windows: mercurial/pure/osutil.py:12: stdlib import follows local import: os mercurial/pure/osutil.py:13: stdlib import follows local import: socket mercurial/pure/osutil.py:14: stdlib import follows local import: stat mercurial/pure/osutil.py:15: stdlib import follows local import: sys

keepalive: use absolute_import

context: use absolute_import

lsprofcalltree: use absolute_import

byterange: use absolute_import There were a lot of imports scattered around this file. They have been consolidated at the top of the file where they belong.

dirstate: use absolute_import

manifest: use absolute_import

pvec: use absolute_import

ui: remove unreachable code

mercurial.spec: remove execute bit

tests: add execute bit and fix shbang line

i18n: add execute bit to check-translation.py

doc: add execute bit and fix shbang line for gendoc.py

contrib: add execute bit for fixpax.py

contrib: add execute bit for check-py3-compat.py

perf: make start revision configurable for perfrevlog This will help isolate performance characteristics of delta chains.

perf: use standard arguments for perfrevlog We have a convention of using -c|-m|FILE elsewhere for reading from revlogs. Use it for `hg perfrevlog`. While I was here, I also added a docstring to document what this command does, as "perfrevlog" is ambiguous.

test-extension: do not depend on demandimport (issue5012) When demandimport is disabled, the test will fail because extroot/foo.py uses import outside PYTHONPATH. This is bad since we have a PyPy migration plan and it does not support demandimport. Fix by setting PYTHONPATH.

commands: split notes into note containers

remove: quote --force in never deletes note Split Note into a note container

import: reword no hunks partial note Split Note into a note container

merge: reword help to use See help resolve

py3compat: use absolute_import

patch: use absolute_import

mdiff: use absolute_import

scmposix: use absolute_import

scmutil: use absolute_import

scmwindows: use absolute_import

store: use absolute_import

help: use absolute_import

test-commit-interactive: updates for the no-execbit case This goes with c84a07530040 and 14b184263846.

test-fileset: conditionalize output with symlink

commands: the first word of each note should be capital or `hg`

revlog: avoid string slice when decompressing u* chunks Revlog chunks can be stored uncompressed. If the first byte of the raw data is \0, we store the data as is. Else we prefix it with 'u'. Before, we performed a string slice to strip out the 'u' prefix. With this patch, we use a buffer to avoid an extra memory copy and associated garbage collection overhead. I was unable to verify any performance impact of this patch. For both mozilla-central and the hg repos, the number of manifest revisions with 'u' prefixes is very small - under 1%. So this change likely isn't called enough to have an impact on manifest reading. However, the reasoning behind this change is solid, so it should be safe.

osutil: implement pure version of recvfds() for PyPy This is less portable than the C version, but PyPy can't load CPython extensions. So for now, this will be used on PyPy. I've tested it on Linux amd64 and Mac OS X.

osutil: implement recvmsg() of SCM_RIGHTS for chg command server It will be used to attach client's stdio files to a background chg command server. The socket module of Python 2.x doesn't provide recvmsg(). This could be implemented by using ctypes, but it would be less portable than the C version because the handling of socket ancillary data heavily depends on preprocessor. Also, some length fields are wrongly typed in the Linux kernel.

cleanup: back out performance hacks amended into previous commit

commands: consistently indent notes 3 spaces most notes have 3 spaces for indentation, these had 2...

perf: add perfrevlogrevision As part of investigating performance improvements to revlog reading, I needed a mechanism to measure every part of revlog reading so I knew where time was spent and how effective optimizations were. This patch implements a perf command for benchmarking the various stages of reading a single revlog revision. When executed against a manifest revision at the end of a 30,000+ long delta chain in mozilla-central, the command demonstrates that ~80% of time is spent in zlib decompression.

commands: use revlog._deltachain in debugdeltachain We have a nice API now. Use it. This does mean we introduce an extra index lookup for each revision. Considering this is a debug command, the overhead should be acceptable. We could add the chain size to revlog._deltachain(). However, that feels like avoidable overhead.

revlog: refactor delta chain computation into own function This code is already written in multiple locations. While this code needs to be fast and extracting it to its own function adds overhead, code paths reading delta chains typically read, decompress, and do binary patching on revlog data from the delta chain. This other work (especially zlib decompression) almost certainly accounts for a lot more time than the overhead of introducing a Python function call. So I'm not worried about the performance impact of this change.

perf: call clearcaches() in perfmanifest The old code only partially cleared the caches. Now that we have a comprehensive method for wiping all caches, let's call it. This appears to introduce a marginal regression in `hg perfmanifest` on mozilla-central. This is good because the new result is more accurate since caches aren't being used.

manifest: implement clearcaches() The manifest implements its own caches in addition to revlog's. Extend the base clearcaches() to wipe these as well.

revlog: make clearcaches() more effective clearcaches() was added several years ago in e8d37b78acfb as part of implementing a perf command. Since revlog instances have many caches and since the spirit of this mostly unused method is to facilitate performance testing, I think it's appropriate for all the revlog's caches to get cleared when it is called.

fileset: detect unintentional existing() invocation at runtime A fileset predicate can invoke 'matchctx.existing()' successfully, even if it isn't marked as "existing caller". It is aborted only in some corner cases: e.g. there were one deleted file in the working directory (see 8a0513bf030a for detail). This patch makes 'matchctx.existing()' invocation abort if not '_existingenabled', which is true only while "existing caller" running. After this changes, non-"existing caller" predicate function is aborted immediately, whenever it invokes 'matchctx.existing()'. This prevent developer from forgetting to mark a predicate as "existing caller". BTW, unintentional 'matchctx.status()' invocation can be detected easily without any additional trick like this patch, because it returns 'None' if a predicate isn't marked as "status caller", and referring field (e.g. '.modified') of it is always aborted.

fileset: use set instead of list to mark predicates for efficiency (API) This reduces cost of examining whether given predicate calls 'matchctx.status()' or 'matchctx.existing()' in 'getfileset()' at runtime. This kind of examination is used also in subsequent patch, which detects unintentional 'matchctx.existing()' invocation per each predicate evaluation.

fileset: use decorator to mark a predicate as "existing caller" This can localize changes for adding (or removing) an "existing caller" predicate function in source code.

fileset: use decorator to mark a predicate as "status caller" Before this patch, predicates calling 'matchctx.status()' are listed up by immediate list value in 'getfileset()'. This prevents 3rd party extensions from adding specific predicate calling 'matchctx.status()'. This uses decorator to mark a predicate as "status caller". This can also localize changes for adding (or removing) a "status caller" predicate function in source code.

fileset: use decorator to mark a function as fileset predicate Using decorator can localize changes for adding (or removing) a fileset predicate function in source code. It is also useful to pick predicates up for specific purpose. For example, subsequent patches marks predicates as "call status" or "use existing" via decorator. To avoid (1) redundancy between "predicate name" and (the beginning of) help document, and (2) accidental typo of help document, this patch also makes decorator put predicate declration into the beginning of help.

fileset: treat encoding and eol as the predicate calling _existing Before this patch, predicate function 'encoding' and 'eol' aren't listed up in '_existingcallers', even though they invoke 'existing()'. This causes unexpected failure of these predicate, if there is a (manually) deleted file in the working directory. 8a0513bf030a and 3ce3f2b059a1 seem to overlook putting already existing 'encoding' or newly introduced 'eol' into '_existingcallers'. This patch also changes order of fileset "eol(unix)" output in test, because "existing caller" predicates show "A(dded)" files before "C(lean)" ones.

rebase: mention conflict in documentation instead of merge

rebase: simplify documentation about heads

rebase: simplify documentation about --keep Also include a warning about bookmarks

rebase: simplify documentation about selecting commits to rebase