merge with stable

Added signature for changeset ced632394371

Added tag 3.2 for changeset ced632394371

setdiscovery: limit the size of all sample (issue4411) Further digging on this issue show that the limit on the sample size used in discovery never works for heads. Here is a quote from the code itself: desiredlen = size - len(always) if desiredlen <= 0: # This could be bad if there are very many heads, all unknown to the # server. We're counting on long request support here. The long request support never landed and evolution make the "very many heads, all unknown to the server" case quite common. We implement a simple and stupid hard limit of sample size for all query. This should prevent HTTP 414 error with the current state of the code.

hook: protect commit hooks against stripping of temporary commit (issue4422) History rewriting commands like histedit tend to use temporary commits. They may schedule hook execution on these temporary commits for after the lock has been released. But temporary commits are likely to have been stripped before the lock is released (and the hook run). Hook executed for missing revisions leads to various crashes. We disable hooks execution for revision missing in the repo. This provides a dirty but simple fix to user issues.

mq: do not call [0] on revset The __getitem__ method have been removed. The "first" method is to be used instead. Test have been extended to test this code path.

addset: fix `first` and `last` on sorted addset (issue4426) The lazy sorting were not enforced on addset. This was made visible through MQ.

clone: properly mark branches closed with --uncompressed (issue4428) On streaming clone, we were priming the local branch cache with the remote branchmap, without checking which heads were closed. This fixes an issue introduced in: changeset: 17740:e6067bec18da user: Tomasz Kleczek <tomasz.kleczek@fb.com> date: Wed Oct 03 13:19:53 2012 -0700 summary: branchcache: fetch source branchcache during clone (issue3378) that was exposed in 2.9 by: changeset: 20192:38fad5e76ee8 user: Brodie Rao <brodie@sf.io> date: Mon Sep 16 01:08:29 2013 -0700 summary: branches: simplify with repo.branchmap().iterbranches()

Makefile: update .PHONY Based on sed -n 's/^\([a-z0-9-]*\):\(\s.*\)\?$/\1/gp' Makefile | xargs echo add check, check-code, update-pot, some packaging targets

buildrpm: fix use of invalid $PLATFORM in mercurial.repo

debuglocks: add missing usage summary

help: don't crash on help for 'sections' with multiple '.'

help: fix config description of ui.reportoldssl

i18n-ja: synchronized with f3baaeffe841

i18n-pt_BR: synchronized with 0b7853f969ac

merge with i18n

i18n-ja: synchronized with 5459b30aa498

clone: fix copying bookmarks in uncompressed clones (issue4430) ef62c66bee1b broke bookmarks getting copied during uncompressed clones. Since most of the pull logic has been moved into exchange.py, lets just call exchange.pull to fix up the repo with the latest bits after the streaming clone has bootstrapped the repo. This keeps us from having to duplicate the bookmark logic.

revset: don't recreate matcher for every revision The matcher variable 'm' in checkstatus() is reset to None on each call, so the caching of the matcher no longer happens as it was intended. This seems to be a regression in ed7b674824a3 (revset: added lazyset implementation to checkstatus, 2014-01-03). Fix by moving the cached matcher into the enclosing function so it's actually cached across calls. This speeds up hg log -r 'modifies(mercurial/context.py)' >/dev/null from 7.5s to 4s. Also see similar fix in f2aeff8a87b6 (revset: avoid recalculating filesets, 2014-10-22).

help: use ":hg:`command`" instead of incorrect ":hg:'command'" notation

i18n: add i18n comment to error messages of filesets predicates

i18n: add i18n comment to error messages of template functions

help: refer ":merge3" instead of "internal:merge3" According to warning message (introduced by 38e0363dcbe0) in filemerge.py, the former should be used as official name.

i18n: make hint message of exception translatable

help: use "hg files" instead of "hg locate" in "hg help filesets" The latter command is already deprecated.

files: refer "hg help filesets" instead of "hg help revsets" in help text "specifying FILE patterns" should refer the former.

tests: silence output race in test-run-tests.t

merge with i18n

i18n-pt_BR: synchronized with 8d43c6bb38c0

i18n-pt_BR: synchronized with c312ef382033

transaction: only generate file when we actually close the transaction Before this change, the file were written for every call to `tr.close()` exposing data to reader far too early.

transaction: extract file generation into its own function We extract the code generating files into its own function. We are about to move this code around to fix a bug. We'll need it in a function soon to reuse it for "pending" logic. So we move the code into a function instead of moving it twice.

amend: abort early if no username is configured with evolve enabled (issue4211) Amend will reuse the original username if a new one is not provided with -U, but obsolete.createmarkers() only considers ui.username() for the obsolete marker's metadata. Allowing the metadata field to be spoofed seems to defeat the point of the field in the first place. This covers 'evolve amend' and 'ci --amend' with evolve enabled. Without this, the transaction aborts but the parent changeset is set to -1. The corresponding test will be added to evolve separately.

revset: fix O(2^n) perf regression in addset hg log -r 1 ... -r 100 was never returning due to a regression in the way addset computes __nonzero__. It used 'bool(self._r1 or self._r2)' which required executing self._r1.__nonzero__ twice (once for the or, once for the bool). hg log with a lot of -r's happens to build a one sided addset tree of N length, which ends up being 2^N performance. This patch fixes it by converting to bool before or'ing. This problem can be repro'd with something as simple as: hg log `for x in $(seq 1 50) ; do echo "-r $x "; done` Adding '1 + 2 + ... + 20' to the revsetbenchmark.txt didn't seem to repro the problem, so I wasn't able to add a revset benchmark for this issue.

tests: add missing glob for Windows

test-convert-svn-sink: properly isolate symlink section This was fixed earlier by moving all the symlink bits to a section to the end of the file, but then it was broken (by the same person) by adding more tests at the end.

util.fspath: use a dict rather than a linear scan for lookups Previously, we'd scan through the entire directory listing looking for a normalized match. This is O(N) in the number of files in the directory. If we decide to call util.fspath on each file in it, the overall complexity works out to O(N^2). This becomes a problem with directories a few thousand files or larger. Switch to using a dictionary instead. There is a slightly higher upfront cost to pay, but for cases like the above this is amortized O(1). Plus there is a lower constant factor because generator comprehensions are faster than for loops, so overall it works out to be a very small loss in performance for 1 file, and a huge gain when there's more. For a large repo with around 200k files in it on a case-insensitive file system, for a large directory with over 30,000 files in it, the following command was tested: ls | shuf -n $COUNT | xargs hg status This command leads to util.fspath being called on $COUNT files in the directory. COUNT before after 1 0.77s 0.78s 100 1.42s 0.80s 1000 6.3s 0.96s I also tested with COUNT=10000, but before took too long so I gave up.

test-clone.t: drop message about listing bookmarks with no hardlinks

tests: don't try to test unix sockets on vfat

tests: change obsolete timestamp to avoid "gmtime()" problem on Windows Before this patch, "test-obsolete.t" fails on Windows environment, because strings corresponded to "tm_wday" (day of the week) field are incorrect. On POSIX environment, "gmtime()" returns correct "tm_wday" value even for negative "time_t" value. On the other hand, it returns incorrect one on Windows environment. At least, "gmtime()" of the Windows runtime library bundled with Python 2.7.3 does. According to 9a7d0f7e0561 introducing original timestamp value '56 120', it shouldn't cause negative "time_t" value. test-obsolete: remove subminute timezone in test Obsmarker format "1" does not supports sub minute timezone. So we change the test to something slightly more sensible. It replaced "-d '56 12'" by "-d '56 120'".

tests: use "%HG_ARGS%" in shell alias on Windows instead of "$HG_ARGS" Before this patch, a part of "test-alias.t" fails unexpectedly on Windows environment, because "cmd.exe" can't evaluate "$HG_ARGS" expression in shell alias correctly. This patch uses "%HG_ARGS%" in shell alias on Windows instead of "$HG_ARGS" to expand it correctly.

tests: introduce "checkeditform-n-cat.sh" script to invoke "cat" in it safely Before this patch, a part of "test-transplant.t" fails unexpectedly on Windows environment, because semicolon (";") in HGEDITOR isn't recognized as the command separator by "cmd.exe". This patch newly introduces "checkeditform-n-cat.sh" script to invoke "cat" in it safely anywhere.

doc: change 'revision or range' to 'revision or revset' The phrase "revision or range" comes from a pre-revset era. Since the documentation for ranges now is under the revset docs, and as a helpful hint nudging users towards revsets, I think it's better to say "revision or revset"

largefiles: add examination of exec bit in "hg status --rev REV" case Before this patch, "hg status --rev REV" doesn't list largefiles up with "M" mark, even if exec bit of them is changed, because "lfilesrepo.status" doesn't examine exec bit in such case.

largefiles: ignore removal status of files not managed in the target context Before this patch, "hg status --rev REV" listed largefiles removed in the working directory up with "R" mark, even if they aren't managed in the REV. Normal files aren't listed up in such case. When "lfilesrepo.status" is invoked for "hg status --rev REV", it treats files on conditions below as "removed" (to avoid manifest full scan in "ctx.status" ?): - marked as "R" in lfdirstate, or - files managed in the target revision but unknown in the manifest of the working context (= not including "R" files) But the former can include files not managed in the target context. To ignore removal status of files not managed in the target context, this patch drops files unknown in the target revision from "removed" list.

changelog: use headrevsfiltered 2b5940f64750 introduced use of the new filtering headrevs C implementation. It caught TypeError to detect when to fall back to the implementation that was compatible with old extensions. That method was however not reliable. Instead, use the new headrevsfiltered function when passing a filter. It will reliably fail with AttributeError when an old extension that predates headrevsfiltered is used.

parsers: introduce headrevsfiltered in C extension All extensions that have this function do support filtering. The existing headrevs function may support filtering but we cannot reliably detect whether it does.

httppeer: close the temporary bundle file after two-way streaming it This fixes several push tests in test-bundle2-exchange.t that were failing on Windows with messages like the following: $ hg -R main push http://localhost:$HGPORT2/ -r 32af7686d403 \ --bookmark book_32af pushing to http://localhost:$HGPORT2/ searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 1 changes to 1 files remote: 1 new obsolescence markers updating bookmark book_32af abort: The process cannot access the file because it is being used by another process: 'C:\path\to\tmp\bundle.hg' [255]

status: make 'hg status --rev' faster when there are deleted files In order not to avoid listing files as both added and deleted, for example, we check for every file in the manifest if it is in the _list_ of deleted files. This can get quite slow when there are many deleted files. Change it to a set to make the containment check faster. On a somewhat contrived example of the Mozilla repo with the entire testing/ directory deleted (~14k files), this makes 'hg status --rev .^' go from 26s to 2s.

setdiscovery: limit the size of the initial sample (issue4411) The set discovery start by sending a "known" command with all local heads. When the number of local heads is massive (eg: using hidden changesets) such request becomes too large. This lead to 414 error over http, aborting the whole process. We limit the size of the sample used by the first query to fix this. The test are impacted because they do test massive number of heads. But they do not test it over real world http setup.

setdiscovery: extract sample limitation in a `_limitsample` function We need to reuse this logic for the initial query. We extract it in a function to unsure sample limiting is applied consistently in all cases.

exchange: don't report failure from identical bookmarks b901645a8784 regressed the behavior of pushing an unchanged bookmark to a remote. Before that commit, pushing a unchanged bookmark would result in "exporting bookmark @" being printed. After that commit, we now see an incorrect message "bookmark %s does not exist on the local or remote repository!" This patch fixes the regression introduced by b901645a8784 by having the bookmark error reporting code filter identical bookmarks and adds a test for the behavior.

bookmarks: explicitly track identical bookmarks bookmarks.compare() previously lumped identical bookmarks in the "invalid" bucket. This patch adds a "same" bucket. An 8-tuple for holding this state is pretty gnarly. The return value should probably be converted into a class to increase readability. But that is beyond the scope of a patch intended to be a late arrival to stable.

context.status: remove incorrect swapping of added/removed in workingctx The comment in workingctx.status() says that "calling 'super' subtly reveresed the contexts", but that is simply not true, so we should not be swapping added and removed fields.

keyword: use parent of working copy as base for status Instead of calling repo[None].status(), use the more common form that uses the parent of the working copy as the base: repo['.'].status(). Note that the former defaults to comparing to revision '.', while the latter defaults to revision None, so the contexts being compared are the same. It might seem like this would result in a reverse diff, but it turns out that workingctx.status() incorrectly reverses the result. That bug will be fixed in a later commit.

revset: avoid O(n) lookup of invalid revision in rev() ba89f7b542c9 was not the final version of that patch. It was really slow because `l not in repo.changelog` iterates revisions up to `l`. Instead, rev() should utilize spanset.__contains__(). revset #0: rev(210000) 0) wall 0.000039 comb 0.000000 user 0.000000 sys 0.000000 (best of 67978) 1) wall 0.002721 comb 0.000000 user 0.000000 sys 0.000000 (best of 1055) 2) wall 0.000059 comb 0.000000 user 0.000000 sys 0.000000 (best of 45599) (0: 3.2-rc, 1: ba89f7b542c9, 2: this patch) Note that the benchmark result described in ba89f7b542c9 is wrong because it is the one of the initial version.

run-tests: output diffs with only forward slashes This removes some gratuitous variance when testing on Windows with test-run-tests.t.

i18n: add hint to digest mismatch message

help/config: fix typo

files: fix example list syntax

parsers: use 'k' format for Py_BuildValue instead of 'n' because Python 2.4 'n' was introduced in Mercurial in 2b5940f64750 and broke Python 2.4 support in mysterious ways that only showed failure in test-glog.t. Py_BuildValue failed because of the unknown format and a TypeError was thrown ... but it never showed up on the Python side and it happily continued processing with wrong data. Quoting https://docs.python.org/2/c-api/arg.html : n (integer) [Py_ssize_t] Convert a Python integer or long integer to a C Py_ssize_t. New in version 2.5. k (integer) [unsigned long] Convert a Python integer or long integer to a C unsigned long without overflow checking. This will use unsigned long instead of Py_ssize_t. That is not a good solution, but good is not an option when we have to support Python 2.4.

rebase: improve base revset performance The old revset had pretty terrible performance on large repositories (12+ seconds). This new revset achieves the same result in only 0.7s. As we improve the underlying revset APIs we can probably get this revset down to 'only(base, dest)::', but at the moment that version still takes 2s.

amend: fix amending rename commit with diverged topologies (issue4405) This addresses the bug described in issue4405: when obsolescence markers are enabled, amending a commit with a file move can lead to the copy information being lost. However, the bug is more general and can be reproduced without obsmarkers as well, as demonstracted by Pierre-Yves and put into the updated test. Specifically, graph topology divergences between the filelogs and the changelog can cause copy information to be lost during amends.

hgweb: disable SSLv3 serving (BC) Because of recent attacks[0] on SSLv3, let's just drop support entirely. 0: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

sslutil: only support TLS (BC) In light of the POODLE[0] attack on SSLv3, let's just drop the ability to use anything older than TLSv1 entirely. This only fixes the client side. Another commit will fix the server side. There are still a few SSLv[23] constants hiding in httpclient, but I'll fix those separately upstream and import them when we're not in a code freeze. 0: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

eol: fix crash when handling removed files ci --amend would in some cases fail after 650b5b6e75ed failed to refactor the eol extension too.

bundle2: transmit exception during part generation If an exception is raised during a bundle2 part payload generation it is now recorded in the bundle. If such exception occurs, we capture it, transmit an abort exception through the bundle, cleanly close the current part payload and raise it again. This allow to generate valid bundle even in case of exception so that the consumer does not wait forever for a dead producer. This also allow to raise the exception during unbundling at the exact point it happened during bundling make debugging easier.

bundle2: add a interrupt mechanism It is now possible to emit a single part in the middle of a payload production. This part will be processed with limitation (only access to a `ui` object). The goal is to let the server raise exception and output while a part is being processed. The source motivation is to transmit exception that occurs while generating a part. This change is was the motivation to bump the bundle2 format from HG2X to HG2Y. Somehow, the format bump made it into 3.2 without it. So this change go on stable. It is low risk as bundle2 is still disabled by default.

transactions: change backupfiles format to use newlines Previously the journal.backupfiles file was delimited by \0. Now we delimit it using \n (same as the journal file). This allows us to change the number of values in each line more easily, rather than relying on the count of \0's.

transactions: add version number to journal.backupfiles The transaction format will be changing a bit over the next releases, so let's go ahead and add a version number to make backwards compatibility easier. This whole file format was broken prior to 3.2 (see previous patch), so changing it now is pretty low risk.

transactions: fix hg recover with fncache backups The transaction backupfiles logic was broken for 'hg recover'. The file format is XXX\0XXX\0YYY\0YYY\0 but the parser did a couple things wrong. 1) It went one step beyond the final \0 and tried to read past the end of the array. 2) array[i:i+1] returns a single item, instead of two items as intended. Added a test to catch it, which turns out to be the first actual 'hg recover' test.

revset: have rev() drop out-of-range or filtered rev explicitly (issue4396) The recent optimization of "and" operation relies on the assumption that the rhs set does not contain invalid revisions. So rev() has to remove invalid revisions. This is still faster than using `.filter(lambda r: r == l)`. revset #0: rev(25) 0) wall 0.026341 comb 0.020000 user 0.020000 sys 0.000000 (best of 113) 1) wall 0.000038 comb 0.000000 user 0.000000 sys 0.000000 (best of 66567) 2) wall 0.000062 comb 0.000000 user 0.000000 sys 0.000000 (best of 43699) (0: bbf4f3dfd700^, 1: 3.2-rc, 2: this patch)

revset: avoid recalculating filesets This fixes a regression in 8dabcc889e33 that moved matcher building into a callback, thus causing it be rebuilt for each revision matched against.

tests: open file in binary mode to use POSIX end-of-line style anywhere Before this patch, "test-context.py" fails on Windows environment, because "diff" output contains unexpected "\r" character. Opening the target file in text mode causes this automatic end-of-line conversion.

tests: add "(glob)" for l10n messages in test-clone.t for Windows This patch follows the style of other tests avoiding same kind of issue.

tests: add "(glob)" for l10n messages in test-largefiles-update.t for Windows This patch follows the style of other tests avoiding same kind of issue.

tests: add "(glob)" to paths in test-revset-outgoing.t for Windows

tests: add "(glob)" to paths in test-hgrc.t for Windows

tests: add "(glob)" to paths in test-url-rev.t for Windows

tests: adjust expected prompt choice outputs in test-record.t for Windows Changeset 9ab18a912c44 introduced "showing prompt choice if input is not a tty but is forced to be interactive" and adjusted expected output in test-record.t. But some tests for no-execbit platform (= Windows) were not adjusted by 9ab18a912c44. This causes unexpected failure of test-record on Windows environment. This patch adds below to prompt lines for such tests: - prompt choice at the end of prompt line, and - empty line after prompt line

ui: separate option to show prompt echo, enabled only in tests (issue4417) The problem in commandserver was addressed by 524b786bd54f, but it is tricky to reuse ui.nontty option to disable echo back. Instead, this patch introduces new option to enable echoing of prompt response. Prompt echoing is changed to be off by default, which should avoid possible breakage of output parsing in user scripts. For now, this option is undocumented because it exists for internal use.

Added signature for changeset 7f8d16af8cae

Added tag 3.2-rc for changeset 7f8d16af8cae

merge with i18n

i18n-ru: synchronized with 6b4dc7968bf0

merge default into stable for 3.2 freeze

hook: schedule run "b2x-transactionclose" for after lock release Hooks that run after the transaction need to be able to touch the repository. So we need to run them after the lock release. This is similar to what the "changegroup" hook is doing in the `addchangegroup` function.

repoview: issue a special message when filtering hidden changesets Hidden changesets are by far the most common error case and is the only one[1] that can reach the user. We move to a friendlier message with a hint about how to access the data anyway. We should probably point to a help topic instead but we do not have such a topic yet. Example of the new output abort: hidden revision '4'! (use --hidden to access hidden revisions) [1] Actually, filtering from "served" can also reach the user during certain exchange operations.

repoview: include the filter name in filtered revision error messages This will help user to debug. A more precise message will be issued for the most common case ("visible" filter) in the next changesets. example output: - abort: filtered revision '4'! + abort: filtered revision '4' (not in 'visible' subset)!

largefiles: inline redundant toname function in status Simpler and an optimization.

largefiles: inline redundant inctx function in status

ssl: only use the dummy cert hack if using an Apple Python (issue4410) The hack for using certificate store in addition to the provided CAs resides in Apple's OpenSSL. Apple's own Pythons will use it, but other custom built Pythons might use a custom built OpenSSL without that hack and will fail when exposed to the dummy cacert introduced in d7f7f1860f00. There do not seem to be a simple way to check from Python if we are using a patched OpenSSL or if it is an Apple OpenSSL. Instead, check if the Python executable resides in /usr/bin/python* or in /System/Library/Frameworks/Python.framework/ and assume that all Pythons found there will be native Pythons using the patched OpenSSL. Custom built Pythons will not get the benefit of using the CAs from the certificate store.

largefiles: move initialization of standins variable to clarify its "scope"

largefiles: the update override only needs lfdirstate and status for --check

largefiles: remove confusing rev parameter for lfdirstatestatus Dirstate only works on the repo wctx.

largefiles: replace repo._isaddremove hack with a simple function parameter