py3: handle iter/iterkeys+iteritems python3 divergence in import-checker

py3: use print_function in import-checker

py3: use absolute_import in import-checker

run-tests: make _processoutput picky about optional globs 1ad0ddf8cccc enabled lines that were not matched to be found later in cases of jitter. Unfortunately, in this model an optional line would always jitter to the end when it is not present. That is not ideal. It would be possible to do better, by queuing all writes until the end in case an optional line jitters, but for now, it is simpler to assume optional lines have a fixed place in the stream.

py24: remove check-code py24 notation We require python2.6+

py3: convert hghave output to text Before this, Python3 generated: b'skipped: unknown feature: not-py3k\n'

py3: convert prereq bytes to string in run-tests Without this, run-tests would generate: WARNING: Did not find prerequisite tool: b'python3.5'

record: deprecate the extension The feature has been moved into core behind the -i flag. We can safely deprecated the extension and point people at the --interactive flag in core.

templater: use templatefunc to mark a function as template function Using decorator can localize changes for adding (or removing) a template function in source code. This patch also removes leading ":FUNC(ARG...):" part in help document of each function, because using templatefunc makes it useless. This patch uses not 'func' but 'templatefunc' as a decorator name, because the former is too generic one, even though the latter is a little redundant in 'templater.py'.

registrar: add templatefunc to mark a function as template function (API) This patch also adds loadfunction() to templater, because this combination helps to figure out how they cooperate with each other. Listing up loadfunction() in dispatch.extraloaders causes implicit loading template function at loading (3rd party) extension. This patch explicitly tests whether templatefunc decorator works as expected, because there is no bundled extension, which defines template function. This change requires that "templatefunc" attribute of (3rd party) extension is registrar.templatefunc or so.

keyword: use templatefilter to mark a function as template filter This patch also adds test for filter 'svnisodate' and 'svnutcdate' for safety, because there is no test using them, yet.

templatefilters: use templatefilter to mark a function as template filter Using decorator can localize changes for adding (or removing) a template filter function in source code. This patch also removes leading ":FILTER:" part in help document of each filters, because using templatefilter makes it useless. This patch uses not 'filter' but 'templatefilter' as a decorator name, because the former name hides Python built-in one, even though the latter is a little redundant in 'templatefilters.py'.

registrar: add templatefilter to mark a function as template filter (API) This patch also adds loadfilter() to templatefilters, because this combination helps to figure out how they cooperate with each other. Listing up loadfilter() in dispatch.extraloaders causes implicit loading template filter functions at loading (3rd party) extension. This change requires that "templatefilter" attribute of (3rd party) extension is registrar.templatefilter or so.

revset: inline _getaliasarg() function This function is now much simpler than before. Inlining small functions helps to extract a reusable alias processor.

revset: drop redundant check for unknown alias arguments Since _parsealiasdefn() rejects unknown alias arguments, _checkaliasarg() is unnecessary. New test is added to make sure unknown '$n' symbols are rejected.

revset: move tagging of alias arguments from tokenization to parsing phase In short, this patch moves the hack from tokenizedefn() to _relabelaliasargs(), which is called after parsing. This change aims to eliminate tight dependency on the revset tokenizer. Before this patch, we had to rewrite an alias argument to a pseudo function: "$1" -> "_aliasarg('$1')" ('symbol', '$1') -> ('function', ('symbol', '_aliasarg'), ('string', '$1')) This was because the tokenizer must generate tokens that are syntactically valid. By moving the process to the parsing phase, we can assign a unique tag to an alias argument. ('symbol', '$1') -> ('_aliasarg', '$1') Since new _aliasarg node never be generated from a user input, we no longer have to verify a user input at findaliases(). The test for _aliasarg("$1") is removed as it is syntactically valid and should pass the parsing phase.

revset: add test that should fail if '_aliasarg' tag is removed I'm going to refactor the alias processing functions. We need '_aliasarg' tag to limit the scope of the alias expansion, but it wasn't covered by the test. This patch adds the test that should fail if '_aliasarg' were 'symbol'. This is the first half of the second part of the "template alias" series. The whole series will consist of the following parts: 1. make parsed template tree to be compatible with parser functions (1d461ee26e1b and 73d01cba5810) 2. refactor alias processing to be less dependent on revset module (1/2 in this series) 3. extract reusable component to parser module 4. clean up it 5. extend it to support template syntax 6. add debugging/testing functions of template aliases 7. add alias expansion routine to templater

templater: do not strip non-quote characters from template config Before this patch, the first and last characters were stripped from ui.logtemplate and template.* if they were the same. It could lead to a strange result as quotes are optional. See the test for example.

rebase: fix crash when rebase aborts while rebasing obsolete revisions Before this patch, rebase --continue would crash when trying to resume a rebase of obsolete revisions whose successors were in the destination. This patch adds logic to recompute the mapping when rebase is resumed. This patch also adds a test that showcased the crash before the code change.

rebase: refactor of error handling code path for rebaseskipobsolete This patch extracts the error handling code path to go in a separate function. In the next patch we will able to reuse this logic and avoid duplicated code.

destutil: show message and hint at updating to the closed head as warning

destutil: make messages at updating to the closed head usual form This patch makes messages at updating to the closed head usual form for Mercurial as below: one line description of the problem with no period (a suggestion about how to move forward or get more info)

py3: make test-ui-color use print_function

py3: make test-ui-config use print_function

py3: make test-ui-config use absolute_import

py3: make test-ui-verbosity use absolute_import

py3: make test-ui-verbosity use print_function

py3: make test-url use print_function

py3: make test-walkrepo use print_function

py3: make test-wireproto use print_function

py3: handle ugettext + unicode in i18n

py3: glob line numbers in test-check-py3-compat

bundle: remove obsolete (and duplicate) comment Change 1e28ec9744bf (changegroup: move chunk extraction into a getchunks method of unbundle10, 2014-04-10) extracted some code to a getchunks() method and copied a comment about the changegroup format to the new method. The copy that remains in the old place, doesn't make much sense there, so let's remove it.

convert: delete unused imports in git.py As reported by pyflakes

merge with stable

bundle: avoid crash when no good changegroup version found When using treemanifests, only changegroup3 bundles can be created. However, there is currently no way of requesting a changegroup3 bundle, so we run into an assertion in changegroup.getbundler() when trying to get a changroup2 bundler. Let's avoid the traceback and print a short error message instead.

exchange: make _pushb2ctx() look more like _getbundlechangegrouppart() The functions already have a lot in common, but were structured a little differently.

exchange: get rid of "getcgkwargs" variable This also makes the "version" argument explicit (never relies on getlocalchangegroupraw()'s default), which I think is a good thing.

bundle: move writebundle() from changegroup.py to bundle2.py (API) writebundle() writes a bundle2 bundle or a plain changegroup1. Imagine away the "2" in "bundle2.py" for a moment and this change should makes sense. The bundle wraps the changegroup, so it makes sense that it knows about it. Another sign that this is correct is that the delayed import of bundle2 in changegroup goes away. I'll leave it for another time to remove the "2" in "bundle2.py" (alternatively, extract a new bundle.py from it).

Added signature for changeset ae279d4a19e9

Added tag 3.7.3 for changeset ae279d4a19e9

convert: test for shell injection in git calls (SEC) CVE-2016-3069 (5/5) Before recent refactoring we were not escaping calls to git at all which made such injections possible. Let's have a test for that to avoid this problem in the future. Reported by Blake Burkhart.

convert: rewrite gitpipe to use common.commandline (SEC) CVE-2016-3069 (4/5)

convert: dead code removal - old git calling functions (SEC) CVE-2016-3069 (3/5)

convert: rewrite calls to Git to use the new shelling mechanism (SEC) CVE-2016-3069 (2/5) One test output changed because we were ignoring git return code in numcommits before.

convert: add new, non-clowny interface for shelling out to git (SEC) CVE-2016-3069 (1/5) To avoid shell injection and for the sake of simplicity let's use the common.commandline for calling git.

subrepo: set GIT_ALLOW_PROTOCOL to limit git clone protocols (SEC) CVE-2016-3068 (1/1) Git's git-remote-ext remote helper provides an ext:: URL scheme that allows running arbitrary shell commands. This feature allows implementing simple git smart transports with a single shell shell command. However, git submodules could clone arbitrary URLs specified in the .gitmodules file. This was reported as CVE-2015-7545 and fixed in git v2.6.1. However, if a user directly clones a malicious ext URL, the git client will still run arbitrary shell commands. Mercurial is similarly effected. Mercurial allows specifying git repositories as subrepositories. Git ext:: URLs can be specified as Mercurial subrepositories allowing arbitrary shell commands to be run on `hg clone ...`. The Mercurial community would like to thank Blake Burkhart for reporting this issue. The description of the issue is copied from Blake's report. This commit changes submodules to pass the GIT_ALLOW_PROTOCOL env variable to git commands with the same list of allowed protocols that git submodule is using. When the GIT_ALLOW_PROTOCOL env variable is already set, we just pass it to git without modifications.

parsers: detect short records (SEC) CVE-2016-3630 (2/2) This addresses part of a vulnerability in binary delta application.

parsers: fix list sizing rounding error (SEC) CVE-2016-3630 (1/2) This addresses part of a vulnerability in application of binary deltas.

merge with stable

debugsetparents: remove redundant invocations of begin/endparentchange Method localrepo.setparents invokes begin/endparentchange internally, so there is no need to invoke it explicitly in debugsetparents.

sslutil: add docstring to wrapsocket() Security should not be opaque.

sslutil: remove indentation in wrapsocket declaration It is no longer needed because we have a single code path.

sslutil: always use SSLContext Now that we have a fake SSLContext instance, we can unify the code paths for wrapping sockets to always use the SSLContext APIs. Because this is security code, I've retained the try..except to make the diff easier to read. It will be removed in the next patch. I took the liberty of updating the inline docs about supported protocols and how the constants work because this stuff is important and needs to be explicitly documented.

sslutil: move _canloaddefaultcerts logic We now have a newer block accessing SSLContext. Let's move this code to make subsequent refactorings of the former block easier.

sslutil: implement SSLContext class Python <2.7.9 doesn't have a ssl.SSLContext class. In this patch, we implement the interface to the class so we can have a unified code path for all supported versions of Python. This is similar to the approach that urllib3 takes.

sslutil: store OP_NO_SSL* constants in module scope An upcoming patch will introduce a global SSLContext type so we have a single function used to wrap sockets. Prepare for that by introducing module level constants for disabling SSLv2 and SSLv3.

sslutil: better document state of security/ssl module Pythons older than 2.7.9 are lacking the modern ssl module and have horrible security. Let's document this explicitly.

tests: make tinyproxy.py use print_function

run-tests: use canonpath for with-python3