extensions: set attributes to wrappers so we can trace them back This patch adds two attributes about the original function and the unbound wrapper. It allows us to get a chain of wrappers. See the next patch.

ui: drop values returned by inspect.*frame*() to avoid cycles "f = inspect.currentframe()" instantly creates a cycle because "f.f_locals['f']" is "f" itself. This patch explicitly sets those frame objects to None to avoid cycles.

dispatch: split global error handling out so it can be reused We may want a similar error handling at worker.py. This patch extracts the error handling logic to "callcatch" so it can be reused.

py3: conditionalize _winreg import _winreg module is renamed to winreg in python 3. Added the conditionalize statements in the respective file because adding this in pycompat will result in pycompat throwing error as this is a windows registry module and we have buildbots and most of the contributors on linux.

py3: conditionalize the raise statement raise E,V,T is not acceptable in Python 3, thats is conditionalized. Moreover this will result in syntax error so we have to use exec() to execute this. Related PEP- https://www.python.org/dev/peps/pep-3109/#id14 My implementation is motivated from the six implementation except they are defining a new function exec_() to prevent adding an extra frame AFAIK :) https://bitbucket.org/gutworth/six/src/ca4580a5a648/six.py#six.py-680

match: added matchessubrepo method to matcher Previously there were three local implementations of this function in cmdutil.files, cmdutil.remove and scmutil.addremove.

changegroup: move branch cache debug message to proper location Before, we logged about performing a branch cache update when we weren't actually doing it. Fix that.

journal: take wlock for writting the 'shared' file As we did for the shared extension itself, we add some locking around the write of the 'shared' file.

debugbuilddag: take wlock to cover '.hg/localtags' This debug command can write local tags. local tags are in the .hg directory and should be covered by the 'wlock'. This is now covered.

fakemergerecord: take wlock to write the merge state The merge state is supposed to be covered by the wlock. We fix the test extensions to comply to that.

shared: take wlock for writting the 'shared' file I do not see a reason why this should not be covered by the wlock.

mq: take wlock when 'qqueue' is doing write operations Apparently when this command was added, the locking was forgotten. No code changes beside the indentation from the locking context.

py3: make check-py3-compat.py use correct module name at loading pure modules Before this patch, check-py3-compat.py implies unintentional ".pure" sub-package name at loading pure modules, because module name is composed by just replacing "/" in the path to actual ".py" file by ".". This makes pure modules belong to "mercurial.pure" package, and prevents them from importing a module belonging to "mercurial" package relatively by "from . import foo" or so. This is reason why pure modules fail to import another module relatively only at examination by check-py3-compat.py.

py3: update output of check-py3-compat.py with python3 Recent work on mercurial/pure/mpatch.py made it import mercurial.policy, and changed output of check-py3-compat.py with python3 in test-check-py3-compat.t. But test-check-py3-compat.t isn't yet updated.

mpatch: raise MemoryError instead of mpatchError if lalloc() failed MemoryError is handled differently in dispatch._runcatch(). Since mpatch_errors[] isn't that useful now, I've changed it to a simple switch statement.

hgweb: config option to control zlib compression level Before this patch, the HTTP transport protocol would always zlib compress certain responses (notably "getbundle" wire protocol commands) at zlib compression level 6. zlib can be a massive CPU resource sink for servers. Some server operators may wish to reduce server-side CPU requirements while requiring more bandwidth. This is common on corporate intranets, for example. Others may wish to use more CPU but reduce bandwidth. This patch introduces a config option to allow server operators to control the zlib compression level. On the "mozilla-unified" generaldelta repository, setting this value to "0" (disable compression) results in server-side CPU utilization for a `hg clone` going from ~180s to ~124s CPU time on my i7-6700K. A level of "1" (which increases the transfer size from ~1,074 MB at level 6 to ~1,222 MB) utilizes ~132s CPU time.

help: don't try to render a section on sub-topics This patch subtly changes the behavior of the parsing of "X.Y" values to not set the "section" variable when rendering a known sub-topic. Previously, "section" would be the same as the sub-topic name. This required the sub-topic RST to have a section named the same as the sub-topic name. When I made this change, the descriptions from help.internalstable started being rendered in command line output. This didn't look correct to me, as it didn't match the formatting of main help pages. I corrected this by moving the top section to help.internalstable and changing the section levels of all the "internals" topics. The end result is that "internals" topics now match the rendering of main topics on both the CLI and HTML. And, "internals" topics no longer require a main section matching the name of the topic.

branchmap: remove extra indent This clean up the rest of the previous changeset.

branchmap: simplify error handlind when writing rev branch cache Now that we have a general try except, we can move the error handling from the individual writes in it. Code will be reindented in the next changeset to help this on readability.

branchmap: acquires lock before writting the rev branch cache We now attempt to acquire a lock and write the branch cache within that lock. This would prevent cache corruption when multiple processes try to write the cache at the same time.

branchmap: preparatory indent of indent the branch rev writing code The rev branch cache is written without a lock, we are going to fix this but we indent the code beforehand to make the next changeset clearer.

mpatch: silence warning about maybe-uninitialized variable It's false positive, but it wouldn't be possible for gcc to know PyBytes_FromStringAndSize() sets PyErr_Occurred(). mercurial/mpatch_module.c:105:47: warning: 'r' may be used uninitialized in this function [-Wmaybe-uninitialized] PyErr_SetString(mpatch_Error, mpatch_errors[-r]);

mpatch: change lalloc() to local function It was mistakenly made public at b9b9f9a92481.

mpatch: remove superfluous whitespaces

cmdutil: remove duplicated badmatch call in cat() Subrepo logic is handled in ctx.walk().

statichttprepo: do not try to write caches The static http repository are read only, there is no hope in any cache writing attempt.

demandimport: omit default value of "level" at construction of _demandmod This makes construction of _demandmod require explicit level value, and helps to avoid issues like issue5208 in the future.

demandimport: import sub-module relatively as expected (issue5208) Before this patch, importing sub-module might (1) fail or (2) success but import incorrect module, because demandimport tries to import sub-module with level=-1 (on Python 2.x) or level=0 (on Python 3.x), which is default value of "level" argument at construction of "_demandmod" proxy object. (1) on Python 3.x, importing sub-module always fails to import existing sub-module (2) both on Python 2.x and 3.x, importing sub-module might import same name module on root level unintentionally On Python 2.x, existing sub-module is prior to this unexpected module. Therefore, this problem hasn't appeared. To import sub-module relatively as expected, this patch specifies "1" as import level explicitly at construction of "_demandmod" proxy object for sub-module.

wireproto: remove gboptslist (API) This variable has been unused since ce25f465e572, which was over 2 years ago. gboptsmap should be used instead. Marking as API because this could break extensions.

wireproto: unescape argument names in batch command (BC) Clients escape both argument names and values when using the batch command. Yet the server was only unescaping argument values. Fortunately we don't have any argument names that need escaped. But that isn't an excuse to lack symmetry in the code. Since the server wasn't properly unescaping argument names, this means we can never introduce an argument to a batchable command that needs escaped because an old server wouldn't properly decode its name. So we've introduced an assertion to detect the accidental introduction of this in the future. Of course, we could introduce a server capability that says the server knows how to decode argument names and allow special argument names to go through. But until there is a need for it (which I doubt there will be), we shouldn't bother with adding an unused capability.

wireproto: consolidate code for obtaining "cmds" argument value Both wireproto.py and sshpeer.py had code for producing the value to the "cmds" argument used by the "batch" command. This patch extracts that code to a standalone function and uses it.

revlog: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

md5sum: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

util: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

url: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

sshserver: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

sshpeer: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

patch: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

commands: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

changegroup: use `iter(callable, sentinel)` instead of while True This is functionally equivalent, but is a little more concise.

extdiff: isolate path variable of saved command to independent paragraph Otherwise, the whole paragraph wouldn't be translated.

extdiff: export __doc__ of saved command for translation

extdiff: refactor closure of saved diff command as a top-level class This allows us to collect __doc__ for translation.

i18n: use inspect.getsourcelines() to obtain lineno from func or class Before, func must be a function object. I want to make it parse docstring of classes, too.

chg: just take it as EOF if recv() returns 0 hgc->sockfd is a blocking stream socket. recv() should never return 0 other than EOF. See 4fc4b8cc9957 for the original problem.

vfs: use propertycache for open The current open method is currently behaving like a property cache. We use our utility decorator to make this explicit.

check-commit: allow underbars in cffi_-prefix function names It seems reasonable to give cffi functions slightly more verbose names in some circumstances, given the way they interface with C.

bundlerepo: add support for treemanifests in cg3 bundles This is a little messier than I'd like, and I'll probably come back and do some more refactoring later, but as it is this unblocks narrowhg. An alternative approach (which I may do as part of the mentioned refactoring) would be to construct *all* dirlog instances up front, so that we don't have to keep track of the linkmapper method. This would avoid a reference cycle between the bundlemanifest and the bundlerepository, but I was hesitant to do all the work up front like that. With this change, it's possible to do 'hg incoming' and 'hg pull' from bundles in .hg/strip-backup in a treemanifest repository. Sadly, this doesn't make it possible to 'hg clone' one of those (if you do 'hg strip 0'), because the cg3 in the bundle gets written without a treemanifest flag. Since that's going to be an involved refactor in a different part of the code (which I *suspect* won't touch any of the code I've just written here), let's leave it as an idea for Later.

auditvfs: forward options property from nested vfs This was breaking my ability to use treemanifests in bundlerepos, and was deeply mysterious. We should probably just make the options property a formal part of the vfs API, and make it a required construction parameter. Sadly, I don't have time to dive into that refactor right now.

bundlerepo: use supportedincomingversions instead of allsupportedversions Since bundlerepo is really a pull-like operation, this is the correct method to use here.

bundlerepo: introduce method to find file starts and use it This moves us to the modern iter() technique instead of the `while True` pattern since it's easy. Factored out as a function because I'm about to need this in a second place.

bundlerevlog: use for loop over iterator instead of while True The iter() builtin has a neat pattern where you give it a callable of no arguments and a sentinel value, and you can then loop over the function calls like a normal iterator. This cleans up the code a little.

bundlerepo: use for loop over iterator instead of while True The iter() builtin has a neat pattern where you give it a callable of no arguments and a sentinel value, and you can then loop over the function calls like a normal iterator. This cleans up the code a little.

localrepo: jettison now-unused dirlog() method from localrepo

repair: build dirlogs using manifest, rather than repo shortcut method As before, this was rarely used, so let's get rid of the convenience method.

cmdutil: open dirlogs via manifest property, not via repo This was a convenience method that was rarely used, so let's get rid of it.

wirepeer: rename confusing `source` parameter It's named "url" everyplace else this method is defined, so let's be consistent.

develwarn: use the lock helper in local repo We have a helper function to know if a lock is taken. When checking lock usage we now use it instead of manually accessing the locks.

debugextension: change "testedwith" to a list (BC) It wasn't a list since the formatter couldn't process a list. We have no such problem now and the -T option is experimental, so we can change it.

debugextensions: unindent nested if

debugextensions: give short name to util.version()

debugextensions: simply keep testedwith variable as a list There should be no need to distinguish [] and None.

test-treemanifest: ensure manifest command isn't broken I realized we weren't testing this while hunting a broken manifest command bug that ended up being narrowhg's fault.

py3: use unicode literals in pure/osutil.py The first element of _fields_ tuples must be a str in Python 3. Also fix some function calls that were also expecting str.

py3: use unicode literals in crecord.py

py3: use unicode literals in changelog.py collections.namedtuple type and field names must be str in Python 3. Our custom module importer was rewriting them to bytes literals, making this fail. In addition, __slots__ values must also be unicode.

mpatch: write a cffi version of mpatch.patches

mpatch: remove dependency on Python.h in mpatch.c Now all the CPython-related stuff are referenced only from mpatch_module.c with mpatch.c being freely usable from a future cffi module

mpatch: split mpatch into two files

mpatch: provide things that will be exported later with a prefixed name For cffi a bunch of mpatch functions need to be visible through a .h file. This change renames them so it won't create potential c namespace conflicts.

mpatch: change Py_ssize_t to ssize_t in places that will be later copied

discovery: move code to create outgoing from roots and heads changegroup.changegroupsubset() contained somewhat low-level code for constructing an "outgoing" instance from a list of roots and heads nodes. It feels like discovery.py is a more appropriate location for this code. This code can definitely be optimized, as outgoing.missing will recompute the set of changesets we've already discovered from cl.between(). But code shouldn't be refactored during a move, so I've simply inserted a TODO calling attention to that.

bundle2: remove 'experimental.bundle2-exp' boolean config (BC) All users are migrated to 'devel.legacy.exchange', we can clean up the experimental namespace. Marking as (BC) because I know some large installation have bundle2 off and I want to make sure they notice the change.

tests: remove all remaining usage of experimental.bundle2-exp The only remaining usage of the experimental config were enforcing bundle2 on. These are very old remains of when bundle2 was off by default. This was also useful to highlight the fact that this was a bundle2 run and that a bundle1 one was nearby. However, we want a future developer working on bundle3 to notice possible output/behavior change on these tests and take them in account. So we do not enforce bundle2 for these runs. We leave a comment around to make sure dev still notice the bundle1 version.

tests: use 'legacy.exchange' option in various mixed tests The new option will stay around. The experimental option was only meant to be temporary. We update various tests that validate both bundle1 and bundle2 version side by side. This changeset only takes care of enforcing bundle1. The other use of 'experimental.bundle2-exp=True' will be taken care of in the next changeset.

tests: use 'legacy.exchange' option in various bundle1 tests The new option will stay around. The experimental option was only meant to be temporary.

tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.

bundlerepo: also read the 'devel.legacy.exchange' config Bundlerepo does its own bundle2 related logic.

bundle2: add a devel option controling bundle version used for exchange We need an official way to force bundle1 to be used in test. We introduce a new option 'devel.legacy.exchange' to control this. When specified, this option will control the list of bundle version Mercurial consider when exchanging with a peer. Current valid value are 'bundle1' and 'bundle2'. Using this option in all tests will allow us to remove the 'experimental.bundle2-exp' option. We will simplify the code once the experimental option is dropped.

bundle2: rename the _canusebundle2 method to _forcebundle1 We rename and invert the logic of the _canusebundle2 utility. The idea here is that we need to have a way to enforce the use of bundle1 in the tests. The Mercurial philosophy is to try to use the best method available. Currently that best method is bundle2, but this might change in the future. Therefore expressing "do not use bundle2" is a loosy way to say "use bundle 1" and will likely create issue in the future. As the config option will be explicitly about bundle1, we rename the function beforehand to align with this. This will make the life of a future developer working on bundle3 easier.

tests: remove bundle2 activation from test-bookmark-pushpull.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: update bugzilla link in test-issue1175.t

journal: use fm.formatdate() to pass date tuple in appropriate type (BC)

formatter: add function to convert date tuple to appropriate format

journal: use fm.formatlist() to pass hashes in appropriate type (BC)

formatter: add function to convert list to appropriate format (issue5217) Before, it wasn't possible for formatter to handle array structure other than date tuple. We've discussed that at the last sprint, which ended we would probably want to allow only templatable data structure, i.e. a list of dicts: data(tags=[{'tag': a}, {'tag': b}, ...]) Unfortunately, it turned out not working well with template functions: "{ifcontains(a, tags, ...)}" ^^^^^^^^^^^^^^^^^^ "a in tags", where tags should be a plain list/set of tags So the formatter must at least know if the type [{}] was constructed from a plain list or was actually a list of dicts. This patch introduces new explicit interface to convert an array structure to an appropriate data type for the current formatter, which can be used as follows: fm.write('tags', _('tags: %s\n'), fm.formatlist(tags, name='tag')) No separate fm.data() call should be necessary.

journal: use fm.hexfunc() to get full hash in JSON/template output (BC) We generally do that.

py3: update test-check-py3-compat.t output The lower part of test-check-py3-compat.t runs only on py3 and hence its remain unchanged. Hence this patch updates the output so that change in output in the next patches will be only related to the change in the patch.

deprecation: enforce thew 'tr' argument of 'dirstate.write' (API) Compatibility was meant to be drop after 3.9 is released.

tests: remove initial bundle2 enabling in various bundle2 tests Bundle 2 is enable by default since 3.6 so we don't need this anymore. The remaining use of 'experimental.bundle2-exp' are there to test legacy behavior with bundle 1. Future patches will introduce a way outside of experimental to test that.

tests: remove bundle2 activation from test-largefiles.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-tags.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

templatekw: change joinfmt to a mandatory argument of _hybrid object We've fixed several bugs caused by the misuse of the default joinfmt. Make it more explicit to prevent future bugs. dict.values()[0] is replaced by dict[element] as showlist() knows what the key is.

url: drop compatibility wrapper of socket.create_connection() It should be available on Python 2.6+.

doc: remove double imports of inspect from hgmanpage.py

merge: concatenate default conflict marker at parsing phase of .py "+" operations are unnecessary.

tests: remove bundle2 activation from test-push-warn.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-phases-exchanges.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-obsolete.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-http-proxy.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-hook.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

tests: remove bundle2 activation from test-acl.t This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.

merge with stable

Added signature for changeset 149433e68974

Added tag 3.9.2 for changeset 149433e68974

py3: update test expectation on stable

grep: rewrite help to better document current (confusing) behavior

crecord: properly handle files with No newline at eof (issue5268) Yes, this bug was a single character with the wrong case...

annotate: pre-calculate the "needed" dictionary (issue5360) The "needed" dict is used as a reference counter to free items in the giant "hist" dict. However, currently it is not very accurate and can lead to dropping "hist" items unnecessarily, for example, with the following DAG, -3- / \ 0--1--2--4-- The current algorithm will visit and calculate rev 1 twice, undesired. And it tries to be smart by clearing rev 1's parents: "pcache[1] = []" at the time hist[1] being accessed (note: hist[1] needs to be used twice, by rev 2 and rev 3). It can result in incorrect results if p1 of rev 4 deletes chunks belonging to rev 0. However, simply removing "needed" is not okay, because it will consume 10x memory: # without any change % HGRCPATH= lrun ./hg annotate mercurial/commands.py -r d130a38 3>&2 [1] MEMORY 49074176 CPUTIME 9.213 REALTIME 9.270 # with "needed" removed MEMORY 637673472 CPUTIME 8.164 REALTIME 8.249 This patch moves "needed" (and "pcache") calculation to a separate DFS to address the issue. It improves perf and fixes issue5360 by correctly reusing hist, while maintaining low memory usage. Some additional attempt has been made to further reduce memory usage, like changing "pcache[f] = []" to "del pcache[f]". Therefore the result can be both faster and lower memory usage: # with this patch applied MEMORY 47575040 CPUTIME 7.870 REALTIME 7.926 [1]: lrun is a lightweight sandbox built on Linux cgroup and namespace. It's used to measure CPU and memory usage here. Source code is available at github.com/quark-zju/lrun.

Added signature for changeset ccd436f7db6d

Added tag 3.9.1 for changeset ccd436f7db6d

bundle2: localize handleoutput remote prompts Code archaeology suggests that there was no good reason for this not to be localized. 'remote: ' is already localized elsewhere.

i18n-pt_BR: synchronized with a12d13eac513

bundle2: fail faster when interrupted Before this patch, bundle2 application attempted to consume remaining bundle2 part data when the process is interrupted (SIGINT) or when sys.exit is called (translated into a SystemExit exception). This meant that if one of these occurred when applying a say 1 GB changegroup bundle2 part being downloaded over a network, it may take Mercurial *several minutes* to terminate after a SIGINT because the process is waiting on the network to stream megabytes of data. This is not a great user experience and a regression from bundle1. Furthermore, many process supervisors tend to only give processes a finite amount of time to exit after delivering SIGINT: if processes take too long to self-terminate, a SIGKILL is issued and Mercurial has no opportunity to clean up. This would mean orphaned locks and transactions. Not good. This patch changes the bundle2 application behavior to fail faster when an interrupt or system exit is requested. It does so by not catching BaseException (which includes KeyboardInterrupt and SystemExit) and by explicitly checking for these conditions in yet another handler which would also seek to the end of the current bundle2 part on failure. The end result of this patch is that SIGINT is now reacted to significantly faster: the active transaction is rolled back immediately without waiting for incoming bundle2 data to be consumed. This restores the pre-bundle2 behavior and makes Mercurial treat signals with the urgency they deserve.

revset: fix keyword arguments to go through optimization process Before, a keyvalue node was processed by the last catch-all condition of _optimize(). Therefore, topo.firstbranch=expr would bypass tree rewriting and would crash if an expr wasn't trivial.

help: add example of '[templates]' usage V2: - move from shortest() with minlength 8 to minlength 4 - mention [templates] in config.txt - better describe the difference between [templatealias] and [templates] V3: - choose a better example template

exchange: correctly specify url to unbundle (issue5145) This parameter is slightly confusingly named in wireproto, so it got mis-specified from the start as 'push' instead of the URL to which we are pushing. Sigh. I've got a patch for that which I'll mail separately since it's not really appropriate for stable. Fixes a regression in bundle2 from bundle1.

convert: move svn config initializer out of the module level The svn_config_get_config config call was being called at the module level, but had the potential to throw permission denied errors if ~/.subversion/servers was not readable. This could happen in certain test environments where the user permissions were very particular. This prevented the remotenames extension from loading, since it imports convert's hg module, which imports convert's subversion module, which calls this. The config is only ever used from this one constructor, so let's just move it in to there.

osx: update bugzilla link in readme

win32: update link to mailing list in readme

win32: update wiki link in mercurial.ini

debian: update source URL in copyright file

help: update link to wiki/CommandServer

Added signature for changeset 299546f84e68

Added tag 3.9 for changeset 299546f84e68

merge with i18n

doc: make previous line of certificate example end with "::" Before this patch, certificate example is formatted just as normal text.

doc: fix incorrect use of rst hg role in help text