merge with stable

hghave: add a check for docker support This currently refuses to operate if on a non-Linux host. I suspect that Docker running on FreeBSD 11 or on an Illumos derivative would work fine, but I don't have ready access to such a system. On OS X using boot2docker (I used a hacky xhyve-based one for testing), it won't work because $TESTTEMP doesn't end up inside the set of directories that get forwarded to the boot2docker VM, so you can't actually drop debs in the $TESTTEMP at all. It would be possible (probably even trivial) to hack around this by using a randomly-named temporary directory inside the working directory, but that seems unlikely to be useful enough to justify the ugliness.

hghave: add a check for debian packaging tools

run-tests: add support for marking tests as very slow I want to add tests for our packaging rules, but those necessarily run a whole build, or possibly two if both native packaging and docker are available. This lets us flag such tests with a `#require slow` so that they don't unnecessarily slow down normal test runs.

builddeb: rework how output dir and platform are specified This makes it possible to write tests for both builddeb and dockerdeb that actually build .debs and then sanity check the contents.

parsers: fix two cases of unsigned long instead of Py_ssize_t We had to do this before because Python 2.4 didn't understand the n format specifier in Py_BuildValue and friends. We no longer have that problem.

templatefilters: remove redundant 'date' and 'strip' filters These filters are defined as 'date()' and 'strip()' functions. Help messages are moved to the corresponding functions.

templater: introduce unified filter syntax for unary functions "filter(expr)" is allowed already. This patch adds the opposite, "expr|func".

templater: inline getfilter() to buildfilter() This prepares for the unified filter syntax that will be introduced by the next patch.

clone: fix updaterev to update to latest branch changeset (issue4528) Before this patch if clone --updaterev points to branch which head on src repo wasnt in dest repo, clone updated dest repo to default branch. After applying this patch, if changeset from src repo pointing at given branch is not in dest repo, it searches for changeset pointing for given branch locally in dest repo. Lookup in destination repo: 559: uprev = destrepo.lookup(update) is wrapped by try/except block to preserve current behaviour when given revset to -u is not found - it will not fail,but silently update dest repo to head of default branch.

revsets: makes follow() supports file patterns (issue4757) (BC) Before this patch, follow only supports full, exact filenames. This patch makes follow argument to be treated like file pattern same way like log treats their arguments. It preserves current behaviour of follow() matching paths relative to the repository root by default.

i18n-ja: synchronized with b930d4ef7739

histedit: improve discoverability of edit commit message

revset: cache smartset's min/max As the content of a smartset never changes, min and max will never change either. This will save us time when this function is called multiple times. This is relevant for issue4782 but does not fix it.

spelling: behaviour -> behavior

glossary: fixup use of periods at end of entries * Include trailing period for entries * Move trailing period inside quotations for example sentences

histedit: fix grammar for help for safety aborts

revset: mark reachablerootspure as private

reachableroots: construct and sort baseset in revset module This can remove the dependency from changelog to revset, which seems a bit awkward for me.

reachableroots: use smartset min smartset min are likely to be optimised, cached or other magical property.

graphmod: stop building a list out of branchgroupiter Not only does this defeat the purpose of having `branchgroupiter` an iterable, but this also makes revs something else than a smartset preventing to use revs.min() in the call to reachablerevs.

reachableroots: sort the smartset in the pure version too Changeset be8a4e0800d8 uses smartset lazy sorting for the C version. We need to apply the same to the pure version for consistency. This is fixing the tests with --pure.

builddeb: actually run make when building the deb (issue4778) As of this change, we no longer produce broken debs, but I've already got followups written that will produce much more standard-looking packages and test the resulting packages.

docker-debian-jessie: fix errant mkdir

makefile: rename debian-jessie to deb `make debian-jessie` was a poor name choice on my part, because it really builds a package for the current debian platform, whatever that happens to be, rather than a specific platform. Rename it to just `make deb` so that it's a little clearer. As a result, it now places built debs in a "debian-unknown" directory.

run-tests: ignore failed removal of nonexistent installerrs When running tests with -j100 or so on a large machine, I see this os.remove call failing semi-regularly. Since it's not really a problem when the file is already gone, just suppress the error in that case.

templater: move verbosity-to-template matcher to constructor We're going to reuse this and it's silly for it to be done per-cset anyway.

templater: remove pseudo-ternary

templater: fix variable name that shadows built-in

templater: minor whitespace tweaks

lock: make trylock private

lock: loop a finite number of times in trylock (issue4787) ..before falling back to lock, which will sleep. This only addresses the CPU consumption of the reported issue.

reachableroots: silence warning of implicit integer narrowing issued by clang Tested with CFLAGS=-Wshorten-64-to-32 CC=clang which is the default of Mac OS X. Because a valid revnum shouldn't exceed INT_MAX, we don't need long width for large tovisit array.

reachableroots: narrow scope of minidx variable minidx is never used if includepath is false, so let's define it where it is used.

convert: fix syncing deletes from p2 merge commit Recently we fixed converting merges to correctly sync changes from p2. We missed the case of deletes though (so p2 deleted a file that p1 had not yet deleted, and the file does not belong to the source). The fix is to detect when p2 doesn't have the file, so we just sync it as a delete to p1 in the merge. Updated the test, and verified it failed before the fix.

convert: add convert.git.skipsubmodules option This adds an option to not pull in gitsubmodules during a convert. This is useful when converting large git repositories where gitsubmodules were allowed historically, but are no longer wanted.

paths: move path validation logic to its own function Hard coding the '.hg' path in the paths class made it difficult for the hggit extension to pull from gitrepos. This patch moves the logic out to it's own function so extensions can add additional checks to what is a valid path (i.e. a git repo is valid when hggit is enabled).

parsers: avoid int/unsigned conversions Detected with make local CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter' CC=clang

base85: fix comparison of an int and a long Caught using make local CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter' CC=clang

progress: restore testedwith to the stub Without this, hg always claims progress is at fault for problems, which is completely crazy.

hgweb: make refresh interval configurable hgwebdir refreshes the set of known repositories periodically. This is necessary because refreshing on every request could add significant request latency. More than once I've found myself wanting to tweak this interval at Mozilla. I've also wanted the ability to always refresh (often when writing tests for our replication setup). This patch makes the refresh interval configurable. Negative values indicate to always refresh. The default is left unchanged.

filemerge: add 'union' merge to internal merge tool 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging e.g. the Changelog file.

filemerge: split internal merge into api entry point and internal helper This is a step toward adding 'union merge' to the internal merge tool. 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging to e.g. the Changelog file.

simplemerge: add 'mode' kwarg which - when 'union' - suppresses markers This is a step toward adding 'union merge' to the internal merge tool. 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging to e.g. the Changelog file. Note: the identifiers with underscores in this commit are all using existing variable names and/or keyword arguments.

hghave: remove quiet option The --quiet flag appears to be unused. Kill it.

hghave: move feature checking into hghave.py Upcoming patches will kill hghave (the script - not hghave.py) and will move to a model where requirements checking is performed as a function call. Start diminishing the utility of hghave by moving some code to hghave.py.

tests: move '#require bzr' into .t files The bzr tests are the only .t tests calling hghave (aside from the tests testing hghave itself). This pattern is a one-off and prevents desired refactors to how requirements checking is integrated into tests. Use the convention used everywhere else.

commands.push: use "pushloc" path attribute Path instances now have a "pushloc" attribute that exposes the path to be used for pushing. The "default-push" path is automatically captured in the "pushloc" attribute of the "default" path. Drop references to "default-push" and use the "pushloc" attribute for looking up where to push to.

ui: capture push location on path instances Currently, we treat "default" and "default-push" as separate paths, even though they are the same logical entity but with different paths for different operations. Because they are the same entity and because we will eventually be implementing an official mechanism for declaring push URLs for paths, we establish a "pushloc" attribute on path instances. We populate this attribute on the "default" path with the "default-push" value, if present. This will enable consumers stop referencing "default-push" which will make their code simpler.

worker: use multiprocessing to find cpu count The multiprocessing package was added in Python 2.6. The implementation of worker.countcpus() is very similar to multiprocessing.cpu_count(). Ditch our one-off code. multiprocessing does result in a number of imports. However, the lazy importer ensures that we don't import anything until cpu_count() is called. Furthermore, if we are doing something with multiple cores, chances are the time of that operation will dwarf the import time, so module bloat isn't a concern here.

import-checker: ensure multiprocessing is treated as from stdlib On my linux machines multiprocessing appears to defeat the logic in import-checker to detect stdlib modules. Since we now only use versions of Python which ship with multiprocessing, let's just whitelist the module.

reachableroots: use baseset lazy sorting smartset sorting is lazy (so faster in some case) and better (informs that the set is sorted allowing some optimisation). So we rely on it directly. Some test output are updated because we now have more information (ordering).

baseset: keep the input set around Baseset needs a list to operate, but will convert that list back to a set for membership testing. It seems a bit silly to convert the set into a list to convert it back afterward.

reachableroots: unroll loop that checks if one of parents is reachable The difference is small, but fewer loops should be better in general: revset #0: 0::tip 0) 0.001609 1) 0.001510 93%

reachableroots: handle error of PyList_Append()

commands.push: use paths API ui.path instances now collect most of the data used by commands.push(). Move away from ui.expandpath() and call ui.paths.getpath() to get a path instance. Some "pushing to" output was dropped as one test demonstrates. I believe the dropped message was redundant with the error message and the change to be acceptable.

ui: move URL and path detection into path API ui.expandpath() has code for recognizing URLs or local filesystem paths. Our goal is to use ``path`` class instances everywhere a path is represented. Changing ui.expandpath() to return path instances is a lot of work. Our goal is to slowly marginalize it by moving logic into the paths API and to convert callers to the paths API. Many callers of ui.expandpath() pass in a value that could be a local filesystem path or URI. We move the detection of these strings from ui.expandpath() to paths.getpath() and path.__init__(). To do this properly in a way that is compatible with future callers, we need to parse the "#branch" syntax out of locations. This is a bit complicated, but it is necessary. The code for URL parsing is essentially a copy of hg.parseurl(). Once all consumers are speaking the paths API, it is likely that this function won't be called any more and it can be deleted.

reachableroots: return list of revisions instead of set Now we don't need a set of reachable revisions, and the caller wants a sorted list of revisions, so constructing a set is just a waste of time. revset #0: 0::tip 2) 0.002536 3) 0.001598 63% PyList_New() should set an appropriate exception on error, so we don't need to call PyErr_NoMemory() manually. This patch lacks error handling of PyList_Append() as it was before for PySet_Add(). It should be fixed later.

reachableroots: use internal "revstates" array to test if rev is reachable This is faster than using PySet_Contains(). revset #0: 0::tip 1) 0.003678 2) 0.002536 68%

reachableroots: use internal "revstates" array to test if rev is a root The main goal of this patch series is to reduce the use of PyXxx() function that is likely to require ugly error handling and inc/decref. Plus, this is faster than using PySet_Contains(). revset #0: 0::tip 0) 0.004168 1) 0.003678 88% This patch ignores out-of-range roots as they are in the pure implementation. Because reachable sets are calculated from heads, and out-of-range heads raise IndexError, we can just take out-of-range roots as unreachable. Otherwise, the test of "hg log -Gr '. + wdir()'" would fail. "heads" argument is changed to a list. Should we have to rename the C function as its signature is changed?

parsers: set exception when there's too little string data to extract parents Previously we were returning NULL from this function without actually setting up an exception. This fixes that problem, which was detected with cpychecker.

parsers: drop spurious check of readlen value We're about to check if len < 40 after assigning readlen to len, which means that if len < 40 we'll still abort, but I'm about to add a sensible exception to that failure, so let's just discard this useless check.

pathencode: check result of .digest() method in sha1hash Without this it was theoretically possible .digest() would fail and return NULL but we wouldn't notice. Detected with cpychecker.

parsers: correctly decref normed value after PyDict_SetItem Previously we were leaving this PyObject* with a refcount that was one too high. Detected with cpychecker.