bundle2: don't try to recover from a GeneratorExit (issue4785) GeneratorExit means the other end of the conversation has already stopped listening, so don't try and yield out error information. Instead, just let the GeneratorExit propagate normally. This should resolve esoteric issues observed with servers that have aggressive timeouts waiting for data to send to clients logging internal Python errors[0]. This has been observed with both gunicorn's gevent worker model and with scm-manager's built-in webserver (which itself is something sitting inside jetty.) 0: Exception RuntimeError: 'generator ignored GeneratorExit' in <generator object getchunks at 0x7fd2f6c586e0> ignored

revset: fix resolving strings from a list When using multiple revsets that get optimized into a list (like hg log -r r1235 -r r1237 in hgsubversion), the revset list code was assuming the strings were resolvable via repo[X]. hgsubversion and other extensions override def stringset() to allow processing different revision identifiers (such as r1235 or g<githash>), and there for the _list() implementation was circumventing that resolution. The fix is to just call stringset(). The default implementaiton does the same thing that _list was already doing (namely repo[X]). This has always been broken, but it was recently exposed by 4ee4f7415095 which made "--rev X --rev Y" produce a combined revset "X | Y".

dispatch: error out on invalid -R path even if optionalrepo (issue4805) (BC) Before this patch, repo could be set to None for wrong -R. It's okay for commands that can reject repo=None, but the command server have a problem because: - it accepts repo=None for "unbound" mode - and it reenters dispatch() where repo object is created for cwd by default Test outputs are changed because the error is detected earlier. I think new message is better than ".hg not found".

test-ssh: remove superfluous triple slashes from absolute path test Because pwd should start with "/", "/`pwd`" is enough to make absolute path.

vagrant: update to official Debian 8.1 base box Debian now rolls their own official Vagrant base boxes, so use that. At the same time, we're updating from Debian 7.4 (wheezy) to 8.1 (jessie), and switching from 32-bit to 64-bit (Debian does not provide 32-bit base boxes).

buildrpm: mkdir -p two needed directories (issue4779) Without this, building RPMs can fail.

merge with stable

hghave: use subprocess instead of os.popen os.popen was deprecated in Python 2.6 in favor of subprocess, so let's move into the future.

hgweb: move archive related attributes to requestcontext As part of this, "archive_specs" was renamed to "archivespecs" to align with naming conventions. "archive_specs" didn't technically need to be moved from hgweb. But it seemed to make sense to have all the archive code in the same class. As part of this, hgweb.configlist is no longer used, so it was deleted.

hgweb: move some config options to requestcontext Various config options from the repository were stored on the hgweb instance. While unlikely, there could be race conditions between a new request updating these values and an in-flight request seeing both old and new values, leading to weird results. We move some of options/attributes from hgweb to requestcontext. As part of this, we establish config* helpers on requestcontext. As part of the move, we changed int() casts to configint() calls. The int() usage likely predates the existence of configint(). We also removed config option updating from once every refresh to every request. I don't believe obtaining config options is expensive enough to warrant only doing when the repository has changed. The excessive use of object.__setattr__ is unfortunate. But it will eventually disappear once the proxy is no longer necessary.

hgweb: establish class for holding per request context Currently, hgweb applications have many instance variables holding mutated state. This is somewhat problematic because multiple threads may race accessing or changing this state. This patch starts a series that will add more thread safety to hgweb applications. It will do this by moving mutated state out of hgweb and into per-request instances of the newly established "requestcontext" class. Our new class currently behaves like a proxy to hgweb instances. This should change once all state is captured in it instead of hgweb. The effectiveness of this proxy is demonstrated by passing instances of it - not hgweb instances/self - to various functions.

hgweb: don't make request optional The "request" argument has been optional in this code since it was introduced in 38170eeed18c in 2009. There are no consumers that don't pass this argument. So don't make it an optional argument.

hgweb: add some documentation It took longer than I wanted to grok how the various parts of hgweb worked. So I added some class and method documentation to help whoever hacks on this next.

hgweb: limit branches shown on summary page to 10 Tags and bookmarks on summary page are already limited to 10, let's limit branches as well. Each of the blocks (tags, bookmarks, branches) currently has a link to the full list.

hgweb: use webutil.branchentries for branches on summary page This allows showing correct status for each branch, which was missing on /summary. Usually that means that closed branches get the same css class (resulting in e.g. different color/shade) as they do on /branches page. The sorting of the branches on summary page also changes and is now the same as on /branches page: closed branches are now at the end of the list.

hgweb: move branchentries code from webcommands to webutil

templater: add optional timezone argument to localdate() The keyword extension uses "utcdate" for a different function, so we can't add new "utcdate" filter or function. Instead, this patch extends "localdate" to a general timezone converter.

templater: port localdate filter to a function It will be extended to accept a timezone argument.

util: extract function that parses timezone string It will be used to parse a timezone argument passed to a template function.

templater: drop unneeded destructuring of argument tuple at buildfilter Because evalfuncarg() accepts an argument tuple, there is little meaning to pass (func, data, filt) in place of ((func, data), filt).

templater: extract helper that evaluates filter or function argument It will be used to get a date tuple from an argument. Perhaps some of "stringify(args[n][0], ...)" can be replaced by this function.

contrib: add showstack extension This allows getting a Python stack trace at any time on Unix by hitting Ctrl-\ (or Ctrl-T on BSDs). Useful for debugging mysterious hangs on the fly. Sample output: $ hg log -k nosuchmessage ^\ File "/home/mpm/hg/mercurial/revset.py", line 3089, in _iterfilter if cond(x): File "/home/mpm/hg/mercurial/util.py", line 415, in f cache[arg] = func(arg) File "/home/mpm/hg/mercurial/revset.py", line 1215, in matches for t in c.files() + [c.user(), c.description()]) File "/home/mpm/hg/mercurial/context.py", line 525, in files return self._changeset[3] File "/home/mpm/hg/mercurial/util.py", line 531, in __get__ result = self.func(obj) File "/home/mpm/hg/mercurial/context.py", line 498, in _changeset return self._repo.changelog.read(self.rev()) File "/home/mpm/hg/mercurial/changelog.py", line 338, in read text = self.revision(node) File "/home/mpm/hg/mercurial/revlog.py", line 1092, in revision bins = self._chunks(chain) File "/home/mpm/hg/mercurial/revlog.py", line 1013, in _chunks ladd(decompress(buffer(data, chunkstart - offset, chunklength))) File "/home/mpm/hg/mercurial/revlog.py", line 91, in decompress return _decompress(bin) ----

Added signature for changeset 1a45e49a6bed

Added tag 3.5.1 for changeset 1a45e49a6bed

hgweb: fix trust of templates path (BC) Long ago we disabled trust of the templates path with a comment describing the (insecure) behavior before the change. At some later refactor, the code was apparently changed back to match the comment, unaware that the intent of the comment was to describe the behavior to avoid. This change disables the trust and updates the comment to explicitly say not only what the old problem was, but also that it was in fact a problem and the action taken to prevent it. Impact: prior to this change, if you had a UNIX-based hgweb server where users can write hgrc files, those users could potentially read any file readable by the web server. This is marked as a backwards compatibility issue because people may have configured templates without proper trust settings. Issue spotted by Greg Szorc.

wix: avoid an abort with 'hg help -k foo' Previously: $ /c/Program\ Files/Mercurial/hg help -k merge-tools abort: No such file or directory: c:\Program Files\Mercurial\help\scripting.txt The Inno installer seems OK, but the TortoiseHg required the same fix. That got queued with an additional change of 'helpFolder.guid' in guids.wxi (probably by Steve). I'm not sure if that is necessary here too.

revlog: add an aggressivemergedelta option This adds an option for delta'ing against both p1 and p2 when applying merge revisions and picking whichever is smallest. Some before and after stats on manifest.d size: internal large repo: before: 1.2 GB after: 930 MB mozilla-central: before: 261 MB after: 92 MB

revlog: change generaldelta delta parent heuristic The old generaldelta heuristic was "if p1 (or p2) was closer than the last full text, use it, otherwise use prev". This was problematic when a repo contained multiple branches that were very different. If commits to branch A were pushed, and the last full text was branch B, it would generate a fulltext. Then if branch B was pushed, it would generate another fulltext. The problem is that the last fulltext (and delta'ing against `prev` in general) has no correlation with the contents of the incoming revision, and therefore will always have degenerate cases. According to the blame, that algorithm was chosen to minimize the chain length. Since there is already code that protects against that (the delta-vs-fulltext code), and since it has been improved since the original generaldelta algorithm went in (2011), I believe the chain length criteria will still be preserved. The new algorithm always diffs against p1 (or p2 if it's closer), unless the resulting delta will fail the delta-vs-fulltext check, in which case we delta against prev. Some before and after stats on manifest.d size. internal large repo old heuristic - 2.0 GB new heuristic - 1.2 GB mozilla-central old heuristic - 242 MB new heuristic - 261 MB The regression in mozilla central is due to the new heuristic choosing p2r as the delta when it's closer to the tip. Switching the algorithm to always prefer p1r brings the size back down (242 MB). This is result of the way in which mozilla does merges and pushes, and the result could easily swing the other direction in other repos (depending on if they merge X into Y or Y into X), but will never be as degenerate as before. I future patch will address the regression by introducing an optional, even more aggressive delta heuristic which will knock the mozilla manifest size down dramatically.

revlog: move textlen calculation to be above delta chooser This moves the textlen calculation to be above the delta chooser. Since textlen is needed for calling isgooddelta, we need it above the delta chooser so future patches can call isgooddelta.

revlog: move delta check to it's own function This moves the delta vs fulltext comparison to its own function. This will allow us to reuse the function in future patches for more efficient delta choices. As a side effect, this will also allow extensions to modify our delta criteria.

monoblue: add archive links on summary page

help: distinguish sections when multiple match (issue4802)

merge with stable

hghave: add a check for docker support This currently refuses to operate if on a non-Linux host. I suspect that Docker running on FreeBSD 11 or on an Illumos derivative would work fine, but I don't have ready access to such a system. On OS X using boot2docker (I used a hacky xhyve-based one for testing), it won't work because $TESTTEMP doesn't end up inside the set of directories that get forwarded to the boot2docker VM, so you can't actually drop debs in the $TESTTEMP at all. It would be possible (probably even trivial) to hack around this by using a randomly-named temporary directory inside the working directory, but that seems unlikely to be useful enough to justify the ugliness.

hghave: add a check for debian packaging tools

run-tests: add support for marking tests as very slow I want to add tests for our packaging rules, but those necessarily run a whole build, or possibly two if both native packaging and docker are available. This lets us flag such tests with a `#require slow` so that they don't unnecessarily slow down normal test runs.

builddeb: rework how output dir and platform are specified This makes it possible to write tests for both builddeb and dockerdeb that actually build .debs and then sanity check the contents.

parsers: fix two cases of unsigned long instead of Py_ssize_t We had to do this before because Python 2.4 didn't understand the n format specifier in Py_BuildValue and friends. We no longer have that problem.

templatefilters: remove redundant 'date' and 'strip' filters These filters are defined as 'date()' and 'strip()' functions. Help messages are moved to the corresponding functions.

templater: introduce unified filter syntax for unary functions "filter(expr)" is allowed already. This patch adds the opposite, "expr|func".

templater: inline getfilter() to buildfilter() This prepares for the unified filter syntax that will be introduced by the next patch.

clone: fix updaterev to update to latest branch changeset (issue4528) Before this patch if clone --updaterev points to branch which head on src repo wasnt in dest repo, clone updated dest repo to default branch. After applying this patch, if changeset from src repo pointing at given branch is not in dest repo, it searches for changeset pointing for given branch locally in dest repo. Lookup in destination repo: 559: uprev = destrepo.lookup(update) is wrapped by try/except block to preserve current behaviour when given revset to -u is not found - it will not fail,but silently update dest repo to head of default branch.

revsets: makes follow() supports file patterns (issue4757) (BC) Before this patch, follow only supports full, exact filenames. This patch makes follow argument to be treated like file pattern same way like log treats their arguments. It preserves current behaviour of follow() matching paths relative to the repository root by default.

i18n-ja: synchronized with b930d4ef7739

histedit: improve discoverability of edit commit message

revset: cache smartset's min/max As the content of a smartset never changes, min and max will never change either. This will save us time when this function is called multiple times. This is relevant for issue4782 but does not fix it.

spelling: behaviour -> behavior

glossary: fixup use of periods at end of entries * Include trailing period for entries * Move trailing period inside quotations for example sentences

histedit: fix grammar for help for safety aborts

revset: mark reachablerootspure as private

reachableroots: construct and sort baseset in revset module This can remove the dependency from changelog to revset, which seems a bit awkward for me.

reachableroots: use smartset min smartset min are likely to be optimised, cached or other magical property.

graphmod: stop building a list out of branchgroupiter Not only does this defeat the purpose of having `branchgroupiter` an iterable, but this also makes revs something else than a smartset preventing to use revs.min() in the call to reachablerevs.

reachableroots: sort the smartset in the pure version too Changeset be8a4e0800d8 uses smartset lazy sorting for the C version. We need to apply the same to the pure version for consistency. This is fixing the tests with --pure.

builddeb: actually run make when building the deb (issue4778) As of this change, we no longer produce broken debs, but I've already got followups written that will produce much more standard-looking packages and test the resulting packages.

docker-debian-jessie: fix errant mkdir

makefile: rename debian-jessie to deb `make debian-jessie` was a poor name choice on my part, because it really builds a package for the current debian platform, whatever that happens to be, rather than a specific platform. Rename it to just `make deb` so that it's a little clearer. As a result, it now places built debs in a "debian-unknown" directory.

run-tests: ignore failed removal of nonexistent installerrs When running tests with -j100 or so on a large machine, I see this os.remove call failing semi-regularly. Since it's not really a problem when the file is already gone, just suppress the error in that case.

templater: move verbosity-to-template matcher to constructor We're going to reuse this and it's silly for it to be done per-cset anyway.

templater: remove pseudo-ternary

templater: fix variable name that shadows built-in

templater: minor whitespace tweaks

lock: make trylock private

lock: loop a finite number of times in trylock (issue4787) ..before falling back to lock, which will sleep. This only addresses the CPU consumption of the reported issue.

reachableroots: silence warning of implicit integer narrowing issued by clang Tested with CFLAGS=-Wshorten-64-to-32 CC=clang which is the default of Mac OS X. Because a valid revnum shouldn't exceed INT_MAX, we don't need long width for large tovisit array.

reachableroots: narrow scope of minidx variable minidx is never used if includepath is false, so let's define it where it is used.

convert: fix syncing deletes from p2 merge commit Recently we fixed converting merges to correctly sync changes from p2. We missed the case of deletes though (so p2 deleted a file that p1 had not yet deleted, and the file does not belong to the source). The fix is to detect when p2 doesn't have the file, so we just sync it as a delete to p1 in the merge. Updated the test, and verified it failed before the fix.

convert: add convert.git.skipsubmodules option This adds an option to not pull in gitsubmodules during a convert. This is useful when converting large git repositories where gitsubmodules were allowed historically, but are no longer wanted.

paths: move path validation logic to its own function Hard coding the '.hg' path in the paths class made it difficult for the hggit extension to pull from gitrepos. This patch moves the logic out to it's own function so extensions can add additional checks to what is a valid path (i.e. a git repo is valid when hggit is enabled).

parsers: avoid int/unsigned conversions Detected with make local CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter' CC=clang

base85: fix comparison of an int and a long Caught using make local CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter' CC=clang

progress: restore testedwith to the stub Without this, hg always claims progress is at fault for problems, which is completely crazy.

hgweb: make refresh interval configurable hgwebdir refreshes the set of known repositories periodically. This is necessary because refreshing on every request could add significant request latency. More than once I've found myself wanting to tweak this interval at Mozilla. I've also wanted the ability to always refresh (often when writing tests for our replication setup). This patch makes the refresh interval configurable. Negative values indicate to always refresh. The default is left unchanged.

filemerge: add 'union' merge to internal merge tool 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging e.g. the Changelog file.

filemerge: split internal merge into api entry point and internal helper This is a step toward adding 'union merge' to the internal merge tool. 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging to e.g. the Changelog file.

simplemerge: add 'mode' kwarg which - when 'union' - suppresses markers This is a step toward adding 'union merge' to the internal merge tool. 'union merge' is a merge strategy which adds both left and right hand side of a conflict region. Git implements this merge strategy which is very practical to have for merging to e.g. the Changelog file. Note: the identifiers with underscores in this commit are all using existing variable names and/or keyword arguments.

hghave: remove quiet option The --quiet flag appears to be unused. Kill it.

hghave: move feature checking into hghave.py Upcoming patches will kill hghave (the script - not hghave.py) and will move to a model where requirements checking is performed as a function call. Start diminishing the utility of hghave by moving some code to hghave.py.

tests: move '#require bzr' into .t files The bzr tests are the only .t tests calling hghave (aside from the tests testing hghave itself). This pattern is a one-off and prevents desired refactors to how requirements checking is integrated into tests. Use the convention used everywhere else.

commands.push: use "pushloc" path attribute Path instances now have a "pushloc" attribute that exposes the path to be used for pushing. The "default-push" path is automatically captured in the "pushloc" attribute of the "default" path. Drop references to "default-push" and use the "pushloc" attribute for looking up where to push to.

ui: capture push location on path instances Currently, we treat "default" and "default-push" as separate paths, even though they are the same logical entity but with different paths for different operations. Because they are the same entity and because we will eventually be implementing an official mechanism for declaring push URLs for paths, we establish a "pushloc" attribute on path instances. We populate this attribute on the "default" path with the "default-push" value, if present. This will enable consumers stop referencing "default-push" which will make their code simpler.

worker: use multiprocessing to find cpu count The multiprocessing package was added in Python 2.6. The implementation of worker.countcpus() is very similar to multiprocessing.cpu_count(). Ditch our one-off code. multiprocessing does result in a number of imports. However, the lazy importer ensures that we don't import anything until cpu_count() is called. Furthermore, if we are doing something with multiple cores, chances are the time of that operation will dwarf the import time, so module bloat isn't a concern here.

import-checker: ensure multiprocessing is treated as from stdlib On my linux machines multiprocessing appears to defeat the logic in import-checker to detect stdlib modules. Since we now only use versions of Python which ship with multiprocessing, let's just whitelist the module.

reachableroots: use baseset lazy sorting smartset sorting is lazy (so faster in some case) and better (informs that the set is sorted allowing some optimisation). So we rely on it directly. Some test output are updated because we now have more information (ordering).

baseset: keep the input set around Baseset needs a list to operate, but will convert that list back to a set for membership testing. It seems a bit silly to convert the set into a list to convert it back afterward.

reachableroots: unroll loop that checks if one of parents is reachable The difference is small, but fewer loops should be better in general: revset #0: 0::tip 0) 0.001609 1) 0.001510 93%

reachableroots: handle error of PyList_Append()

commands.push: use paths API ui.path instances now collect most of the data used by commands.push(). Move away from ui.expandpath() and call ui.paths.getpath() to get a path instance. Some "pushing to" output was dropped as one test demonstrates. I believe the dropped message was redundant with the error message and the change to be acceptable.

ui: move URL and path detection into path API ui.expandpath() has code for recognizing URLs or local filesystem paths. Our goal is to use ``path`` class instances everywhere a path is represented. Changing ui.expandpath() to return path instances is a lot of work. Our goal is to slowly marginalize it by moving logic into the paths API and to convert callers to the paths API. Many callers of ui.expandpath() pass in a value that could be a local filesystem path or URI. We move the detection of these strings from ui.expandpath() to paths.getpath() and path.__init__(). To do this properly in a way that is compatible with future callers, we need to parse the "#branch" syntax out of locations. This is a bit complicated, but it is necessary. The code for URL parsing is essentially a copy of hg.parseurl(). Once all consumers are speaking the paths API, it is likely that this function won't be called any more and it can be deleted.

reachableroots: return list of revisions instead of set Now we don't need a set of reachable revisions, and the caller wants a sorted list of revisions, so constructing a set is just a waste of time. revset #0: 0::tip 2) 0.002536 3) 0.001598 63% PyList_New() should set an appropriate exception on error, so we don't need to call PyErr_NoMemory() manually. This patch lacks error handling of PyList_Append() as it was before for PySet_Add(). It should be fixed later.

reachableroots: use internal "revstates" array to test if rev is reachable This is faster than using PySet_Contains(). revset #0: 0::tip 1) 0.003678 2) 0.002536 68%

reachableroots: use internal "revstates" array to test if rev is a root The main goal of this patch series is to reduce the use of PyXxx() function that is likely to require ugly error handling and inc/decref. Plus, this is faster than using PySet_Contains(). revset #0: 0::tip 0) 0.004168 1) 0.003678 88% This patch ignores out-of-range roots as they are in the pure implementation. Because reachable sets are calculated from heads, and out-of-range heads raise IndexError, we can just take out-of-range roots as unreachable. Otherwise, the test of "hg log -Gr '. + wdir()'" would fail. "heads" argument is changed to a list. Should we have to rename the C function as its signature is changed?

parsers: set exception when there's too little string data to extract parents Previously we were returning NULL from this function without actually setting up an exception. This fixes that problem, which was detected with cpychecker.

parsers: drop spurious check of readlen value We're about to check if len < 40 after assigning readlen to len, which means that if len < 40 we'll still abort, but I'm about to add a sensible exception to that failure, so let's just discard this useless check.

pathencode: check result of .digest() method in sha1hash Without this it was theoretically possible .digest() would fail and return NULL but we wouldn't notice. Detected with cpychecker.

parsers: correctly decref normed value after PyDict_SetItem Previously we were leaving this PyObject* with a refcount that was one too high. Detected with cpychecker.

parsers: fix two leaks in index_ancestors Both happy paths through this function leaked the returned list: 1) If the list was of size 0 or 1, it was retained an extra time and then returned. 2) If the list was passed to find_deepest, it was never released before exiting this function. Both paths spotted by cpychecker.

merge with stable

merge with i18n

i18n-pt_BR: synchronized with d14590f90cb6

reachableroots: extend "revstates" to array of bit flags

reachableroots: rename "seen" array to "revstates" for future extension It will be an array of bit flags, SEEN | ROOT | REACHABLE.

reachableroots: give anonymous name to short-lived "numheads" variable I'll reuse it for the length of the roots list.

reachableroots: reduce nesting level by jumping to next iteration by continue This can eliminate lines over 80 columns. No code change except for the outermost "if" condition.

test-convert-git: work around output format changes in git git version 2.4.3: --- /home/augie/hg/tests/test-convert-git.t +++ /home/augie/hg/tests/test-convert-git.t.err @@ -659,7 +659,7 @@ $ touch a && git add a && git commit -am "commit a" [master (root-commit) 8ae5f69] commit a Author: nottest <test@example.org> 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 a $ cd .. $ git clone git-repo7 git-repo7-client git version 1.7.9.5: --- /home/augie/hg/tests/test-convert-git.t +++ /home/augie/hg/tests/test-convert-git.t.err @@ -659,7 +659,7 @@ $ touch a && git add a && git commit -am "commit a" [master (root-commit) 8ae5f69] commit a Author: nottest <test@example.org> - 1 file changed, 0 insertions(+), 0 deletions(-) + 0 files changed create mode 100644 a $ cd .. $ git clone git-repo7 git-repo7-client I don't know when this changed in git and am too lazy to try and bisect it, so just work around the change.

histedit: correct spelling etc in more comments Spotted during review of another patch.

histedit: add a missing "s" in a comment

convert: fix convert dropping p2 contents during filemap merge When converting a merge commit using a filemap convert (i.e. when moving contents from the root of the repo into subdir1/), convert would silently drop the entire contents of the target repo's p2. This was because when it built the target commit, it did so by taking the target p1 and adding only the files that changed in the source repo's merge commit. This breaks in the case where the target repo has files that are unrelated to the source repo (like in the case where you use convert to import a repo as a subdirectory of another). The fix is to use Mercurial's merge logic to detect which files in p2 we should carry over to the merge. It follows three rules: 1) if the file belongs to the source, don't try to merge it. Rely on the list of files provided to putcommit to be correct. 2) if the file requires merging or user input (change vs deleted), throw an exception. We don't have enough info to do this. 3) if p2 has the newest, non-merge-requiring version of the file, take it I've also added a test to cover this issue.

convert: implements targetfilebelongstosource for filemap source This is an implementation of the new targetfilebelongstosource() function for the filemapper. It simply checks if the given file name is prefixed by any of the rename destinations. It is not a perfect implementation since it doesn't account for the filemap specifying includes or excludes, but that makes the problem much harder, and this implementation should suffice for most cases.

convert: add function to test if file is from source This adds a base implementation of a function that tests if a given file from a target repo came from the source repo. This will be used later to detect which files did not come from the source repo during a merge, so we can merge those files correctly instead of dropping them.

revsetbenchmarks: run make after update so that C extensions are built

reachableroots: fix memleak of integer objects at includepath loop In the first visit loop, val is decref-ed correctly after PySet_Add(). Let's do the same for the includepath loop.

reachableroots: bail if integer object cannot be allocated This patch also replaces Py_XDECREF() by Py_DECREF() because we known "val" and "p" are not NULL. BTW, we can eliminate some of these allocation and error handling of int objects if the internal "seen" array has more information. For example, enum { SEEN = 1, ROOT = 2, REACHABLE = 4 }; /* ... build ROOT mask from roots argument ... */ if (seen[revnum + 1] & ROOT) { /* instead of PySet_Contains(roots, val) */ >From my quick hack, it is 2x faster.

devel-warn: issue a warning when writing bookmarks without holding the wlock I saw an issue in an extension that we develop where we were writing bookmarks without holding the wlock. Another extension was taking a lock at the same time and wiped out the bookmarks we were about to write. This patch adds a devel-warning to urge people to fix their invalid code.

merge with stable

rebase: lock the repo during the full rebase operation Running `hg pull --rebase` would move bookmarks without any repository locking. So we now lock the repository. For good measure and avoiding sneaky race conditions, we lock the repository for the whole operation. There is no code change besides the indentation.

update: wlock the repo for the whole 'hg update' command The update command is touching the repository and should lock it for the length of its operations. Equally importantly, it should lock the repository when it is writing bookmarks. It wasn't doing so until now, leaving doors open for all kinds of drunk beaver parties. This results in some minor tests changes, and the fixing of a couple of bugs from race conditions. Code does not receive any changes beside extra indentation.

help: fix typo in scripting documentation

hg: avoid auto sharing when the clone destination is remote Before this patch, when auto sharing is enabled, 'hg.clone()' tries to create local clone regardless of locality of the clone destination on the host, and causes failure. To avoid auto sharing when the clone destination is remote, this patch adds examination of 'islocal(dest)' before auto sharing in 'hg.clone()'. 'islocal(dest)' is examined after 'sharepool', because: - the former is more expensive than the latter - without enabling share extension, the later is always negative

largefiles: ensure lfutil.getstandinmatcher() only matches standins Previously, simply having the largefiles extension loaded without any largefiles added would crash when amending with -I. The problem was with no files in the matcher, the pattern list of files joined with 'standindir' was empty, and scmutil.match() would match everything. In lfutil.composestandinmatcher(), the match function is used to test if the file is a standin, and after getting a false positive, proceeds to call lfutil.splitstandin(). This returns None because it isn't a standin, which blows up when passed to rmatcher.matchfn(). Manually overriding _always in getstandinmatcher() probably isn't necessary anymore, but we leave well enough alone on stable. This regressed in ab618e52788a.

dirstate: add --minimal flag to debugrebuilddirstate On repositories with hundreds of thousands of files, hg debugrebuilddirstate causes every dirstate entry to be marked lookup, and the next hg status can take many minutes. This adds a --minimal flag that allows us to only rebuild the parts of the dirstate that are inconsistent. This follows two rules: 1) If a file is in the dirstate but not in the parent manifest, and it is not marked 'add', it is busted and we should drop it. 2) If a file is not in the dirstate at all, but it is in the parent manifest, it should be added to the dirstate and we need to mark it as lookup. This allows us to fix repositories where the dirstate doesn't match the manifest much more quickly. Tested by artificially adding bad dirstate entries (via code) for both cases above.

tests: make filterpyflakes.py read target files relatively to cwd Before this patch, 'filterpyflakes.py' reads target files relatively to own location. But this prevents third party tools from using it in own source tree, because their files are placed separately from 'filterpyflakes.py'. In fact, 'test-check-pyflakes.t', which is the only user of 'filterpyflakes.py', changes current working directory (cwd) to the root of "test target" source tree before using it. Therefore, composing the root of source tree in 'filterpyflakes.py' is redundant. This patch makes 'filterpyflakes.py' read target files relatively to cwd by invoking 'open()' without any path composition. This also removes importing 'os' module, because there is no user of it after this patch. This is a one of preparation of issue4677.

monoblue: remove duplicate font-family property from td.source

monoblue: remove unused elements and related css Since 91b0ada2d94b and c9b88695d894, when monoblue was introduced, the code this patch removes was untouched. Presumably, there supposed to be nice graphics in the screen corners, but there never were due to: - the css being commented out - ids of the elements and of the css selectors being different - and the png files absent The "corner" elements were unstyled and didn't affect the rest of the page, so I think it's safe to remove all this.

revpair: restrict odd-range handling to top-level x:y expression (issue4774) The odd-range hack was introduced by 2a0efa1112ac for backward compatibility, but it was too widely applied. I've checked cmdutil.revpair() at 1.6, and found that ".:", ":0" and ":" are also handled as pairs. So let's enable the hack only for "x:y", "x:", "y:" and ":". test-revset.t is updated because "tip^::tip^ or tip^" shouldn't be taken as an odd range. This patch adds "tip^:tip^" instead. This patch is written for the default branch because parse() of the stable branch lacks compatibility hack for "foo+bar" tag. If we want to mitigate the issue in stable, we can add something like "and '::' in revs[0]".

revpair: update test to make a difference if odd range not handled specially It was added at 2a0efa1112ac, but there was no difference between -r2 and -r2:2 because the working directory was clean.

reachableroots: verify type of each item of heads argument Though PyInt_AS_LONG() can return a value no matter if it isn't an int object, it could exceed the boundary of the underlying struct. I think C API should be defensive to such errors.

reachableroots: verify integer range of heads argument (issue4775) Now it raises IndexError instead of SEGV for 'wdir()' as it was before.

reachableroots: unify bail cases to raise exception correctly Before this patch, release_seen_and_tovisit did not return NULL, so the exception was not raised immediately. As Py_XDECREF() and free() are safe for NULL, we can simply bail in any case.

reachableroots: pass NULL to PySet_New() as it expects a pointer, not an int

merge with stable

copy: add flag for disabling copy tracing Copy tracing can be up to 80% of rebase time when rebasing stacks of commits in large repos (hundreds of thousands of files). This provides the option of turning off the majority of copy tracing. It does not turn off _forwardcopies() since that is used to carry copy information inside a commit across a rebase. This will affect the situation where a user edits a file, then rebases on top of commits that have moved that file. The move will not be detected and the user will have to manually resolve the issue (possibly by redoing the rebase with this flag off). The reason to have a flag instead of trying to fix the actual copy tracing performance is that copy tracing is fundamentally an O(number of files in the repo) operation. In order to know if file X in the rebase source was copied anywhere, we have to walk the filelog for every new file that exists in the rebase destination (i.e. a file in the destination that is not in the common ancestor). Without an index that lets us trace forward (i.e. from file Y in the common ancestor forward to the rebase destination), it will never be an O(number of changes in my branch) operation. In mozilla-central, rebasing a 3 commit stack across 20,000 revs goes from 39s to 11s.

strip: use the 'finally: tr.release' pattern during stripping The previous code, was calling 'abort' in all exception cases. This was wrong when an exception was raised by post-close callback on the transaction. Calling 'abort' on an already closed transaction resulted in a error, shadowing the original error. We now use the same pattern as everywhere else. 'tr.release()' will abort the transaction if we escape the scope without closing it. We add a test to make sure we do not regress.

changelog: trust C implementation of reachableroots more There are no remaining codepaths in reachableroots where it will return None, so just trust it completely and simplify this method. Result by revset ================ Revision: 0) Revision 1c75249e159b: style: adjust whitespaces in webutil.py 1) Revision d1d91b8090c6: changelog: trust C implementation of reachableroots more revset #0: 0::tip plain 0) 0.067684 1) 0.006622 9% revset #1: 0::@ plain 0) 0.068249 1) 0.009394 13% IOW this is a 10x speedup in my repo for hg itself for 0::tip and similar revsets now that the C code is correctly wired up.

reachableroots: return NULL if we're throwing an exception Based on my reading of [0] and surrounding sections, if we want an exception to be properly raised when something goes wrong in the C code, we need to make sure we return NULL here. Do so. https://docs.python.org/2/extending/extending.html#back-to-the-example

reachableroots: fix transposition of set and list types in PyArg_ParseTuple This is being masked by the function not properly returning NULL when it raises an exception, so the client code was just falling back to the native codepath when it got None back. A future change removes all reason for this C function to return None, which exposed this problem during development.

reachableroots: consistently use short-form of PyErr_NoMemory()

reachableroots: if allocating a new set fails, use PyErr_NoMemory() My inspection of the implementation of PySet_New() indicates that it does *not* reliably set an exception in the cases where it returns NULL (as far as I can tell it'll never do that!), so let's set that up ourselves.

reachableroots: default to the C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. Before this patch, reachableroots was computed in pure Python by default. This patch makes the C implementation the default and provides a speedup for reachableroots.

changelog: add way to call the reachableroots C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. This patch allows us to call the new C implementation of reachableroots from python by creating an entry point in the changelog class.

reachableroots: add a C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. This patch introduces a C implementation for reachableroots following closely the Python implementation but optimized by using C data structures.

revset: remove grandparent by using reachableroots This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. Before this patch, we had a custom computation for grandparent that was very close to the idea of reacheablerooots. This patch expresses grandparent with reachableroots to reduce the amount of code.

revset: rename revsbetween to reachableroots and add an argument This patch is part of a series of patches to speed up the computation of revset.revsbetween by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of revsbetween is 10-50x faster and smartlog on is 2x-5x faster. This patch rename 'revsbetween' to 'reachableroots' and makes the computation of the full path optional. This will allow graphlog to compute grandparents using 'reachableroots' and remove the need for a dedicated grandparent function.

revset: make revsbetween public This patch is part of a series of patches to speed up the computation of revset.revsbetween by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of revsbetween is 10-50x faster and smartlog on is 2x-5x faster. Later in this serie, we want to reuse the implementation of revsbetween in the changelog module, therefore, we make it public.

match: fix a caseonly rename + explicit path commit on icasefs (issue4768) The problem was that the former name and the new name are both normalized to the case in dirstate, so matcher._files would be ['ABC.txt', 'ABC.txt']. localrepo.commit() calls localrepo.status(), passing along the matcher. Inside dirstate.status(), _walkexplicit() simply grabs matcher.files() and processes those items. Since the old name isn't present, it is silently dropped. There's a fundamental tension here, because the status command should also accept files that don't match the filesystem, so we can't drop the normalization in status. The problem originated in baa11dde8c0e. Unfortunately with this change, the case of the old file must still be specified exactly, or the old file is again silently excluded. I went back to baa11dde8c0e^, and that had the same behavior, so we are no worse off. I'm open to ideas from a matcher or dirstate expert on how to fix that half.

style: adjust whitespaces in webutil.py Turns out, all this came from the single d605a82cf189.

merge with stable

convert: fix git copy file content conversions There was a bug in the git convert code where if you copied a file and modified the copy source in the same commit, and if the copy dest was alphabetically earlier than the copy source, the converted version would use the copy dest contents for both the source and the target. The root of the bug is that the git diff-tree output is formatted like so: :<mode> <mode> <oldhash> <newhash> <state> <src> <dest> :100644 100644 c1ab79a15... 3dfc779ab... C069 oldname newname :100644 100644 c1ab79a15... 03e2188a6... M oldname The old code would always take the 'oldname' field as the name of the file being processed, then it would try to do an extra convert for the newname. This works for renames because it does a delete for the oldname and a create for the newname. For copies though, it ends up associating the copied content (3dfc779ab above) with the oldname. It only happened when the dest was alphabetically before because that meant the copy got processed before the modification. The fix is the treat copy lines as affecting only the newname, and not marking the oldname as processed.

revset: prevent crash caused by empty group expression while optimizing "or" An empty group expression "()" generates None in AST, so it should be tested before destructuring a tuple. "A | ()" is still evaluated to an error because I'm not sure whether "()" represents an empty set or an empty expression (= a unit value). They are identical in "or" operation, but they should be evaluated differently in "and" operation. expression empty set unit value ---------- --------- ---------- () {} A A & () {} A A | () A A

revset: prevent crash caused by empty group expression while optimizing "and" An empty group expression "()" generates None in AST, so the optimizer have to test it before destructuring a tuple. The error message, "missing argument", is somewhat obscure, but it should be better than crash.

win32: use absolute_import

wireproto: use absolute_import

worker: use absolute_import

verify: use absolute_import

url: use absolute_import

ui: use absolute_import And we have no more ambiguous mixed imports \o/

unionrepo: use absolute_import

treediscovery: use absolute_import

transaction: use absolute_import

templater: use absolute_import

templatekw: use absolute_import

templatefilters: use absolute_import

tags: use absolute_import

tagmerge: use absolute_import

subrepo: use absolute_import

strutil: use absolute_import This file doesn't import anything. But chances are we'll eventually implement a static analysis check that ensures absolute_import is used. So we might as well add this.

statichttprepo: use absolute_import

sslutil: use absolute_import

sshserver: use absolute_import

sshpeer: use absolute_import

simplemerge: use absolute_import

setdiscovery: use absolute_import

repoview: use absolute_import

revset: use absolute_import

repair: use absolute_import

pushkey: use absolute_import

progress: use absolute_import

posix: use absolute_import

phases: use absolute_import

peer: use absolute_import

pathutil: use absolute_import

parser: use absolute_import

node: use absolute_import

namespaces: use absolute_import

minirst: use absolute_import

merge: use absolute_import

match: use absolute_import

mail: use absolute_import

lock: use absolute_import

i18n: use absolute_import

httppeer: use absolute_import

hook: use absolute_import