i18n-pt_BR: synchronized with d14590f90cb6

reachableroots: extend "revstates" to array of bit flags

reachableroots: rename "seen" array to "revstates" for future extension It will be an array of bit flags, SEEN | ROOT | REACHABLE.

reachableroots: give anonymous name to short-lived "numheads" variable I'll reuse it for the length of the roots list.

reachableroots: reduce nesting level by jumping to next iteration by continue This can eliminate lines over 80 columns. No code change except for the outermost "if" condition.

test-convert-git: work around output format changes in git git version 2.4.3: --- /home/augie/hg/tests/test-convert-git.t +++ /home/augie/hg/tests/test-convert-git.t.err @@ -659,7 +659,7 @@ $ touch a && git add a && git commit -am "commit a" [master (root-commit) 8ae5f69] commit a Author: nottest <test@example.org> 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 a $ cd .. $ git clone git-repo7 git-repo7-client git version 1.7.9.5: --- /home/augie/hg/tests/test-convert-git.t +++ /home/augie/hg/tests/test-convert-git.t.err @@ -659,7 +659,7 @@ $ touch a && git add a && git commit -am "commit a" [master (root-commit) 8ae5f69] commit a Author: nottest <test@example.org> - 1 file changed, 0 insertions(+), 0 deletions(-) + 0 files changed create mode 100644 a $ cd .. $ git clone git-repo7 git-repo7-client I don't know when this changed in git and am too lazy to try and bisect it, so just work around the change.

histedit: correct spelling etc in more comments Spotted during review of another patch.

histedit: add a missing "s" in a comment

convert: fix convert dropping p2 contents during filemap merge When converting a merge commit using a filemap convert (i.e. when moving contents from the root of the repo into subdir1/), convert would silently drop the entire contents of the target repo's p2. This was because when it built the target commit, it did so by taking the target p1 and adding only the files that changed in the source repo's merge commit. This breaks in the case where the target repo has files that are unrelated to the source repo (like in the case where you use convert to import a repo as a subdirectory of another). The fix is to use Mercurial's merge logic to detect which files in p2 we should carry over to the merge. It follows three rules: 1) if the file belongs to the source, don't try to merge it. Rely on the list of files provided to putcommit to be correct. 2) if the file requires merging or user input (change vs deleted), throw an exception. We don't have enough info to do this. 3) if p2 has the newest, non-merge-requiring version of the file, take it I've also added a test to cover this issue.

convert: implements targetfilebelongstosource for filemap source This is an implementation of the new targetfilebelongstosource() function for the filemapper. It simply checks if the given file name is prefixed by any of the rename destinations. It is not a perfect implementation since it doesn't account for the filemap specifying includes or excludes, but that makes the problem much harder, and this implementation should suffice for most cases.

convert: add function to test if file is from source This adds a base implementation of a function that tests if a given file from a target repo came from the source repo. This will be used later to detect which files did not come from the source repo during a merge, so we can merge those files correctly instead of dropping them.

revsetbenchmarks: run make after update so that C extensions are built

reachableroots: fix memleak of integer objects at includepath loop In the first visit loop, val is decref-ed correctly after PySet_Add(). Let's do the same for the includepath loop.

reachableroots: bail if integer object cannot be allocated This patch also replaces Py_XDECREF() by Py_DECREF() because we known "val" and "p" are not NULL. BTW, we can eliminate some of these allocation and error handling of int objects if the internal "seen" array has more information. For example, enum { SEEN = 1, ROOT = 2, REACHABLE = 4 }; /* ... build ROOT mask from roots argument ... */ if (seen[revnum + 1] & ROOT) { /* instead of PySet_Contains(roots, val) */ >From my quick hack, it is 2x faster.

devel-warn: issue a warning when writing bookmarks without holding the wlock I saw an issue in an extension that we develop where we were writing bookmarks without holding the wlock. Another extension was taking a lock at the same time and wiped out the bookmarks we were about to write. This patch adds a devel-warning to urge people to fix their invalid code.

merge with stable

rebase: lock the repo during the full rebase operation Running `hg pull --rebase` would move bookmarks without any repository locking. So we now lock the repository. For good measure and avoiding sneaky race conditions, we lock the repository for the whole operation. There is no code change besides the indentation.

update: wlock the repo for the whole 'hg update' command The update command is touching the repository and should lock it for the length of its operations. Equally importantly, it should lock the repository when it is writing bookmarks. It wasn't doing so until now, leaving doors open for all kinds of drunk beaver parties. This results in some minor tests changes, and the fixing of a couple of bugs from race conditions. Code does not receive any changes beside extra indentation.

help: fix typo in scripting documentation

hg: avoid auto sharing when the clone destination is remote Before this patch, when auto sharing is enabled, 'hg.clone()' tries to create local clone regardless of locality of the clone destination on the host, and causes failure. To avoid auto sharing when the clone destination is remote, this patch adds examination of 'islocal(dest)' before auto sharing in 'hg.clone()'. 'islocal(dest)' is examined after 'sharepool', because: - the former is more expensive than the latter - without enabling share extension, the later is always negative

largefiles: ensure lfutil.getstandinmatcher() only matches standins Previously, simply having the largefiles extension loaded without any largefiles added would crash when amending with -I. The problem was with no files in the matcher, the pattern list of files joined with 'standindir' was empty, and scmutil.match() would match everything. In lfutil.composestandinmatcher(), the match function is used to test if the file is a standin, and after getting a false positive, proceeds to call lfutil.splitstandin(). This returns None because it isn't a standin, which blows up when passed to rmatcher.matchfn(). Manually overriding _always in getstandinmatcher() probably isn't necessary anymore, but we leave well enough alone on stable. This regressed in ab618e52788a.

dirstate: add --minimal flag to debugrebuilddirstate On repositories with hundreds of thousands of files, hg debugrebuilddirstate causes every dirstate entry to be marked lookup, and the next hg status can take many minutes. This adds a --minimal flag that allows us to only rebuild the parts of the dirstate that are inconsistent. This follows two rules: 1) If a file is in the dirstate but not in the parent manifest, and it is not marked 'add', it is busted and we should drop it. 2) If a file is not in the dirstate at all, but it is in the parent manifest, it should be added to the dirstate and we need to mark it as lookup. This allows us to fix repositories where the dirstate doesn't match the manifest much more quickly. Tested by artificially adding bad dirstate entries (via code) for both cases above.

tests: make filterpyflakes.py read target files relatively to cwd Before this patch, 'filterpyflakes.py' reads target files relatively to own location. But this prevents third party tools from using it in own source tree, because their files are placed separately from 'filterpyflakes.py'. In fact, 'test-check-pyflakes.t', which is the only user of 'filterpyflakes.py', changes current working directory (cwd) to the root of "test target" source tree before using it. Therefore, composing the root of source tree in 'filterpyflakes.py' is redundant. This patch makes 'filterpyflakes.py' read target files relatively to cwd by invoking 'open()' without any path composition. This also removes importing 'os' module, because there is no user of it after this patch. This is a one of preparation of issue4677.

monoblue: remove duplicate font-family property from td.source

monoblue: remove unused elements and related css Since 91b0ada2d94b and c9b88695d894, when monoblue was introduced, the code this patch removes was untouched. Presumably, there supposed to be nice graphics in the screen corners, but there never were due to: - the css being commented out - ids of the elements and of the css selectors being different - and the png files absent The "corner" elements were unstyled and didn't affect the rest of the page, so I think it's safe to remove all this.

revpair: restrict odd-range handling to top-level x:y expression (issue4774) The odd-range hack was introduced by 2a0efa1112ac for backward compatibility, but it was too widely applied. I've checked cmdutil.revpair() at 1.6, and found that ".:", ":0" and ":" are also handled as pairs. So let's enable the hack only for "x:y", "x:", "y:" and ":". test-revset.t is updated because "tip^::tip^ or tip^" shouldn't be taken as an odd range. This patch adds "tip^:tip^" instead. This patch is written for the default branch because parse() of the stable branch lacks compatibility hack for "foo+bar" tag. If we want to mitigate the issue in stable, we can add something like "and '::' in revs[0]".

revpair: update test to make a difference if odd range not handled specially It was added at 2a0efa1112ac, but there was no difference between -r2 and -r2:2 because the working directory was clean.

reachableroots: verify type of each item of heads argument Though PyInt_AS_LONG() can return a value no matter if it isn't an int object, it could exceed the boundary of the underlying struct. I think C API should be defensive to such errors.

reachableroots: verify integer range of heads argument (issue4775) Now it raises IndexError instead of SEGV for 'wdir()' as it was before.

reachableroots: unify bail cases to raise exception correctly Before this patch, release_seen_and_tovisit did not return NULL, so the exception was not raised immediately. As Py_XDECREF() and free() are safe for NULL, we can simply bail in any case.

reachableroots: pass NULL to PySet_New() as it expects a pointer, not an int

merge with stable

copy: add flag for disabling copy tracing Copy tracing can be up to 80% of rebase time when rebasing stacks of commits in large repos (hundreds of thousands of files). This provides the option of turning off the majority of copy tracing. It does not turn off _forwardcopies() since that is used to carry copy information inside a commit across a rebase. This will affect the situation where a user edits a file, then rebases on top of commits that have moved that file. The move will not be detected and the user will have to manually resolve the issue (possibly by redoing the rebase with this flag off). The reason to have a flag instead of trying to fix the actual copy tracing performance is that copy tracing is fundamentally an O(number of files in the repo) operation. In order to know if file X in the rebase source was copied anywhere, we have to walk the filelog for every new file that exists in the rebase destination (i.e. a file in the destination that is not in the common ancestor). Without an index that lets us trace forward (i.e. from file Y in the common ancestor forward to the rebase destination), it will never be an O(number of changes in my branch) operation. In mozilla-central, rebasing a 3 commit stack across 20,000 revs goes from 39s to 11s.

strip: use the 'finally: tr.release' pattern during stripping The previous code, was calling 'abort' in all exception cases. This was wrong when an exception was raised by post-close callback on the transaction. Calling 'abort' on an already closed transaction resulted in a error, shadowing the original error. We now use the same pattern as everywhere else. 'tr.release()' will abort the transaction if we escape the scope without closing it. We add a test to make sure we do not regress.

changelog: trust C implementation of reachableroots more There are no remaining codepaths in reachableroots where it will return None, so just trust it completely and simplify this method. Result by revset ================ Revision: 0) Revision 1c75249e159b: style: adjust whitespaces in webutil.py 1) Revision d1d91b8090c6: changelog: trust C implementation of reachableroots more revset #0: 0::tip plain 0) 0.067684 1) 0.006622 9% revset #1: 0::@ plain 0) 0.068249 1) 0.009394 13% IOW this is a 10x speedup in my repo for hg itself for 0::tip and similar revsets now that the C code is correctly wired up.

reachableroots: return NULL if we're throwing an exception Based on my reading of [0] and surrounding sections, if we want an exception to be properly raised when something goes wrong in the C code, we need to make sure we return NULL here. Do so. https://docs.python.org/2/extending/extending.html#back-to-the-example

reachableroots: fix transposition of set and list types in PyArg_ParseTuple This is being masked by the function not properly returning NULL when it raises an exception, so the client code was just falling back to the native codepath when it got None back. A future change removes all reason for this C function to return None, which exposed this problem during development.

reachableroots: consistently use short-form of PyErr_NoMemory()

reachableroots: if allocating a new set fails, use PyErr_NoMemory() My inspection of the implementation of PySet_New() indicates that it does *not* reliably set an exception in the cases where it returns NULL (as far as I can tell it'll never do that!), so let's set that up ourselves.

reachableroots: default to the C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. Before this patch, reachableroots was computed in pure Python by default. This patch makes the C implementation the default and provides a speedup for reachableroots.

changelog: add way to call the reachableroots C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. This patch allows us to call the new C implementation of reachableroots from python by creating an entry point in the changelog class.

reachableroots: add a C implementation This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. This patch introduces a C implementation for reachableroots following closely the Python implementation but optimized by using C data structures.

revset: remove grandparent by using reachableroots This patch is part of a series of patches to speed up the computation of revset.reachableroots by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of reachableroots is 10-50x faster and smartlog on is 2x-5x faster. Before this patch, we had a custom computation for grandparent that was very close to the idea of reacheablerooots. This patch expresses grandparent with reachableroots to reduce the amount of code.

revset: rename revsbetween to reachableroots and add an argument This patch is part of a series of patches to speed up the computation of revset.revsbetween by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of revsbetween is 10-50x faster and smartlog on is 2x-5x faster. This patch rename 'revsbetween' to 'reachableroots' and makes the computation of the full path optional. This will allow graphlog to compute grandparents using 'reachableroots' and remove the need for a dedicated grandparent function.

revset: make revsbetween public This patch is part of a series of patches to speed up the computation of revset.revsbetween by introducing a C implementation. The main motivation is to speed up smartlog on big repositories. At the end of the series, on our big repositories the computation of revsbetween is 10-50x faster and smartlog on is 2x-5x faster. Later in this serie, we want to reuse the implementation of revsbetween in the changelog module, therefore, we make it public.

match: fix a caseonly rename + explicit path commit on icasefs (issue4768) The problem was that the former name and the new name are both normalized to the case in dirstate, so matcher._files would be ['ABC.txt', 'ABC.txt']. localrepo.commit() calls localrepo.status(), passing along the matcher. Inside dirstate.status(), _walkexplicit() simply grabs matcher.files() and processes those items. Since the old name isn't present, it is silently dropped. There's a fundamental tension here, because the status command should also accept files that don't match the filesystem, so we can't drop the normalization in status. The problem originated in baa11dde8c0e. Unfortunately with this change, the case of the old file must still be specified exactly, or the old file is again silently excluded. I went back to baa11dde8c0e^, and that had the same behavior, so we are no worse off. I'm open to ideas from a matcher or dirstate expert on how to fix that half.

style: adjust whitespaces in webutil.py Turns out, all this came from the single d605a82cf189.

merge with stable

convert: fix git copy file content conversions There was a bug in the git convert code where if you copied a file and modified the copy source in the same commit, and if the copy dest was alphabetically earlier than the copy source, the converted version would use the copy dest contents for both the source and the target. The root of the bug is that the git diff-tree output is formatted like so: :<mode> <mode> <oldhash> <newhash> <state> <src> <dest> :100644 100644 c1ab79a15... 3dfc779ab... C069 oldname newname :100644 100644 c1ab79a15... 03e2188a6... M oldname The old code would always take the 'oldname' field as the name of the file being processed, then it would try to do an extra convert for the newname. This works for renames because it does a delete for the oldname and a create for the newname. For copies though, it ends up associating the copied content (3dfc779ab above) with the oldname. It only happened when the dest was alphabetically before because that meant the copy got processed before the modification. The fix is the treat copy lines as affecting only the newname, and not marking the oldname as processed.

revset: prevent crash caused by empty group expression while optimizing "or" An empty group expression "()" generates None in AST, so it should be tested before destructuring a tuple. "A | ()" is still evaluated to an error because I'm not sure whether "()" represents an empty set or an empty expression (= a unit value). They are identical in "or" operation, but they should be evaluated differently in "and" operation. expression empty set unit value ---------- --------- ---------- () {} A A & () {} A A | () A A

revset: prevent crash caused by empty group expression while optimizing "and" An empty group expression "()" generates None in AST, so the optimizer have to test it before destructuring a tuple. The error message, "missing argument", is somewhat obscure, but it should be better than crash.

win32: use absolute_import

wireproto: use absolute_import

worker: use absolute_import

verify: use absolute_import

url: use absolute_import

ui: use absolute_import And we have no more ambiguous mixed imports \o/

unionrepo: use absolute_import

treediscovery: use absolute_import

transaction: use absolute_import

templater: use absolute_import

templatekw: use absolute_import

templatefilters: use absolute_import

tags: use absolute_import

tagmerge: use absolute_import

subrepo: use absolute_import

strutil: use absolute_import This file doesn't import anything. But chances are we'll eventually implement a static analysis check that ensures absolute_import is used. So we might as well add this.

statichttprepo: use absolute_import

sslutil: use absolute_import

sshserver: use absolute_import

sshpeer: use absolute_import

simplemerge: use absolute_import

setdiscovery: use absolute_import

repoview: use absolute_import

revset: use absolute_import

repair: use absolute_import

pushkey: use absolute_import

progress: use absolute_import

posix: use absolute_import

phases: use absolute_import

peer: use absolute_import

pathutil: use absolute_import

parser: use absolute_import

node: use absolute_import

namespaces: use absolute_import

minirst: use absolute_import

merge: use absolute_import

match: use absolute_import

mail: use absolute_import

lock: use absolute_import

i18n: use absolute_import

httppeer: use absolute_import

hook: use absolute_import

hbisect: use absolute_import

graphmod: use absolute_import

formatter: use absolute_import

filemerge: use absolute_import

filelog: use absolute_import

fancyopts: use absolute_import

extensions: use absolute_import

error: use absolute_import

discovery: use absolute_import

demandimport: use absolute_import

dagutil: use absolute_import

dagparser: use absolute_import

crecord: use absolute_import

hg: use absolute_import

fileset: use absolute_import

demandimport: support lazy loading for absolute_import Before, we didn't support lazy loading if absolute_import was in effect and a fromlist was used. This meant that "from . import X" wasn't lazy and performance could suffer as a result. With this patch, we now support lazy loading for this scenario. As part of developing this, I discovered issues when module names are defined. Since the enforced import style only allows "from X import Y" or "from .X import Y" in very few scenarios when absolute_import is enabled - scenarios where Y is not a module and thus there is nothing to lazy load - I decided to drop support for this case instead of chasing down the errors. I don't think much harm will come from this. But I'd like to take another look once all modules are using absolute_import and I can see the full extent of what is using names in absolute_import mode.

demandimport: support keyword arguments on _hgextimport __import__ supports keyword arguments since Python 2.5. This proxy should too. An upcoming patch will use this feature.

demandimport: refactor logic and add documentation demandimport doesn't currently support absolute imports (level >= 0). In preparation for this, we add some documentation and a code branch to handle the absolute_import case.

demandimport: add __future__ to ignore list __future__ is special. We should definitely not be trying to lazy load it.

demandimport: remove support for Python < 2.5 The removed code was to support an __import__ function that doesn't support the "level" argument. This argument was added in Python 2.5, which we no longer support.

dispatch: use absolute_import A mixed, ambiguous import has been removed!

commands: break import cycle This was the easiest link in the chain to break as there was only one use of commandserver in commands.py. Other files had multiple users.

cmdutil: break import cycle This was the easiest place to break the chain, as there were only 2 uses of the imported module in the file.

revset: make balanced addsets by orset() without using _combinesets() As scmutil.revrange() was rewritten to not use _combinesets(), we no longer need _combinesets().

revrange: evaluate all revset specs at once This provides an opportunity for revset to optimize given expressions. For example, "-r0 -r1 -r2" can be optimized to "_list(0 1 2)".

revset: add matchany() to construct OR expression from a list of specs This will allow us to optimize "-rREV1 -rREV2 ..." command-line options.

revset: split post-parsing stage from match() _makematcher() will be reused by new matchany(ui, specs, repo=None) function I'll add by the next patch.

filesets: ignore unit case in size() predicate for single value When specifying one plain value in size(), e.g. size(1k), fileset tries to guess the upper bound automatically (see the comment in _sizetomax()). It didn't ignore the specified unit's case, and so size("1 GB"), for example, produced this error: hg: parse error: couldn't parse size: 1 GB Let's do the same thing that util.sizetoint() does: .lower(). The two test lines without output just check that there are no parse errors.

copies: use absolute_import

config: use absolute_import

changelog: use absolute_import

changegroup: use absolute_import

bundlerepo: use absolute_import

bundle2: use absolute_import

branchmap: use absolute_import

bookmarks: use absolute_import

archival: use absolute_import

ancestor: use absolute_import A few months ago, import-checker.py was taught to enforce a more well-defined import style for files with absolute_import. However, we stopped short of actually converting source files to use absolute_import because of problems with certain files. Investigation revealed the following problems with switching to absolute_import universally: 1) import cycles result in import failure on Python 2.6 2) undetermined way to import C/pure modules While these problems need to be solved, they can be put off. This patch starts a series of converting files to absolute_import that won't exhibit any of the aforementioned problems.

discovery: always use batching now that all peers support batching Some peers will transparently downgrade batched requests to non-batched ones, but that simplifies code for everyone using batching.

wireproto: make wirepeer look-before-you-leap on batching This means that users of request batching don't need to worry themselves with capability checking. Instead, they can just use batching, and if the remote server doesn't support batching for some reason the wirepeer code will transparently un-batch the requests. This will allow for some slight simplification in a handful of places. Prior to this change, largefiles would have been silently broken against a server which did not support batching.

batching: migrate basic noop batching into peer.peer "Real" batching only makes sense for wirepeers, but it greatly simplifies the clients of peer instances if they can be ignorant to actual batching capabilities of that peer. By moving the not-really-batched batching code into peer.peer, all peer instances now work with the batching API, thus simplifying users. This leaves a couple of name forwards in wirepeer.py. Originally I had planned to clean those up, but it kind of unclarifies other bits of code that want to use batching, so I think it makes sense for the names to stay exposed by wireproto. Specifically, almost nothing is currently aware of peer (see largefiles.proto for an example), so making them be aware of the peer module *and* the wireproto module seems like some abstraction leakage. I *think* the right long-term fix would actually be to make wireproto an implementation detail that clients wouldn't need to know about, but I don't really know what that would entail at the moment. As far as I'm aware, no clients of batching in third-party extensions will need updating, which is nice icing.

parsers: fix memory leak in compute_phases_map_sets PySet_Add increments the reference of the added object to the set, see: https://hg.python.org/cpython/file/2.6/Objects/setobject.c#l379 Before this patch we were forgetting to decrement the reference count after adding objects to the phaseset. This patch fixes the issue and makes the reference count right so that these objects can be properly garbage collected.

histedit: do not stay on a cleaned nodes on abort There is case where nodes are neither in tmpnodes nor leaf but still get removed. For example, if you used the "edit" action, made a commit and run --abort. The commit you made is not tracked by histedit, yet it will likely be cleaned up with its parent. The commit may not tracked because no replacements computations are done in the --abort case.

histedit: also update away from tmpnodes The working copy may be on a tmpnodes, we need to update away before it is stripped from the repository.

histedit: use revset to check if we need to update during abort The for loop is already quite more complicated than necessary and we are about to add some logic. Instead, we use a simple revset. Revset laziness should provide us with similar performance.

histedit: remove useless 'else' clause This 'else: pass' clause have no effect. We drop it for clarity.

histedit: make cleanupnode more robust The goal of this function is to strip content out of the repository. We do not really care if this content is visible or cleanup node not and we should proceed anyway. None of the internal actions are subject to this, however, a third party extension running arbitrary commands during histedit is affected by this.

convert: fix git convert using servers branches The conversion from git to hg was reading the remote branch list directly from the origin server. If the origin's branch had moved forward since the last git fetch, it would return a git hash which didn't exist locally, and therefore the branch was not converted. This changes it to rely on the local repo's refs/remotes list of branches instead, so it's completely cut off from the server.

revrange: drop old-style parser in favor of revset (API) Now revset can parse nullary ":" operator and existing "foo+bar" tags, we don't need the old-style parser. This means scmutil.revsingle(), revpair() and revrange() no longer accept a binary nodeid. An integer revision is still allowed as it isn't ambiguous.

tag: do not pass binary nullid to scmutil.revsingle() Future patches will remove the old-style parser that happen to accept a binary nodeid. A binary nodeid shouldn't be passed to scmutil.revrange() because it is ambiguous. For example, bin('20' * 19 + '30') is valid binary nodeid, but it can also be parsed as a revset expression, '0'.

revset: port parsing rule of old-style ranges from scmutil.revrange() The old-style parser will be removed soon.

debugrevspec: pass lookup function to visualize fallback mechanism The next patch will move the exceptional parsing of old-style ranges to revset.tokenize(). This patch will allow us to see the result tree. Note that the parsing result of '-a-b-c-' is incorrect at this changeset. It will be fixed soon.

help: fix typo familar -> familiar

highlight: exit early on textual and unknown files (issue3005) When highlight extension encountered files that pygments didn't recognize, it used to fall back to text lexer. Also, pygments uses TextLexer for .txt files. This lexer is noop by design. On bigger files, however, doing the noop highlighting resulted in noticeable extra CPU work and memory usage: to show a 1 MB text file, hgweb required about 0.7s more (on top of ~3.8s, Q8400) and consumed about 100 MB of RAM more (on top of ~150 MB). Let's just exit the function when it's clear that nothing will be highlighted. Due to how this pygmentize function works (it modifies the template in-place), we can just return from it and everything else will work as if highlight extension wasn't enabled.

histedit: extract a simpler function to process replacement on abort The process replacement is building a full mapping to allow moving bookmarks and creating obsolescence marker. We do not need the full logic for abort so we extract it. It will be useful as abort is missing some data about the replacement and can crash when third party extensions push it a bit too far.

merge with stable

exchange: fix dead assignment The assignment of the value from bundle2.processbundle() to 'r' is unused. It is currently the same as its third argument (if given), and since that argument may eventually go away (according to the method's docstring), let's reassign the return value to 'op' instead to better prepare for that.

exchange: s/phase/bookmark/ in _pushb2bookmarks()

histedit: backout ebb5bb9bc32e The faulty changeset use obsolescence marker to roll the repository back on --abort. This is a problematic approach because --abort should be as close as an actually transaction rollback as possible stripping all created data from the repository (cf `hg rebase --abort` stripping all created changesets). Instead ebb5bb9bc32e made all content created during the aborted histedit still available in the repository adding obsolescence marker to make them hidden. This will cause trouble to evolution user as a re-run of the same histedit (with success) will likely result in the very same node to be "recreated" while obsolescence marker would be in place for them. And canceling an obsoletion is still a fairly complicated process. This also rollback using obsmarkers instead of strip to clean up temporary node on successful histedit run because the two change were not split in separated changeset. Rolling that part back does not have significant consequence a will have to be resubmitted independently

test-bookmarks.t: avoid nested repo This is (a) pretty unnecessary and (b) breaks tests for the third-party hgwatchman extension, which doesn't support nested repos.

revlog: remove unused shaoffset constants Call sites were removed at 61c9bc3da402, "revlog: remove lazy index".

revlog: correct comment about size of v0 index format

merge with stable

Added signature for changeset 21aa1c313b05

Added tag 3.5 for changeset 21aa1c313b05

i18n: fix unclosed inline span in pt_BR This was causing test-gendoc.t to complain: WARNING: Inline interpreted text or phrase reference start-string without end-string.

merge with i18n

i18n-ja: synchronized with 7fcad0c4ef8c

convert: when converting from Perforce use original local encoding by default On Windows Perforce command line client uses default system locale to encode output. Using 'latin_1' causes locale-specific characters to be replaced with question marks. With this patch we will use default locale by default whilst allowing to specify it explicity with 'convert.p4.encoding' config option. This is a potentially breaking change for any scripts relying on output treated as in 'latin_1' encoding. Also because hgext.convert.convcmd overwrites detected default system locale with UTF-8 we had to introduce an import cycle in hgext.convert.p4 to retrieve originally detected encoding from hgext.convert.convcmd.

i18n-pt_BR: synchronized with 3e84f40232c7

convert: when getting file from Perforce concatenate data at the end As it turned out, even when getting relatively small files, concatenating string data every time when new chunk is received is very inefficient. Maintaining a string list of data chunks and concatenating everything in one go at the end seems much more efficient - in my testing it made getting 40 MB file 7 times faster, whilst converting of a particularly big changelist with some big files went down from 20 hours to 3 hours.

help: scripting help topic There are a lot of non-human consumers of Mercurial. And the challenges and considerations for machines consuming Mercurial is significantly different from what humans face. I think there are enough special considerations around how machines consume Mercurial that a dedicated help topic is warranted. I concede the audience for this topic is probably small compared to the general audience. However, lots of normal Mercurial users do things like create one-off shell scripts for common workflows that I think this is useful enough to be in the install (as opposed to, say, a wiki page - which most users will likely never find). This text is by no means perfect. But you have to start somewhere. I think I did cover the important parts, though.

convert: document convert.hg.startrev

transplant: restore dirstate correctly at unexpected failure Before this patch, transplant can't restore dirstate as expected at failure other than one while patching. This causes: - unexpected file status - dirstate refers already rollback-ed parent (only at failure of transplanting the 2nd or later revision) To restore dirstate correctly also at unexpected failure, this patch encloses scope of store lock and transaction by 'dirstateguard'. This is temporary fixing for stable branch. See DirstateTransactionPlan wiki page for detail about the future plan to treat dirstate consistently around scope boundary of transaction. https://mercurial.selenic.com/wiki/DirstateTransactionPlan This patch also adds 'if lock' examination for safety 'lock.release()', because creating 'dirstateguard' object may fail unexpectedly (e.g. IOError for saving dirstate). BTW, in the test script, putting section header '[extensions]' into '.hg/hgrc' is needed to fix incomplete disabling 'abort' extension at 4d1382fd96ff.

localrepo: make journal.dirstate contain in-memory changes before transaction Before this patch, in-memory dirstate changes aren't written out at opening transaction, even though 'journal.dirstate' is created directly from '.hg/dirstate'. Therefore, subsequent 'hg rollback' uses incomplete 'undo.dirstate' to restore dirstate, if dirstate is changed and isn't written out before opening transaction. In cases below, the condition "dirstate is changed and isn't written out before opening transaction" isn't satisfied and this problem doesn't appear: - "wlock scope" and "transaction scope" are almost equivalent e.g. 'commit --amend', 'import' and so on - dirstate changes are written out before opening transaction e.g. 'rebase' (via 'dirstateguard') and 'commit -A' (by separated wlock scopes) On the other hand, 'backout' may satisfy the condition above. To make 'journal.dirstate' contain in-memory changes before opening transaction, this patch explicitly invokes 'dirstate.write()' in 'localrepository.transaction()'. 'dirstate.write()' is placed before not "writing journal files out" but "invoking pretxnopen hooks" for visibility of dirstate changes to external hook processes. BTW, in the test script, 'touch -t 200001010000' and 'hg status' are invoked to make file 'c' surely clean in dirstate, because "clean but unsure" files indirectly cause 'dirstate.write()' at 'repo.status()' in 'repo.commit()' (see fe03f522dda9 for detail) and prevents from certainly reproducing the issue.

dirstate: ensure mv source is marked deleted when walking icasefs (issue4760) Previously, importing a case-only rename patch on a case insensitive filesystem caused the original file to be marked as '!' in status. The source was being forgotten properly in patch.workingbackend.close(), but the call it makes to scmutil.marktouched() then put the file back into the 'n' state (but it was still missing from the filesystem). The cause of this was scmutil._interestingfiles() would walk dirstate, and since dirstate was able to lstat() the old file via the new name, was treating this as a forgotten file, not a removed file. scmutil.marktouched() re-adds forgotten files, so dirstate got out of sync with the filesystem. This could be handled with less code in the "kind == regkind or kind == lnkkind" branch of dirstate._walkexplicit(), but this avoids filesystem accesses unless case collisions occur. _discoverpath() is used instead of normalize(), since the dirstate case is given first precedence, and the old file is still in it. What matters is the actual case in the filesystem.

extdiff: allow modifications in subrepos to be copied back This check was a legacy bit from when the file data was being fetched manually with 'ctx[wfn]', but archive() does that now. 49966b5ab16f seems to indicate that this avoided a problem where a merge adds a file to another branch, and that test still passes. Unfortunately, I don't see a way to create a test that modifies the file in the temporary directory before the command exits. I wonder if the os.lstat() call needs to be wrapped in an exception handler for the case where archive didn't create a file because the file didn't exist in that revision. But I wasn't able to trigger a problem without it on a real repository.

ignore: fix path concatenation of .hgignore on Windows Since 3de48ff62733, .hgignore is ignored on Windows because a pat may have a drive letter, but pathutil.join is posixpath.join. "z:\foo\bar/z:\foo\bar\.hgignore" Instead, this patch uses os.path.join() and util.localpath() to process both parts as file-system paths. Maybe we can remove os.path.join() at dirstate._ignore because 'include:' is resolved relative to repo root? It was introduced by a04c7b74b3d5.

repair: fix typo in warning message

monoblue: link to branches on summary page "Tags" and "Bookmarks" sections on this page already have the similar links, and so does the "Branches" on summary page in gitweb, so let's do the same in monoblue.

monoblue: fix branches link on help topics page

hgweb: show ages in repos' Last modified column in monoblue and gitweb Index page, which shows the list of available repositories, has a column where the last modification date for each repo is shown. paper, coal and spartan already show the dates in relative format (e.g. "2 weeks ago"), because these styles have the required process_dates() js function call in their footer templates, which are included on every page. But monoblue and gitweb styles have more things in the footer templates, such as repo name and its atom/rss links, so they don't include the footer on index page (as this page doesn't have a single repo context). Let's call process_dates() without including the footer.

ignore: fix include: rules depending on current directory (issue4759) When reading pattern files, we just call open(path), which is relative to the current directory. Let's fix this by resolving the paths before attempting to read the file.

test: move ignore test run into a subdirectory Previously the hgignore test just called hg init in the test directory. A future patch needs to test hgignore stuff from outside of the repo, so let's move the entire test repo into a subdirectory.

monoblue: show relative age of bookmarks In b24e5a708fad, age calculation was made dynamic (i.e. in javascript), but for some reason bookmarkentry template in monoblue/map got a wrong class. It resulted in /summary and /bookmarks pages always showing exact dates for bookmarks, no age calculation was performed. Let's fix this by using "age" class that is already used in branchentry and tagentry templates in the same map file. As usual, the exact date for such elements is still available in title attribute, so it shows in a tooltip on hover.

highlight: produce correct markup when there's a blank line just before EOF Due to how the colorized output from pygments was stripped of <pre> elements, when there was an empty line at the end of a file, highlight extension produced an incorrect markup (no closing tags from the fileline/annotateline template). It wasn't usually noticeable, because browsers were smart enough to see where the missing tags should've been, but in monoblue style it resulted in the last line having twice the normal height. Instead of awkwardly trying to strip outer <pre></pre> tags, let's make the formatter with nowrap=True, which should do what we need in pygments since at least 0.5 (2006-10-30). Example from monoblue style: Before: <div class="source"> <div style="font-family:monospace" class="parity0"> <pre><a class="linenr" href="#l1" id="l1"> 1</a> </pre> </div> <div style="font-family:monospace" class="parity1"> <pre><a class="linenr" href="#l2" id="l2"> 2</a> </div> Now: <div class="source"> <div style="font-family:monospace" class="parity0"> <pre><a class="linenr" href="#l1" id="l1"> 1</a> </pre> </div> <div style="font-family:monospace" class="parity1"> <pre><a class="linenr" href="#l2" id="l2"> 2</a> </pre> </div> </div> (Notice the missing </pre></div> now in place)

test-check-config: convert directory separators to '/' for MSYS The output of the files command uses native separator. MSYS then seems to drop the '\' on Windows when invoking python: --- c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t +++ c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t.err @@ -6,22 +6,10 @@ $ hg files "set:(**.py or **.txt) - tests/**" | > xargs python contrib/check-config.py Traceback (most recent call last): File "contrib/check-config.py", line 93, in <module> sys.exit(main(sys.argv[1:])) File "contrib/check-config.py", line 24, in main for l in open(f): IOError: [Errno 2] No such file or directory: 'contriball-revsets.txt' [123]

test-subrepo: fix globs for Windows It looks like these changes originated with 2538b87660be. I'm not sure that it was intentional, but is seems harmless enough for an error message.

test-clone: fix globs for Windows

tests: diff -r on Solaris emits "Common subdirectories:" When using the -r option to Solaris diff, any directores that compare identically are mentioned in the output. We don't really care about these directories for the purposes of this test, so ignore them.

tests: work around differing hunk headers between GNU and Solaris diff The hunk headers specifying what lines the hunk apply to differ. They're irrelevant to the test, so just eliminate them from the output.

tests: use egrep with regular expression branches for compatibility GNU grep allows you to use "a\|b" in a regular expression to match either "a" or "b", but at least Solaris grep does not; only egrep allows for that. And egrep considers "a+" to be "a{1,}" instead of an "a" and a literal plus sign, so escape that as well.

parsers: silence warning of implicit integer conversion issued by clang "-Wshorten-64-to-32" is enabled by default on Mac OS X. Because "len" should be represented in 32bit integer, this patch simply cast ssize_t to int.

tests: disable test of buffer overflow in parsers.c if --pure It fails with AttributeError and there's no benefit to make it runnable with pure Python code.

win32: update Inno Setup script after the changes done in 95e042d77a5f

Added signature for changeset 96a38d44ba09

Added tag 3.5-rc for changeset 96a38d44ba09

merge with default for code freeze

tests: add a check-config pass

hgweb: document web.certificates option

shelve: make maxbackup doc check-config friendly

share: make option docs more check-config friendly

perf: mark experimental option presleep

check-config: don't continue prematurely The early check for section headers like ^``foo`` was missing actual options like ^``foo.bar``.

subrepo: mark internal-only option

progress: mark experimental option

censor: mark experimental option

bundle2: fix type of experimental option

merge: make merge.preferancestor type and default consistent (and mark it)

merge: mark ancient debugging option

email: fix config default value inconsistency

bookmarks: mark internal-only option

commit: mark internal-only option

localrepo: mark format options

formatter: mark developer options

http2: mark experimental and developer options

bookmarks: mark internal-only config option

filemerge: mark internal-only config option

profiler: mark developer-only config option

debugger: mark developer-only option (and rearrange comment)

commandserver: mark developer-only logging option

generaldelta: mark experimental reordering option

bundlerepo: mark internal-only config variable

win32text: mark deprecated extension option deprecated

transplant: mark some undocumented options deprecated

mq: tweak config reading to make check-config happy The not-really-a-bool handling here upsets the type checker.

patchbomb: make sure all users of smtp.verifycert agree on the default

patchbomb: mark ancient option deprecated This just exists for backwards compatibility with the earliest versions of patchbomb.

histedit: mark defaultrev option experimental

changegroup: compute seen files as changesets are added (issue4750) Before this patch, addchangegroup() would walk the changelog and compute the set of seen files between applying changesets and applying manifests. When cloning large repositories such as mozilla-central, this consumed a non-trivial amount of time. On my MBP, this walk takes ~10s. On a dainty EC2 instance, this was measured to take ~125s! On the latter machine, this delay was enough for the Mercurial server to disconnect the client, thinking it had timed out, thus causing a clone to abort. This patch enables the changelog to compute the set of changed files as new revisions are added. By doing so, we: * avoid a potentially heavy computation between changelog and manifest processing by spreading the computation across all changelog additions * avoid extra reads from the changelog by operating on the data as it is added The downside of this is that the add revision callback does result in extra I/O. Before, we would perform a flush (and subsequent read to construct the full revision) when new delta chains were created. For changelogs, this is typically every 2-4 revisions. Using the callback guarantees there will be a flush after every added revision *and* an open + read of the changelog to obtain the full revision in order to read the added files. So, this increases the frequency of these operations by the average chain length. In the future, the revlog should be smart enough to know how to read revisions that haven't been flushed yet, thus eliminating this extra I/O. On my MBP, the total CPU times for an `hg unbundle` with a local mozilla-central gzip bundle containing 251,934 changesets and 211,065 files did not have a statistically significant change with this patch, holding steady around 360s. So, the increased revlog flushing did not have an effect. With this patch, there is no longer a visible pause between applying changeset and manifest data. Before, it sure felt like Mercurial was lethargic making this transition. Now, the transition is nearly instantaneous, giving the impression that Mercurial is faster. Of course, eliminating this pause means that the potential for network disconnect due to channel inactivity during the changelog walk is eliminated as well. And that is the impetus behind this change.

revlog: add support for a callback whenever revisions are added A subsequent patch will add a feature that performs iterative computation as changesets are added from a changegroup. To facilitate this type of processing in a generic manner, we add a mechanism for calling a function whenever a revision is added via revlog.addgroup(). There are potential performance concerns with this callback, as using it will flush the revlog after every revision is added.

crecord: throws error instead of crashing for large diffs Before this patch, crecord was crashing for large diffs (30k lines on my laptop). This patch catches the exception raised in that case and use the error reporting mechanism introduced in the previous patch for notifying the user of the issue. It is not possible to add a test for that for now as we don't yet have full blown ui tests for the curses interface.

crecord: add error reporting for failure in curses interface initialization Before this patch, we couldn't report to the user any error that occurred: - after we enabled the curses interface but - before the interface is set up and drawn This patch, provides a way to set errors that happens during the initialization of the interface and log them once the curses interface has been displayed.

revset: parse nullary ":" operator as "0:tip" This is necessary for compatibility with the old-style parser that will be removed by future patches.

parser: take suffix action if no infix action is defined If no infix action is defined, a suffix action isn't ambiguous, so it should be taken no matter if the next token can be an operand. This is exactly the same flow as prefix/primary handling. This change has no effect now because all suffix tokens have infix actions.

parser: reorder infix/suffix handling to be similar to prefix/primary flow It can be exactly the same flow as the prefix/primary handling. A suffix action is accepted only if the next token never starts new term.

parser: resolve ambiguity where both prefix and primary actions are defined If both actions are defined, a primary-expression action is accepted only if the next token never starts new term. For example, parsed as primary expression: ":" # next token 'end' has no action "(:)" # next token ')' has no action ":+y" # next token '+' is infix operator parsed as prefix operator: ":y" # next token 'y' is primary expression ":-y" # next token '-' is prefix operator This is mostly the same resolution as the infix/suffix rules.

parser: separate actions for primary expression and prefix operator This will allow us to define both a primary expression, ":", and a prefix operator, ":y". The ambiguity will be resolved by the next patch. Prefix actions in elements table are adjusted as follows: original prefix primary prefix ----------------- -------- ----------------- ("group", 1, ")") -> n/a ("group", 1, ")") ("negate", 19) -> n/a ("negate", 19) ("symbol",) -> "symbol" n/a

changelog: update read pending documentation The pending index contains a full copy of the index + in-transaction data. We replace "extend" with "overwrite" to make this clearer.

extdiff: add support for subrepos Git and svn subrepo support is incomplete, because they don't support archiving the working copy.

extdiff: use archiver to take snapshots of committed revisions This is the last step before supporting extdiff -S. It maintains the existing behavior of diffing the largefile standins instead of the largefiles themselves. Note however that the standins are not updated immediately upon modification, so uncommitted largefile changes are ignored, as they previously were, even with the diff command.

largefiles: allow the archiving of largefiles to be disabled There are currently no users of this, but it is a necessary step before converting extdiff to use archive. It may be useful to add an argument to extdiff in the future and allow largefiles to be diffed, but archiving largefiles can have significant overhead and may not be very diffable, so archiving them by default seems wrong. It is a mystery to me why the lfstatus attribute needs to be set on the unfiltered repo. However if it is set on the filtered repo instead (and the filtered repo is passed to the original command), the lfstatus attribute is False in the overrides for archival.archive() and hgsubrepo.archive() when invoking the archive command. This smells like the buggy status behavior (see 67d63ec85eb7, which was reverted in df463ca0adef). Neither the status nor summary commands have this weird behavior in their respective overrides.

parsers: fix buffer overflow by invalid parent revision read from revlog If revlog file is corrupted, it can have parent pointing to invalid revision. So we should validate it before updating nothead[], phases[], seen[], etc. Otherwise it would cause segfault at best. We could use "rev" instead of "maxrev" as upper bound, but I think the explicit "maxrev" can clarify that we just want to avoid possible buffer overflow vulnerability.

histedit: mark temporary commits as obsolete when allowed to Before this patch, we were stripping temporary commits at the end of a histedit whether it was successful or not. If we can create obs markers, we should create them instead of stripping because it is faster and safer.

histedit: minor refactoring of createmarkers check We use a variable to store whether or not we can create obsolescence markers. It makes the patch series more readable as we are going to reuse this values in other places in the function.

crecord: fix issue when backgrounding editor would leave artefact Before this patch: - if a user was entering a commit message after having ran the curses interface - and then uses ctrl-z, followed by fg to put the editor in the background/foreground - then the curses interface would leave artefact on the screen of the editor, making entering the commit message a difficult task This happened because ncurses registers a signal handler for SIGTSTP and does not restore the original signal handler after running. More info at: http://stackoverflow.com/questions/31440392/ curses-wrapper-messing-up-terminal-after-background-foreground-sequence/ 31441709#31441709 This patch restores the original value of the signal handler after running the curses interface and therefore fixes this issue. It don't know how to add a test for this issue, I tested the scenario above manually and it works correctly with the patch.

censor: make various path forms available like other Mercurial commands Before this patch, censored file should be exactly "a path relative to repository root" regardless of current working directory, because "hg censor" passes "path" to "repo.file()" directly without any preparations. To make various path forms available like other Mercurial commands, this patch gets a target file path in the way of "hg parents FILE". Getting "wctx" is relocated to reuse "wctx" for efficiency.