merge with stable

clonebundles: fix typo s/comand/command/

posix: retry on symlink race in checklink Multiple threads might attempt to check links with the same temporary name. This would cause one side to get an EEXIST error and wrongly fail the support check. Here, we simply retry if our temporary name exists.

changegroup: back code change of e7c618cee8df out The previous changeset is a simpler way of fixing issue4934 without changing the spirit of the code. We can remove the dual call to 'delayupdate' but we keep the tests to show that the issue is still fixed.

changegroup: call 'prechangegroup' hook before setting up write delay The 'prechangegroup' interfere with 'delayupdate' logic because it trigger the one time call of 'changelog._writepending' (see issure4934). There is no reason not to call that hook before setting up 'delayupdate' so we move the call a bit earlier to avoid interference.

changegroup: fix the scope of a try finally The try finally is here to ensure we release the just-created transaction. Therefore we should not do half a dozen operations before actually entry the try scope.

encoding: re-escape U+DCxx characters in toutf8b input (issue4927) This is the final missing piece in fully round-tripping random byte strings through UTF-8b. While this issue means that UTF-8 <-> UTF-8b isn't fully bijective, we don't expect to ever see U+DCxx codepoints in "real" UTF-8 data, so it should remain bijective in practice.

encoding: use getutf8char in toutf8b This correctly avoids the ambiguity of U+FFFD already present in the input and similar confusion by working a character at a time.

encoding: handle non-BMP characters in fromutf8b

posix: use getutf8char to handle OS X filename percent-escaping This replaces an open-coded utf-8 parser that was ignoring subtle issues like overlong encodings.

encoding: add getutf8char helper This allows us to find character boundaries in byte strings when trying to do custom encodings.

unbundle: gratuitous fix white spacing "issue" We were missing one space on this block.

demandimport: fix level passed to loader of sub-modules As the fromlist gives the names of sub-modules, they should be searched in the parent directory of the package's __init__.py, which is level=1. I got the following error by rewriting hgweb to use absolute_import, where the "mercurial" package is referenced as ".." (level=2): ValueError: Attempted relative import beyond toplevel package I know little about the import mechanism, but this change seems correct. Before this patch, the following code did import the os module with no error: from mercurial import demandimport demandimport.enable() from mercurial import os print os.name

parsers: fix width of datalen variable in fm1readmarkers Because parsers.c does not define PY_SSIZE_T_CLEAN, "s#" format requires (const char*, int), not (const char*, Py_ssize_t). https://docs.python.org/2/c-api/arg.html This error had no problem before 042344313939, where datalen wasn't used. But now fm1readmarkers() fails with "overflow in obsstore" on Python 2.6.9 (amd64) because upper bits of datalen seem to be filled with 1, making it a negative integer. This problem seems not visible on our Python 2.7 environment because upper bits happen to be filled with 0.

manifest: skip fastdelta if the change is large In large repos, the existing manifest fastdelta computation (which performs a bisect on the raw manifest for every file that is changing), is excessively slow. This patch makes fastdelta fallback to the normal string delta algorithm if the number of changes is large. On a large repo with a commit of 8000 files, this reduces the commit time by 7 seconds (fastdelta goes from 8 seconds to 1). I tested this change by modifying the function to compare the old and the new values and running the test suite. The only difference is that the pure text-diff algorithm sometimes produces smaller (but functionaly identical) deltatexts than the bisect algorithm.

merge.mergestate: update docstrings for preresolve and resolve Add a docstring for preresolve, and update the one for resolve.

rebase: pass date parameter to concludenode

rebase: add date parameter to concludenode function This parameter will allow us to set the date of rebased commits

hgweb: remove unused colorPart() from mercurial.js Looks like the function wasn't ever used since its introduction in 0dba955c2636, since setColor() below always used "rgb(255, 255, 255)" notation which doesn't need hex digits.

test: enforce non-general delta in 'test-generaldelta.t' If general delta becomes the default, we need to be explicit about what we want to be tested.

test: enforce bundle1 in 'test-push-cgi.t' This test is checking bundle1 application, therefore we have to enforce the generated bundle to be a bundle1.

test: enforce v1 in 'test-debugbundle.t' This test is about bundle1 and should remain so when we move to generaldelta by default.

test: enforce bundle1 in "test-commit-interactive.t" This test generate a bundle to get a binary file to commit. We should ensure this binary file remains the same when we move to general delta as default.

discovery: factor out calculation of heads to not warn about In addition to taking a step towards getting an unreasonably large function factored into smaller, more manageable functions, this will allow extensions such as remotenames have more control over what pushes are allowed or not.

hooks: back 9f272bf3b342 out Changeset 9f272bf3b342 alters the 'HG_PENDING' mechanism to be "always" there. This change is made under the assumption than we previously did it only when "writepending() actually wrote something". This assumption was wrong, 'writepending()' informs of pending changes the first time something is written and for all following calls. We back this change out to restore the former behavior, which was already correct.

merge with stable

hooks: fix hooks not firing if prechangegroup was set (issue4934) We need to call delayupdate again after writing to the changelog. Otherwise the prechangegroup hook consumes the delayupdate subscription and future hooks don't see the pending changes (see issue 4934 for more details). Adds a test that triggers the prechangegroup hook before the pretxnchangegroup hook and verifies that the output of pretxnchangegroup doesn't change.

hooks: always include HG_PENDING Previously we would only include HG_PENDING in the hook args if the transaction's writepending() actually wrote something. This is a bad criteria, since it's possible that a previous call to writepending() wrote stuff and the hooks want to still see that. The solution is to always have hooks execute within the scope of the pending changes by always putting HG_PENDING in the environment.

wireproto: move clonebundles command from extension (issue4931) The SSH peer class accesses wireproto.commands[cmd] as part of encoding command arguments. Previously, the wire protocol command was defined in the clonebundles extension. If the client didn't have this extension enabled (which it likely doesn't since it is meant as a server-side extension), then clients attempting to clone via ssh:// would get a crash due to a KeyError accessing wireproto.commands['clonebundles'] when cloning from a server that is advertising clone bundles. Moving the definition of the wire protocol command to wireproto.py makes this problem go away. A side effect of this code move is servers will always respond to "clonebundles" wire protocol command requests. This should be fine: the server will return an empty response unless a clone bundles manifest file is present and clients shouldn't call the command unless the server is advertising the capability, which only happens if the clonebundles extension is enabled and the manifest file exists.

templatefilters: try round-trip utf-8 conversion by json filter (issue4933) As JSON string is known to be a unicode, we should try round-trip conversion for localstr type. This patch tests localstr type explicitly because encoding.fromlocal() may raise Abort for undecodable str, which is probably not what we want. Maybe we can refactor json filter to use encoding module more later. Still "{desc|json}" can't round-trip because showdescription() modifies a localstr object.

exchange: do not attempt clone bundle if local repo is non-empty (issue4932)

test-clonebundles.t: add test for incremental pull This demonstrates issue4932.

i18n: make sure to include translation of (DEPRECATED) This patch replaces old "DEPRECATED" msgid by "(DEPRECATED)" if that .po file does not have "(DEPRECATED)" but have "... (DEPRECATED)". It is necessary to hide deprecated options correctly.

i18n: do not abuse msgstr of "DEPRECATED" to check for bad translation Because 44cc9f63a2f1 requires the msgstr of "(DEPRECATED)", old *.po files must be blamed. Using "DEPRECATED" would just hide the error. For example, "LANG=da_DK.UTF-8 hg help serve" fails to hide deprecated options right now, but check-translation.py couldn't detect it because da.po has outdated translation of "DEPRECATED".

filemerge.prompt: separate out choice selection and action This will make future patches cleaner.

test-merge-prompt.t: add tests for linear updates + change/delete conflicts Turns out this bit of code was woefully untested.

test-pathencode.py: drop support for Python 2.4

hgweb.server: drop support for Python 2.4

util: drop Python 2.4 compat by directly importing md5 and sha1 There's been a fair amount of cruft here over the years, which we can all just get rid of now.

hgweb: replace some str.split() calls by str.partition() or str.rpartition() Since Python 2.5 str has new methods: partition and rpartition. They are more specialized than the usual split and rsplit, and they sometimes convey the intent of code better and also are a bit faster (faster than split/rsplit with maxsplit specified). Let's use them in appropriate places for a small speedup. Example performance (partition): $ python -m timeit 'assert "apple|orange|banana".split("|")[0] == "apple"' 1000000 loops, best of 3: 0.376 usec per loop $ python -m timeit 'assert "apple|orange|banana".split("|", 1)[0] == "apple"' 1000000 loops, best of 3: 0.327 usec per loop $ python -m timeit 'assert "apple|orange|banana".partition("|")[0] == "apple"' 1000000 loops, best of 3: 0.214 usec per loop Example performance (rpartition): $ python -m timeit 'assert "apple|orange|banana".rsplit("|")[-1] == "banana"' 1000000 loops, best of 3: 0.372 usec per loop $ python -m timeit 'assert "apple|orange|banana".rsplit("|", 1)[-1] == "banana"' 1000000 loops, best of 3: 0.332 usec per loop $ python -m timeit 'assert "apple|orange|banana".rpartition("|")[-1] == "banana"' 1000000 loops, best of 3: 0.219 usec per loop

help: replace some str.split() calls by str.partition() or str.rpartition() Since Python 2.5 str has new methods: partition and rpartition. They are more specialized than the usual split and rsplit, and they sometimes convey the intent of code better and also are a bit faster (faster than split/rsplit with maxsplit specified). Let's use them in appropriate places for a small speedup. Example performance (partition): $ python -m timeit 'assert "apple|orange|banana".split("|")[0] == "apple"' 1000000 loops, best of 3: 0.376 usec per loop $ python -m timeit 'assert "apple|orange|banana".split("|", 1)[0] == "apple"' 1000000 loops, best of 3: 0.327 usec per loop $ python -m timeit 'assert "apple|orange|banana".partition("|")[0] == "apple"' 1000000 loops, best of 3: 0.214 usec per loop Example performance (rpartition): $ python -m timeit 'assert "apple|orange|banana".rsplit("|")[-1] == "banana"' 1000000 loops, best of 3: 0.372 usec per loop $ python -m timeit 'assert "apple|orange|banana".rsplit("|", 1)[-1] == "banana"' 1000000 loops, best of 3: 0.332 usec per loop $ python -m timeit 'assert "apple|orange|banana".rpartition("|")[-1] == "banana"' 1000000 loops, best of 3: 0.219 usec per loop

convert: test clean p2 file missing 216fa1ba9993 introduced "clever" reuse of p2 but did that convert could fail with abort: f1@f73e02ae52c5: not found in manifest! when it tried to reuse a file from p2 but the file didn't exist there, for example because filemap changes. 5ca587348875 fixed that (using changes from a75d24539aba), but with a quite different reasoning and test case. Add another test that makes sure this case is covered too.

uescape: also encode non-printable char under 128 We were assuming everything under 128 was printable ascii, but there are a lot of control characters in that range that can't simply be included in json and other targets. We forcibly encode everything under 32, because they are either control char or oddly printable (like tab or line ending). We also add the hypothesis-powered test that caught this.

testing: add hypothesis fuzz testing Hypothesis a library for adding fuzzing over a range of structure data to your test suite: http://hypothesis.readthedocs.org/en/latest/ This adds the ability to build tests using Hypothesis within the Mercurial test suite. New tests and fixes using this helpers comes in later changesets.

merge with stable

Added signature for changeset 47dd34f2e727

Added tag 3.6 for changeset 47dd34f2e727

i18n: look translation of both "DEPRECATED" and "(DEPRECATED)" up Since 44cc9f63a2f1, deprecated commands, options and so on are detected by "(DEPRECATED)" instead of "DEPRECATED". "hg.pot" generated from recent source files doesn't contain msgid "DEPRECATED", and looking the translation of "DEPRECATED" up in up-to-date *.po files works incorrectly. But on the other hand, there are still old *.po files, which contain msgid "DEPRECATED" but not "(DEPRECATED)". Looking the translation of "(DEPRECATED)" up in such old *.po files also works incorrectly. This patch resolves this problem by looking translation of both "DEPRECATED" and "(DEPRECATED)" up. This should work correctly, because previous patch makes "deprecated" checker be applied only on translations, of which msgid contains exact "(DEPRECATED)" string. 'p.msgstr' examination in 'deprecatedsetup()' is needed to ignore untranslated entries. This also makes 'deprecatedpe.msgstr' examination in 'deprecated()' meaningless.

i18n: fix regexp pattern to detect translation for DEPRECATED Since 44cc9f63a2f1, deprecated commands, options and so on are detected by "(DEPRECATED)" instead of "DEPRECATED". Therefore, 'deprecated' checker in i18n/check-translation.py should check translation, of which msgid contains "(DEPRECATED)" instead of "DEPRECATED". At glance, it seems to do so, but it actually doesn't, because Python regexp treats "()" as grouping of patterns and "(DEPRECATED)" matches only against "DEPRECATED".

scmutil: abort if an empty revision is given to revpair() When using 'extdiff --patch' to check the changes in a rebase, 'precursors(x)' evaluated to an empty set because I forgot the --hidden flag, so the other revision was used as the replacement for the empty set. The result was the patch for the other revision was diffed against itself, and the tool saying there were no differences. That's misleading since the expected diff args were silently changed, so it's better to bail out. The other uses of scmutil.revpair() are commands.diff and commands.status, and it doesn't make sense to allow an empty revision there either. The code here was suggested by Yuya Nishihara.

i18n-pt_BR: synchronized with a9ed5a8fc5e0

i18n-ja: synchronized with 6474b64045fb

packaging: rework version detection and declaration (issue4912) Previously the -rc in our rc tags got dropped, meaning that those packages looked newer to the packaging system than the later release build. This rectifies the issue, though some damage may already have been done on 3.6-rc builds. I'm mostly cargo-culting the RPM version format - there don't appear to be rules for RPM about how to handle this. Hopefully an RPM enthusiast can fix up what I've done as a followup.

hgweb: escape class keyword when used as a js object property (issue4913)

localrepo: discard objects in _filecache at transaction failure (issue4876) 'repo.invalidate()' deletes 'filecache'-ed properties by 'filecache.__delete__()' below via 'delattr(unfiltered, k)'. But cached objects are still kept in 'repo._filecache'. def __delete__(self, obj): try: del obj.__dict__[self.name] except KeyError: raise AttributeError(self.name) If 'repo' object is reused even after failure of command execution, referring 'filecache'-ed property may reuse one kept in 'repo._filecache', even if reloading from a file is expected. Executing command sequence on command server is a typical case of this situation (5c0f5db65c6b also tried to fix this issue). For example: 1. start a command execution 2. 'changelog.delayupdate()' is invoked in a transaction scope This replaces own 'opener' by '_divertopener()' for additional accessing to '00changelog.i.a' (aka "pending file"). 3. transaction is aborted, and command (1) execution is ended After 'repo.invalidate()' at releasing store lock, changelog object above (= 'opener' of it is still replaced) is deleted from 'repo.__dict__', but still kept in 'repo._filecache'. 4. start next command execution with same 'repo' 5. referring 'repo.changelog' may reuse changelog object kept in 'repo._filecache' according to timestamp of '00changelog.i' '00changelog.i' is truncated at transaction failure (even though this truncation is unintentional one, as described later), and 'st_mtime' of it is changed. But 'st_mtime' doesn't have enough resolution to always detect this truncation, and invalid changelog object kept in 'repo._filecache' is reused occasionally. Then, "No such file or directory" error occurs for '00changelog.i.a', which is already removed at (3). This patch discards objects in '_filecache' other than dirstate at transaction failure. Changes in 'invalidate()' can't be simplified by 'self._filecache = {}', because 'invalidate()' should keep dirstate in 'self._filecache' 'repo.invalidate()' at "hg qpush" failure is removed in this patch, because now it is redundant. This patch doesn't make 'repo.invalidate()' always discard objects in '_filecache', because 'repo.invalidate()' is invoked also at unlocking store lock. - "always discard objects in filecache at unlocking" may cause serious performance problem for subsequent procedures at normal execution - but it is impossible to "discard objects in filecache at unlocking only at failure", because 'releasefn' of lock can't know whether a lock scope is terminated normally or not BTW, using "with" statement described in PEP343 for lock may resolve this ? After this patch, truncation of '00changelog.i' still occurs at transaction failure, even though newly added revisions exist only in '00changelog.i.a' and size of '00changelog.i' isn't changed by this truncation. Updating 'st_mtime' of '00changelog.i' implied by this redundant truncation also affects cache behavior as described above. This will be fixed by dropping '00changelog.i' at aborting from the list of files to be truncated in transaction.

demandimport: fix TypeError when importing Python regex library (issue4920)

bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.

tests: add tests for remote hook output (issue4788) The added tests don't agree in their output. This demonstrates a difference in `hg push` behavior between pre-bundle2 and bundle2. A subsequent patch will attempt to restore some of the pre-bundle2 behavior to bundle2.

help: mention alias and revsetalias in description of HGPLAINEXCEPT

debugrevlog: cope with empty revlog files I have no idea where it came from, but my clone of Mercurial has an empty filelog for `contrib/hgfixes/__init__.py` - it's *valid*, just contains no nodes. Without this change, debugrevlog crashes with a zero division error.

largefiles: don't mute and obfuscate http errors when putlfile fails 'unexpected putlfile response: None' when an http error occurs is not very helpful. Instead, leave the handling of urllib2.HTTPError exceptions to other layers.

largefiles: test coverage of error handling from putlfile This reveals a wrong and unhelpful 'unexpected putlfile response'.

largefiles: check hash of files in the store before copying to working dir If the store somehow got corrupted, users could end up in weird situations that were very hard to recover from or lead to propagation of the corruption. Instead, spend the extra time checking the hash when copying to the working directory. If it doesn't match, emit a warning, and don't put wrong content in the working directory.

largefiles: test coverage of handling of store corruption This reveals that update might put a corrupted largefile in the working directory where it will show up as modified and ready for commit.

merge: improve clarity of table in update docstring

ui: support paths.default-push without paths.default set (issue4914) This behavior regressed as part of the paths API refactoring. Previous behavior was to accept "default-push" without "default" defined. Current behavior aborts with "default repository not configured!." This patch restores the old behavior and adds test coverage for the scenario, which was absent before.

bookmarks: don't allow pushing new head for existing mark with -B (issue4400)

commands: fix help for debugobsolete

largefiles: fix explicit commit of normal/largefile switch Commit of corresponding normal/largefiles pairs would only commit the standin. That is usually fine, except if either the normal file or the standin is a remove while the other is an add. In that case it would either give duplicate colliding entries or lose the file. Instead, commit both filenames if one of them is a remove.

largefiles: test coverage of explicit commit of normal/largefile switch This reveals that a switch from normal to largefile violates the normal largefile invariant and gives a manifest with both a normal and standin file in the repository, while a switch from largefile to normal gives nothing. Not good.

Added signature for changeset b66e3ca0b90c

Added tag 3.6-rc for changeset b66e3ca0b90c

merge default into stable for code freeze

convert: fix Python syntax in 'splice in' message Instead of reporting spliced in ['82544090e14fe18091e04f1fb0f0d7991cbe6e7e'] as parents of 369fd983d9e13330e9f12d9fce820deae84ea223 report spliced in 82544090e14fe18091e04f1fb0f0d7991cbe6e7e as parents of 369fd983d9e13330e9f12d9fce820deae84ea223

rebase: fix wrong 'no changes to commit' when using --collapse --collapse will do that rebase doesn't commit until the final commit. The lack of a new commit would make it look like the rebase didn't contribute any changes. Instead, only warn about no commits when not using --collapse.

pull: all pass along extra opargs Wihtout this patch, commands.pull silently drops opargs defeating the whole purpose of the argument.

push: all pass along opargs Wihtout this patch, commands.push silently drops opargs defeating the whole purpose of the argument.

check-code: allow argument passing py2.6ism this backs out 131f7fe06e9e, because Python2.5 support was dropped

mail: drop python 2.5 self.sock.read workaround

url: drop support for python2.5

win32: drop reference to python2.5

editmerge: dequote other use of $ED We want to support editors with parameters, eg EDITOR="vim -O" or whatever. So remove the quotes from around $ED and assume that the editor variable is properly escaped already.

bundlerepo: properly extract compressed changegroup from bundle2 Before this bundle repository were unable to work with compressed bundle2. We use the same approach as with bundle1, we extract the changegroup in uncompressed form into a temporary file.

bundle2: make unbundle.compressed return True when compressed We were returning 'False' in all cases, even when the bundle2 was actually compressed.

bundlerepo: uncompress changegroup in bundle1 case only Uncompressing bundle2 needs to be handled differently.

bundlerepo: move temp-bundle writing logic into a closure We will reuse this logic for bundle2

shelve: properly process bundle2 bundle

histedit: properly apply bundle2 backups If the histedit fails, we restore a backup. We make sure this backup bundle can be in bundle2 format as general delta will require.

strip: pass source and url to bundle2 processing Restoring from a 'bundle2' was missing this data.

unbundle: use 'url' argument with applybundle This will let the hook get access to the 'url' value.

applybundle: take url as argument We allow specifying the url to carry it to hooks. This gets us closer to 'bundle1.apply(...)' and will allow us to remove regressions in multiple place where we forget to pass the url to hooks.

unbundle: use 'source' argument with applybundle This will let the hook get access to the 'source' value.

applybundle: take source as argument We allow specifying the source to carry it to hooks. This gets us closer to 'bundle1.apply(...)' and will allow us to remove regressions in multiple places where we forget to pass the source to hooks.

applybundle: set 'bundle2=1' env for all transaction This should be set for all bundle2 application, we enforce that at a low level. This is for courtesy with hooks.

unbundle: use 'bundle2.applybundle' This is one such place.

bundle2: introduce an "applybundle" function There is a case where the intent is clear and the transaction is not optional. We want to be able to alter that transaction in a wide and easy way. We cannot get a unified '.apply(repo)' method for bundle1 and bundle2 yet because the api are still a bit too far apart. But this is a good step forward to get the rc out.

commands.resolve: conclude merge driver if no unresolved files are left This can happen when either 'hg resolve --all' is called or a driver-resolved file is explicitly requested. This is done as part of 'hg resolve --all' so that users still have a chance to test their changes before committing them. The exact semantics here are still to be decided. This does not impact any non-experimental features. Thanks to Pierre-Yves David for some advice about this behavior in particular, and merge drivers in general.

commands.resolve: call driverpreprocess if we haven't run it yet We need to be careful about allowing --mark and --unmark to keep working -- we don't want the user to be stuck in a weird state. The exact behavior here is still to be decided, though.

merge.applyupdates: call driverconclude after performing merge actions This will be a chance for the merge driver to finish resolving or generating any driver-resolved files. As before, having a separate error state from 'unresolved' is too big a refactoring for now, so we hack around it by setting unresolved to a positive value when necessary.

merge.applyupdates: call driverpreprocess before starting merge actions We also need to update our internal state to whatever state driverpreprocess leaves it in. Adding an error state separate from the unresolved count is too big a refactoring for now, so we hack around it by setting it to a positive value to indicate an error state.

merge: add stubs for preprocess and conclude steps of merge driver The exact semantics for what should happen (particularly with respect to error handling) are still a bit hard to pin down, so I think it's better to experiment with it as an extension for now. For now this stub will act as a convenient point for extensions to hook on.

commands.resolve: don't allow users to mark or unmark driver-resolved files Users will often be in the habit of running 'hg resolve --mark --all' after resolving merge conflicts in source files. We need to make sure this doesn't cause driver-resolved files to be marked. 'hg resolve --all' will resolve driver-resolved files, though. The weird conditional structure is to accommodate an upcoming patch.

resolve: only create wctx once This allows for status and other results on the wctx to be cached between iterations.

dirstate: fix invalid reference to self.ui

spelling: trivial spell checking

mq: consistently use qrefresh

grammar: use does instead of do where appropriate

l10n: use %d instead of %s for numbers

check-code: block non-portable pipe-and

test-shelve: do not use non-portable pipe-and

parsers: suppress warning of signed and unsigned comparison at nt_init Spotted by CC=clang CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wshorten-64-to-32': mercurial/parsers.c:1580:24: warning: comparison of integers of different signs: 'Py_ssize_t' (aka 'long') and 'unsigned long' [-Wsign-compare] if (self->raw_length > INT_MAX / sizeof(nodetree)) {

parsers: correct type of temporary variables for dirstate tuple fields These fields are defined as int. This eliminates the following warning spotted by CC=clang CFLAGS='-Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wshorten-64-to-32': mercurial/parsers.c:625:29: warning: comparison of integers of different signs: 'uint32_t' (aka 'unsigned int') and 'int' [-Wsign-compare] if (state == 'n' && mtime == now) {

test: enforce v1 type in 'test-bundle2-remote-changegroup.t' This bundle command is meant to generate a bundle1, we enforce that to avoid test's misbehavior when moving to general-delta by default.

test: enforce v1 type in 'test-bundle2-format.t' This bundle command is meant to generate a bundle1, we enforce that to avoid test's misbehavior when moving to general-delta by default.

editmerge: properly quote variables Previously, files with spaces would break editmerge.

commands.resolve: print out warning when only driver-resolved files remain 'hg resolve --all' will be the canonical way to run the 'conclude' step of the merge driver.

merge.mergestate: set merge driver state to 's' if there's none present This allows mergestate.mdstate() to return 's' if there's nothing to be done on the merge driver end.

merge.mergestate: only check for merge driver when property is accessed Otherwise 'hg update --clean', 'hg rebase --abort' etc wouldn't work.

localrepo.commit: abort if merge driver's conclude step hasn't been run yet This case also requires 'hg resolve --all' to be run before continuing.

merge.mergestate: add a way to get the merge driver state This will be useful to check what the status of the merge driver is.

merge.mergestate: add a way to get the other side of the merge It's surprising there was no API at all for this until now. In any case this will be needed from custom merge drivers.

commands.resolve: support printing out driver-resolved files There's no user-visible way to mark files as driver-resolved, so this status won't be visible.

histedit: make histedit prune when obsolete is enabled Back in June we made histedit use obsolete markers to cleanup when possible. This was rolled back as part of 54f9561088c7 (which should have only rolled back the --abort stuff, but rolled back everything). This caused a nasty bug when used in conjuction with the inhibit+directaccess extensions where histedit would leave old nodes around even after they had been squashed away. The root of the problem is that we first clean up old nodes, and then we clean up temp nodes. In the first pass, when we obsoleted old nodes, some would become unobsolete because they had temp nodes on top of them, thus making them stick around even after the histedit finished. The fix is to A) move the temp node cleanup to be before the old node cleanup (since they are topological on top of the old nodes), and B) use obsolete markers instead of stripping.

clonebundles: rewrite documentation There are a lot of considerations server operators need to know before deploying clone bundles. They should be documented. So I rewrote the extension docs to contain this information.

exchange: support streaming clone bundles in clone bundles Now that we have support for detecting compatible stream clone bundles in bundle specifications, we can safely add support for applying stream clone bundles to the clone bundles feature.

exchange: parse requirements from stream clone specification string Stream clone bundles can only be consumed if the consumer supports the exact format requirements that were present on the producer. This patch adds support for encoding and verifying the format requirements on the bundle specification string for a stream clone bundle are supported by the local repository. If they aren't, we raise an UnsupportedBundleSpecification, just like we do when an unknown compression or bundle type is encountered. The impetus for this patch is so the clone bundles manifest can advertise stream clone bundles and so clients can filter out stream clones with unsupported format requirements. e.g. a stream clone produced with the not-yet-invented "revlogv2" format will be ignored by clients that only support "revlogv1."

exchange: support parameters in bundle specification strings Sometimes a basic type string is not sufficient for representing the contents of a bundle. Take bundle2 for example: future bundle2 files may contain parts that today's bundle2 parser can't read. Another example is stream clone data. These require clients to support specific repository formats or they won't be able to read the written files. In both scenarios, we need to describe additional metadata beyond the outer container type. Furthermore, this metadata behaves more like an unordered set, so an order-based declaration format (such as static strings) is not sufficient. We introduce support for "parameters" into the bundle specification string. These are essentially key-value pairs that can be used to encode additional metadata about the bundle. Semicolons are used as the delimiter partially to increase similarity to MIME parameter values (see RFC 2231) and because they are relatively safe from the command line (although values will need quotes to avoid interpretation as multiple shell commands). Alternatives considered were spaces (a bit annoying to encode) and '&' (similar to URL query strings) (which will do bad things in a shell if unquoted). The parsing function now returns a dict of parsed parameters and consumers have been updated accordingly.

commands: support consuming stream clone bundles For the same reasons that we don't produce stream clone bundles with `hg bundle`, we don't support consuming stream clone bundles with `hg unbundle`. We introduce a complementary debug command for applying stream clone bundles. This command is mostly to facilitate testing. Although it may be used to manually apply stream clone bundles until a more formal mechanism is (possibly) adopted.

commands: support creating stream clone bundles Now that we have support for recognizing the streaming clone bundle type, add a debug command for creating them. I decided to create a new debug command instead of adding support to `hg bundle` because stream clone bundles are not exactly used the same way as normal bundle files and I don't want to commit to supporting them through the official `hg bundle` command forever. A debug command, however, can be changed without as much concern for backwards compatibility. As part of this, `hg bundle` will explicitly reject requests to produce stream bundles. This command will be required by server operators using stream clone bundles with the clone bundles feature.

exchange: support for streaming clone bundles Now that we have a mechanism to produce and consume streaming clone bundles, we need to teach the human-facing bundle specification parser and the internal bundle file header reading code to be aware of this new format. This patch does so. For the human-facing bundle specification, we choose the name "packed" to describe "streaming clone bundles" because the bundle is essentially a "pack" of raw revlog files that are "packed" together. There should probably be a bikeshed over the name, especially since it is human facing.

streamclone: support for producing and consuming stream clone bundles Up to this point, stream clones only existed as a dynamically generated data format produced and consumed during streaming clones. In order to support this efficient cloning format with the clone bundles feature, we need a more formal, on disk representation of the streaming clone data. This patch introduces a new "bundle" type for streaming clones. Unlike existing bundles, it does not contain changegroup data. It does, however, share the same concepts like the 4 byte header which identifies the type of data that follows and the 2 byte abbreviation for compression types (of which only "UN" is currently supported). The new bundle format is essentially the existing stream clone version 1 data format with some headers at the beginning. Content negotiation at stream clone request time checked for repository format/requirements compatibility before initiating a stream clone. We can't do active content negotiation when using clone bundles. So, we put this set of requirements inside the payload so consumers have a built-in mechanism for checking compatibility before reading and applying lots of data. Of course, we will also advertise this requirements set in clone bundles. But that's for another patch. We currently don't have a mechanism to produce and consume this new bundle format. This will be implemented in upcoming patches. It's worth noting that if a legacy client attempts to `hg unbundle` a stream clone bundle (with the "HGS1" header), it will abort with: "unknown bundle version S1," which seems appropriate.

spelling: fix typo in transaction error messages

transaction: reorder unlinking .hg/journal and .hg/journal.backupfiles After this reordering, absence of '.hg/journal' just before starting new transaction means also absence of '.hg/journal.backupfiles'. In this case, all temporary files for preceding transaction should be completely unlinked, and HG_PENDING doesn't cause unintentional reading stalled temporary files in. Otherwise, 'repo.transaction()' raises exception with "run 'hg recover' to clean up transaction" hint.

merge: make in-memory changes visible to external update hooks 51844b8b5017 (while 3.4 code-freeze) made all 'update' hooks run after releasing wlock for visibility of in-memory dirstate changes. But this breaks paired invocation of 'preupdate' and 'update' hooks. For example, 'hg backout --merge' for TARGET revision, which isn't parent of CURRENT, consists of steps below: 1. update from CURRENT to TARGET 2. commit BACKOUT revision, which backs TARGET out 3. update from BACKOUT to CURRENT 4. merge TARGET into CURRENT Then, we expects hooks to run in the order below: - 'preupdate' on CURRENT for (1) - 'update' on TARGET for (1) - 'preupdate' on BACKOUT for (3) - 'update' on CURRENT for (3) - 'preupdate' on TARGET for (4) - 'update' on CURRENT/TARGET for (4) But hooks actually run in the order below: - 'preupdate' on CURRENT for (1) - 'preupdate' on BACKOUT for (3) - 'preupdate' on TARGET for (4) - 'update' on TARGET for (1), but actually on CURRENT/TARGET - 'update' on CURRENT for (3), but actually on CURRENT/TARGET - 'update' on CURRENT for (4), but actually on CURRENT/TARGET Root cause of the issue focused by 51844b8b5017 is that external 'update' hook process can't view in-memory changes (especially, of dirstate), because they aren't written out until the end of transaction (or wlock). Now, hooks can be invoked just after updating, because previous patches made in-memory changes visible to external process. This patch may break backward compatibility from the point of view of "scheduling hook execution", but should be reasonable because 'update' hooks had been executed in this order before 3.4. This patch tests "hg backout" and "hg unshelve", because the former activates the transaction before 'update' hook invocation, but the former doesn't.

hook: centralize passing HG_PENDING to external hook process This patch centralizes passing HG_PENDING to external hook process into '_exthook()'. To make in-memory changes visible to external hook process, this patch does: - write (or schedule to write) in-memory dirstate changes, and - set HG_PENDING environment variable, if: - a transaction is running, and - there are in-memory changes to be visible This patch tests some commands with some hooks, because transaction activity of a same hook differs from each other ("---": "not tested"). ======== ========= ========= ============ command preupdate precommit pretxncommit ======== ========= ========= ============ unshelve o --- --- backout x --- --- import --- o o qrefresh --- x o ======== ========= ========= ============ Each hooks are examined separately to prevent in-memory changes from being visible to external process accidentally by side effect of hooks previously invoked.

cmdutil: make in-memory changes visible to external editor (issue4378) Before this patch, external editor process for the commit log can't view some in-memory changes (especially, of dirstate), because they aren't written out until the end of transaction (or wlock). This causes unexpected output of Mercurial commands spawned from that editor process. To make in-memory changes visible to external editor process, this patch does: - write (or schedule to write) in-memory dirstate changes, and - set HG_PENDING environment variable, if: - a transaction is running, and - there are in-memory changes to be visible "hg diff" spawned from external editor process for "hg qrefresh" shows: - "changes newly imported into the topmost" before 49148d7868df(*) - "all changes recorded in the topmost by refreshing" after this patch (*) 49148d7868df changed steps invoking editor process Even though backward compatibility may be broken, the latter behavior looks reasonable, because "hg diff" spawned from the editor process consistently shows "what changes new revision records" regardless of invocation context. In fact, issue4378 itself should be resolved by 800e090e9c64, which made 'repo.transaction()' write in-memory dirstate changes out explicitly before starting transaction. It also made "hg qrefresh" imply 'dirstate.write()' before external editor invocation in call chain below. - mq.queue.refresh - strip.strip - repair.strip - localrepository.transaction - dirstate.write - localrepository.commit - invoke external editor Though, this patch has '(issue4378)' in own summary line to indicate that issues like issue4378 should be fixed by this. BTW, this patch adds '-m' option to a 'hg ci --amend' execution in 'test-commit-amend.t', to avoid invoking external editor process. In this case, "unsure" states may be changed to "clean" according to timestamp or so on. These changes should be written into pending file, if external editor invocation is required, Then, writing dirstate changes out breaks stability of test, because it shows "transaction abort!/rollback completed" occasionally. Aborting after editor process invocation while commands below may cause similar instability of tests, too (AFAIK, there is no more such one, at this revision) - commit --amend - without --message/--logfile - import - without --message/--logfile, - without --no-commit, - without --bypass, - one of below, and - patch has no description text, or - with --edit - aborting at the 1st patch, which adds or removes file(s) - if it only changes existing files, status is checked only for changed files by 'scmutil.matchfiles()', and transition from "unsure" to "normal" in dirstate doesn't occur (= dirstate isn't changed, and written out) - aborting at the 2nd or later patch implies other pending changes (e.g. changelog), and always causes showing "transaction abort!/rollback completed"

dirstate: show develwarn for write() invocation without transaction This is used to detect 'dirstate.write()' invocation without the value gotten by 'repo.currenttransaction()' (mainly focused on 3rd party extensions).

dirstate: make dirstate.write() callers pass transaction object to it Now, 'dirstate.write(tr)' delays writing in-memory changes out, if a transaction is running. This may cause treating this revision as "the first bad one" at bisecting in some cases using external hook process inside transaction scope, because some external hooks and editor process are still invoked without HG_PENDING and pending changes aren't visible to them. 'dirstate.write()' callers below in localrepo.py explicitly use 'None' as 'tr', because they can assume that no transaction is running: - just before starting transaction - at closing transaction, or - at unlocking wlock

dirstate: remove layering violation around writing dirstate out This violation, which passes repo object to dirstate, was introduced by 09bb1ee7e73e. This patch uses 'False' instead of 'None' as default value of 'tr' argument, to distinguish "None as repo.currenttransaction() result" from "legacy invocation without explicit tr passing".

dirstateguard: remove layering violation around saving/restoring backup This violation, which passes 'repo' object to dirstate, was introduced by 59b5e8844eb0 and 020b12d591f3.

rebase: reduce scope of try catch in restorestate Refactoring by reduce the scope of the try catch block so that it only captures what it needs. I could have made it smaller but another patch in the series will add onto it.

rebase: on abort delete rebase state file no matter what When a user's repository is in an unfinished rebase state and they choose to abort, at a minimum, the repo should be out of that state. We've found situations where the user could not leave the state unless manually deleting the rebasestate file. This fix ensures that no matter what exception may be raised during the abort, the rebase state file will be deleted, the user will be out of the rebase state and they can get their repository into a workable condition.

contrib: make editmerge look for merge markers at the beginning of the line This fix adds a caret to the start of the regex looking for merge markers. This avoids the issue arises when you've real merge conflicts in a file that tests for the existance of merge markers in test output. Editmerge will not open on the fake/tested merge markers because they'll be indented in.

commit: abort when a committemplate is not changed If a committemplate is provided and no message is provided on the command line, and no edits are made to the commit template, then abort the commit.

localrepo.commit: disallow commit when driver-resolved files exist This code will not currently be activated because there's no code to mark files as driver-resolved in core. This point is also somewhat hard to plug into from extensions.

merge.mergestate: add a generator for driver-resolved files Just like for unresolved files above, we need to be able to tell what files are driver-resolved.

hook: for python hooks, also return whether an exception was raised The hook code treats python hooks raising an exception and returning True as the exact same. This is OK for hooks themselves, but other code that wants to invoke external code using the same underlying code is a bit more interested in making a distinction.

hook.runhooks: return a dict of result values This will be useful to other calling code that would be interested in what the individual hooks return.

hook: factor out determination of hooks from running them This will allow other code to run a predetermined series of hooks.

mq: generate patch names from first line of description Avoid the pointless numeric rev.diff patch names. Instead, do like mbox extension does and create meaningful patch names.

contrib: offer Python 2.7.10

contrib: drop Python < 2.6 from Makefile.python

mergetools.rc: find OSX FileMerge in the new location inside Xcode 4.3

exchange: don't print error codes after clone bundle failure We don't appear to print error codes elsewhere. The error codes are inconsistent between at least Linux and OS X and are more trouble than they are worth. Humans care about the error string more than the code anyway. A glob was also added to pave over differences in error strings between Linux and OS X.

debian: install bash completion as hg and not mercurial (issue4900)

merge-tools: allow marking a mergetool as completely disabled Very often in my life I'm finding that the only configured merge tool present on the system is vimdiff[0], and it's currently impossible (as far as I can tell) short of specifying `ui.merge = `[1] to actually *disable* a merge tool. This allows vimdiff-haters to put: [merge-tools] vimdiff.disable = yes in their ~/.hgrc and never see vimdiff again. I'm stopping short of putting this as a commented out entry in the sample new user hgrc (seen when a user runs `hg config --edit` with no ~/.hgrc) for now, but I might come back and do that later. 0: vimdiff is at an awkward intersection: it's usually installed by the vim package which is often installed as a vi substitute, so it's mere presence doesn't imply me wanting it, unlike (say) kdiff3. 1: There's a related problem I ran into today where specifying `ui.merge = :merge` failed because :merge isn't a command, which I think is a regression. I'll try and figure that out and at least file a bug.

exchange: add oparg to push so that extensions can wrap pushop

destmerge: extract logic based on branch heads in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destmerge: extract logic based on bookmark into its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: have a generic and extensible way to run each step We want extension to be able to easily override or add new way to select the default update destination. We use the same list + dict approach as in other parts of the code.

destupdate: extract logic based on branch in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: extract logic based on bookmarks in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: extract logic based on obsolescence marker in its own function One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

destupdate: move obsolete handling first This block was overwriting any result from the previous block anyway. So we move it first to prove it is possible and we'll extract it in its own function in the next patch.

destupdate: indent bookmark and branch logic We'll move the obsolete related logic first (as it is overwriting any other anyway) to make the next patch clearer we add indentation in this one.

destupdate: extract validation logic One of the main goal of having consolidated destination function is to allow extension to play with this logic. We extract sub logic to make is wrapping more practical.

rebase: rename and test '_destrebase' We make the name consistent with the other similar revsets and make sure it has minimal tests.

rebase: directly use '_destrebase' There is little value in using the revset instead of the function.

rebase: extra default destination in its own function This makes it much simple to wrap for other extension.

revset: rename and test '_destmerge' We make the name consistent with the one used by '_destupdate' and we ensure the code is run by testing it (abort is expected and merge would).

merge: directly get destination from destutil There is no real value in using the revset over the function. The revset have no remaining users and will be taken care of in a later changesets.

destutil: move default merge destination into a function Function in destutil are much simpler to wrap and more flexible than revset. This also help consistency as 'destupdate' live here and cannot become a pure revset anyway.

revset: reintroduce and experimental revset for update destination The revset is not ready for prime time yet. However it is useful to have some version of it exposed to help candidate users to play with it and provide feedback on what we should aim at. We add a small test to make sure the code runs.

changegroup: move manifest unpacking into its own method The upcoming cg3 will need different logic for unpacking manifests.

changegroup: move manifest packing into a separate function A future change will introduce a new function on a cg3packer that can pack treemanifests as well as flatmanifests.

changegroup: rename manifest linknode closure for clarity Since I'm spending the time to understand this code, I may as well leave it clearer than I found it.

changegroup: reformat packermap and add comment I'm about to add a cg3, and it seems prudent to annotate what formats support what features. It strikes me that we may want to consider moving to a more feature-oriented model in the future, but we'll see how that looks in a little while I guess.

changegroup: document the public surface area of cg?unpackers This should help future readers at least a little.

changegroup: mark cg1unpacker.chunklength as private

changegroup: note why a few methods on cg1unpacker exist I'm not sure what to do abstraction-wise here. It might be more sensible to make a memoryrepo that could apply a bundle in-memory and then we could make the changegroup data be strictly an applyable stream, but that's an idea for Later.

revlog: rename bundle to cg to reflect its nature as a cg?unpacker The new convention is that bundles contain changegroups. bundle1 happens to *only* be a changegroup, but bundle2 is a more featureful container that isn't something you can pass to addgroup().

changegroup: mark _addchangegroupfiles as module-private I'm trying to reason about the public surface area of this module now, so it's worth tagging private things as such.

changegroup: delete now-unused addchangegroup method

localrepo: use cg?unpacker.apply() instead of changegroup.addchangegroup() This is in localpeer, so it lives. Had it been in localrepo instead, I would have tried to exterminate it.

repair: use cg?unpacker.apply() instead of changegroup.addchangegroup()

exchange: use cg?unpacker.apply() instead of changegroup.addchangegroup()

commands: use cg?unpacker.apply() instead of changegroup.addchangegroup()

bundle2: use cg?unpacker.apply() instead of changegroup.addchangegroup()

shelve: use cg?unpacker.apply() instead of changegroup.addchangegroup()

histedit: use cg?unpacker.apply() instead of changegroup.addchangegroup()

changegroup: migrate addchangegroup() to forward to cg?unpacker.apply() I'll clean up callers in subsequent patches, then remove the forwarding.

changegroup: move source check to top of addchangegroup This is preparation for some refactoring.