subrepo: add partial diff support for git subrepos So far, git subrepositories were silently ignored for diffs. This patch adds support for git subrepositories, with the remark that --include and --exclude are not supported. If --include or --exclude are used, the subrepo is ignored.

subrepo: extend git version check to 3 digits This allows more flexibility when a version check is required. Some git features are introduced in a version where only the 3rd digit changes.

subrepo: move git version check into a separate method This allows checking the git version in other methods, instead of only being able to check if the version is ok or not.

rebase: show a note for updated mq patches It deserves more than a debug message. Show a note like: updating mq patch p0.patch to 5:9ecc820b1737 The message could also refer to "qrefresh" instead. Same same.

rebase: improve debug messages while processing the list of rebases

rebase: show warning when rebase creates no changes to commit Similar to graft: note: rebase of 6:eea13746799a created no changes to commit

rebase: show more useful status information while rebasing Show status messages while rebasing, similar to what graft do: rebasing 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: make 'saved backup' globbing less narrow in rebase tests Globbing the hash made it harder to maintain tests with run-tests -i when it was so far by the generated test output. The hashes are stable and we just need to add a (glob).

tests: stabilise mq rebase tests Prepare for including hashes in output ... and less globbing make the tests easier to update.

test-graft: use strip extension instead of mq extension It only needs strip, no reason to load all of mq.

transaction: remove the 'onabort' mechanism It has no known users. If someones needs similar functionality, a new 'addabort' method similar to 'addfinalize' should be added.

transaction: remove the redundant 'onclose' mechanism It is superseded by the 'addfinalize' function and all its user have been migrated.

fncache: document the fact fncache is outdate at hook run time Using 'addfinalize' to generate 'fncache' means that no pending version of the file will be generated for the hooks. We would have to use the 'addfilegenerator' method to get such result. However the 'fncachevfs' (who decide that a write is necessary) have no access to the transaction to register such file generation at add time. Having the transaction accessible to the 'vfs' is too much trouble for no benefit. This outdated 'fncache' file at hook time is not expected to be an issue. The previous move from 'onclose' to 'addfinalize' had no impact on this timing. I'm documenting it now because I looked at it.

fncache: drop dedicated 'onclose' function in favor of 'tr.addfinalize' Now that we have a shiny generic mechanism, we can use it.

merge with stable

graft: drop cset description from empty commit message This is either already redundant in the output or too verbose in quiet mode.

graft: show hashes in user-facing messages Graft was in various places just showing the revision number in status messges. Instead, also show the stable and more useful short hash.

graft: give helpful warning for empty grafts It was just showing a status message with the internal revision number. Instead, show a warning like note: graft of 27:3aaa8b6725f0 "28" created no changes to commit (message tweaked in-flight by mpm)

graft: show more useful status information while grafting Show status messages with first line of commit description and names, like grafting 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: test coverage for empty graft A future change will add a warning to the quiet rebase.

largefiles: drop the unfiltered repo usage in overridepurge() Now that repoview supports replacing methods, we don't need this hack.

repoview: allow methods on the proxy class to be replaced It doesn't seem to be a common idiom for repo instances, but the status() method is replaced in largefiles' purge() override. Since __setattr__ is implemented in repoview to setattr() on the unfiltered repo, the replacement method wouldn't get called unless it was invoked with the unfiltered repo, because the filtered repo remains unchanged. Since this doesn't seem to be commonly used, I didn't bother to filter out methods that perhaps shouldn't be replaced, such as changelog().

log: fix log revset instability The log/graphlog revset was not producing stable results since it was iterating over a dict. Now we sort before iterating to guarantee a fixed order. This fixes some potential flakiness in the tests.

log: fix log -f slow path to actually follow history The revset created when -f was used with a slow path (for patterns and directories) did not actually contain any logic to enforce follow. Instead it was depending on the passed in subset to already be limited (which was limited to :. but not ::.). This fixes it by adding a '& ::.' to any -f log revset. hg log -f <file> is still broken, in that it can return results that aren't actually ancestors of the current file, but fixing that has major perf implications, so we'll deal with it later.

obsstore: cache size computation for fm1 node We have two different types of node type (sha1 and sha256, only sha1 is used now) and therefor different sizes for them. We now compute the value once instead of redoing the computation every loop. This has no visible performance impact.

obsstore: prefetch struct.calcsize This function is widely used and worth but be at module level. No specific performance boost is visible, but this is more consistent.

obsstore: disable garbage collection during initialization (issue4456) Python garbage collection is triggered by container creation. So code that creates a lot of tuples tends to trigger GC a lot. We disable the gc during obsolescence marker parsing and associated initialization. This provides an interesting speedup (25%). Load marker function on my 58758 markers repo: before: 0.468247 seconds after: 0.344362 seconds The benefit is a bit less visible overall. With python2.6 on my system I see: after: 0.60 before: 0.53 The difference is probably explained by the delaying of a costly GC. (but there is still a win). Marking involved tuples, lists and dicts as ignorable by the garbage collector should give us more benefit. But this is another adventure. Thanks goes to Siddharth Agarwal for the lead.

dirstate: use the 'nogc' decorator Now that we have a generic way to disable the gc, we use it. however, we have too use it in a baroque way. See inline comment for details.

util: add a 'nogc' decorator to disable the garbage collection Garbage collection behave pathologically when creating a lot of containers. As we do that more than once it become sensible to have a decorator for it. See inline documentation for details.

generate-working-copy-states: open() in binary mode when writing content This avoids changes to the hashes on Windows in test-merge-force.t like so: @@ -594,12 +594,12 @@ content2 M missing_content2_content3_content4-tracked - <<<<<<< local: 443153eb5b88 - test: local\r (esc) - content4 - ||||||| base - ======= - content2 - >>>>>>> other: 9b2ccd328a08 - test: remote\r (esc) + <<<<<<< local: 0447570f1af6 - test: local + content4 + ||||||| base + ======= + content2 + >>>>>>> other: 85100b8c675b - test: remote missing_content2_content3_content4-tracked.orig: content4

largefiles: don't use 'r' action for standin that doesn't exist When merging and the remote has turned a normal file into a largefile and the user chooses to keep the local largefile, we use the 'r' action for the remote largefile standin. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

largefiles: don't use 'r' action for normal file that doesn't exist When merging and the remote has turned a largefile into a normal file and the user chooses to keep the local largefile, we use the 'r' action for the remote normal file. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

rebase: handle revtodo as a special value when storing/restoring state Revtodo happens to share its value with nullrev, but this is an implementation details, so we move away from it. After this changeset one can successfully change the values for all the constants and the tests still pass, but doing so would require more refactoring if we want to avoid breaking backward compatibility on the state file.

rebase: add a 'revtodo' constant The state mapping is using '-1' to mark revisions that have not been rebased yet. We introduce and use a constant for that purpose. This will help emphasize the fact the value means something other than nullrev.

rebase: use '>= 0' to know is a revision was rebased The fact that the state for "not yet rebased" is -1 is an implementation details. So we change the comparisons to some semantically more correct.

patchbomb: introduce a 'patchbomb.confirm' option When set to true, this option will make patchbomb always ask for confirmation before sending the email. Confirmation is a powerful way to prevent stupid mistakes when the sending patches. This should let me get rid of my global alias adding --confirm to hg email. I know that some people may get bitten when moving from a machine with confirm configured to a machine where it is not, but I think it is worth the risk.

patchbomb: add a 'patchbomb.intro' option This option allows the user to control the default behavior for including an introduction message. This avoids having to tirelessly skip the intro for people contributing to Mercurial. The three possibles values are: - always, - auto (default, current behavior), - never. I was thinking of ("true", "false", "") (empty value being auto) but I ruled it out as too confusing. This new config option reuses the pre-existing 'patchbomb' section.

patchbomb: extract 'getoutgoing' closure into its own function This continues my crusade against closure complication.

perf: add a perfloadmarkers command It is used to benchmark the obsstore initialization time.

rebase: move base calculation from rebasenode() to defineparents() We want to collect all calculation in one place.

largefiles: remove redundant checks for false modify/delete conflicts In a34a99181f36 (largefiles: don't show largefile/normal prompts if one side is unchanged, 2014-12-01), overridecalculateupdates() started checking for false modify/delete conflicts in large files and their standins. Then, in the very next changeset, 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01), calculateupdates() itself started checking for false modify/delete conflicts in all files. Since "large files and their standins" is a subset of "all files", we can now drop the checks in overridecalculateupdates().

merge: make 'keep' message more descriptive Most merge action messages don't describe the action itself, they describe the reason the action was taken. The only exeption is the 'k' action, for which the message is just "keep" and instead there is a code comment folling it that says "remote unchanged". Let's move that comment into the merge action message.

addremove: restore the relative path printing when files are named This fixes the previously mentioned issue with 3778884197f0, and undoes its corresponding test change. The test change demonstrates the correctness when a file is specified (i.e. the glob is required on Windows because relative paths use '\' and absolute paths use '/'). It is admittedly very subtle, but there will be a more robust test in the addremove -S v3 series.

match: introduce uipath() to properly style a file path Several methods print files relative to the repo root, unless files are named on the command line, in which case they are printed relative to cwd. Since the check relies on the 'pats' parameter, which needs to be replaced by a matcher when adding subrepo support, this logic gets folded into the matcher to tidy up the callers. Prior to 3778884197f0, this style decision was based off of whether or not the 'pats' list was empty. That change altered the check to test match.anypats() instead, in order to make paths printed consistent when -I/-X is specified. That however, changed the style when a file is given to the command. So now we test the pattern list to get the old behavior for files, as well as test -I/-X to get the consistency for patterns.

merge with stable

update: don't overwrite untracked ignored files on update When looking for untracked files that would conflict with a tracked file in the target revision (or the remote side of a merge), we explcitly exclude ignored files. The code was added in 7e30f5f2285f (merge: refactor unknown file conflict checking, 2012-02-09), but it seems like only unknown, not ignored, files were considered since the beginning of time. Although ignored files are mostly build outputs and backup files, we should still not overwrite them. Fix by simply removing the explicit check.

update: add tests for untracked local file We don't seem to have any tests for updating to another revision when there are untracked files on the local side that conflict with the those on the remote side, so let's add tests. This shows how we overwrite untracked ignored files when updating to a revision that tracks the file.

merge: don't overwrite conflicting file in locally renamed directory When the local side has renamed a directory from a/ to b/ and added a file b/c in it, and the remote side has added a file a/c, we end up overwriting the local file b/c with the contents of remote file a/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename get ('dg') action.

merge: don't ignore conflicting file in remote renamed directory When the remote side has renamed a directory from a/ to b/ and added a file b/c in it, and the local side has added a file a/c, we end up moving a/c to b/c without considering the remote version of b/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename ('dm') action.

merge: duplicate 'if f in copied' into each branch

merge: branch code into {n1 and n2, n1, n2} top-level cases There are three high-level cases that are of interest in manifestmerge(): 1) The file exists on both sides, 2) The file exists only on the local side, and 3) The file exists only on the remote side. Let's make this clearer in the code. The 'if f in copied' case will be broken up into the two applicable branches in the next patch.

update: remove redundant and misplaced merge tests The "nothing to merge" case is covered by test-merge-default.t. The "uncommitted changes" case is covered by test-merge1.t (and others). The "merge -f" case is covered by test-merge-force.t.

largefiles: use clearer debug messages in actions In overridecalculateupdates(), 'g' (get) actions may be converted into other actions. In most of these cases, it does not make sense to keep the action's message. For example, 'remote created' does not make sense for an 'r' (remove) action.

largefiles: don't clobber merge action message with user message The message in the action is used for debugging and should not be the same as the question presented to the user. Use a different variable for the user message, so the 'msg' variable already in scope does not get overwritten.

bookmarks: factor out repository lookup from writing bookmarks file This will allow the share extension to extend bookmarks functionality to share bookmarks between repositories.

parsers: ensure revlog index node tree is initialized before insertion Currently, the revlog index C implementation assumes its node tree will be initialized before a new element is inserted by revnum. For example, revlog.py executes 'self.index.insert(-1, e)' in _addrevision(). This is only safe because the node tree has been initialized by a "node in self.nodemap" check made in addrevision(). (For context, this was discovered while developing an experimental revlog mixin which stores "elided nodes" via a separate code path from _addrevision(); that new code path segfaults without this patch.)

revset: add 'only' to DoS-safe symbols There is no reason to disable it in hgweb because the same query can be written without using 'only()'.

revset: move 'only' so that functions are sorted alphabetically

tests: drop unnecessary glob that causes a warning on Windows This was triggered by 3778884197f0.

merge: add test with conflicting file and remote directory rename Changset 88629daa727b (merge: demonstrate that directory renames can lose local file content, 2014-12-02) should clearly have added the reverse version of the test: where the remote side renamed a directory, added a new file in that directory, and the local directory added a conflicting file in the source directory. Add such a test now, and also touch up the ones already added slightly (e.g. 'local' was a stupid value for content that can be on either side of a merge).

graft: use a real conflict for the tests One of the graft tests grafts a changeset that changes a file's content from 'a' to 'b' onto a branch that has changed the file's content from 'a', via 'b', and then back to 'a' again. To prepare for not considering this a file in need of merging, let's use 'c' as the file's new content to make sure it has to be considered conflicting. There's a second similar case further down where an ancestor is grafted. Make sure that is also considered a conflict.

add: use lexists so that broken symbolic links are added This restores the add behavior prior to d8cdd46f426d and matches the behavior of addremove.

rebase: avoid redundant repo[rev].rev() - just keep working in rev space

rebase: always store rev in state, also when collapsing

rebase: clarify naming of variables holding node hashes - don't call them rev

bookmarks: factor out bookmark file opening for easier extensibility

merge with stable

diff: explicitly honor all diffopts 'hg diff' should naturally honor all diffopts.

annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.

notify: explicitly honor all diffopts The notify output doesn't seem to be parseable anyway, what with the maxdiff config option. Plus it is designed mainly for servers where hopefully the admins are doing sensible things.

jsonchangeset: don't honor whitespace and format-changing diffopts JSON is meant to be parsed by computers, and format changes can break them.

transplant: don't honor whitespace and format-changing diffopts transplant uses diff purely internally.

hgk: don't honor whitespace and format-changing diffopts The diff-tree command is a plumbing command in Git, which means it is expected to have a stable output format. Ensure that our output is stable too.

patchbomb: don't honor whitespace and format-changing diffopts (BC) The whitespace diffopts break lossless transmission, and the format-changing ones make import harder. We expect parsers to be able to read git-style diffs, though.

merge: add more thorough tests for --force With generate-working-copy-states.py generalized to support arbitrarily many changesets, we can use it for generating test cases for merge: use one changeset each for base, remote and local. With the various working copy states, this is a total of 104 cases. The first candidate for additional testing is 'hg merge --force'. Even though the force option is deprecated, it is convenient for testing because it can be tested without first needing to revert any changes. Except for the lack of checking for uncommitted changes, it differs in only a few cases from unforced merge. The new tests cover all the cases in the existing test-merge-force.t, except for the unforced merge case, which is covered in several other files anyway, so nothing remains of the file after this patch.

merge: display modify/delete conflict prompts in sorted order The order is determined by manifest.diff(), which currently is not sorted. There are currently no tests for this, but we will soon add some that would be flaky without this patch.

generate-working-copy-states: accept depth arguments on command line Add a parameter to generate-working-copy-states.py that indicates how many changesets are wanted. This number impacts all the subcommands. The current 'filelist' subcommand becomes 'filelist 2', the current 'base' and 'parent' subcommands become 'state 2 1' and 'state 2 2' respectively, while 'wc' becomes 'state 2 wc'. See earlier patch for motivation.

generate-working-copy-states: generalize for depth The script can currently generate filenames and contents for exactly two changesets plus the working copy. For some tests (e.g. of plain dirstate status), only one changeset is needed, while for others (e.g. merge), three changesets are needed. Let's prepare for such tests by generalizing the code for any number of changesets.

generate-working-copy-states: make 'tracked' a string The only places the 'tracked' variable is only used is where it's converted to a string, so let's simplify by using the string value directly.

merge: demonstrate that directory renames can lose local file content When a directory has been renamed on the local branch and a file has been added in the old location on a remote branch, we move that new file to the new location. Unfortunately, if there is already a file there, we overwrite it with the contents from the remote branch. For untracked local files, we should probably abort, and for tracked local files, we should merge the contents. To start with, let's add a test to demonstrate the breakage. Also note that while files merged in from a remote branch are normally (and unintuitively) reported as modified, these files are reported as added.

mq: drop updateheader - inserthgheader and insertplainheader is enough

mq: smarter handling of plain headers 6333412245ec and 5ccced6eab0b fixed issue4453 with a simple insertplainheader function that fixed the regression but didn't make the implementation more stable. Now we introduce plain header handling similar to how we handle hg patches. The whole header is scanned for fields to update while determining the best position for inserting the field if it is missing. It also makes sure there is an empty line between headers and body.

largefiles: drop the override for 'fetch' The fetch extension has been calling cmdutil.bailifchanged() since a014fdc97154, so this is redundant. Add test coverage to prevent regression. It doesn't look like there is any testing for fetch with largefiles.

rebase: ignore negative state when updating back to original wc parent The state mapping also contains some magic negative values (detached parent, ignored revision). Blindly reading the state thus lead to unfortunate usage of the negative value as an update destination. We now filter them out. We do a minor alteration of the test to catch this.

bundle2-push: provide transaction to reply unbundler This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. This diff adds an experimental config option "bundle2.pushback" which provides a transaction to the reply unbundler during a push operation. This behavior is opt-in because of potential security issues: the response can contain any part type that has a handler defined, allowing the server to make arbitrary changes to the local repository.

bundle2.processbundle: let callers request default behavior This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. The default transaction getter for processbundle is a private function that raises an exception; this diff lets calling code pass None as the transaction getter to explicitly request this default behavior. The next diff will check a config option to determine whether to provide a transaction to the reply bundle processor. If one shouldn't be provided, the code needs a way to specify that the default behavior should be used.

push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.

pull: extract transaction logic into separate object This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Aside from the transaction logic, the pulloperation class is used primarily as a logic-free data structure for storing state information. This diff extracts the transaction logic into its own class that can be shared with push operations.

record: don't honor format-changing diffopts (issue4459) record does support the whitespace options, so honor those. In upcoming patches we'll replace all the other uses.

patch.difffeatureopts: add a feature for format-changing diffopts Typically, callers that are interested in this set are also interested in the others. We still add it as a first-class flag for convenience.

patch.difffeatureopts: add a feature for whitespace diffopts These aren't exactly format-breaking features -- just ones for which patches applied to a repo will produce incorrect commits, In any case, some commands like record and annotate only care about this feature.

patch.difffeatureopts: add a feature for diff.git This deserves to be its own feature -- indeed, this is often the only feature several commands care about.

patch: rename diffopts to diffallopts This is meant to be used when all features are explicitly requested. diffopts is kept as an alias for now -- it will go away soon.

patch: add a new function to initialize diffopts by feature Not all callers are interested in all diffopts -- for example, commands like record (which use diff internally) break when diffopts like noprefix are enabled. This function will allow us to add flags that callers can use to enable only the features they're interested in.

patch.diffopts: use a dict for initialization In upcoming patches we'll conditionally add to and remove from the dict.

largefiles: split the creation of a normal matcher out of its install method Refactoring addremove to support subrepos will need the ability to keep passing the same matcher and narrowing it, instead of monkey patching scmutil's matcher.

addremove: print relative paths when called with -I/-X (BC) For "hg addremove 'glob:*.py'", we print any paths added or removed as relative to the current directory, but when "hg addremove -I 'glob:*.py'" is used, we use the absolute path (relative from the repo root). It seems like they should be the same, so change it so we use relative paths in both cases. Continue to use absolute paths when no patterns are given.

merge with stable

merge with i18n

i18n-pt_BR: synchronized with 3bd577a3283e

i18n-ja: synchronized with edf29f9c15f0

Added signature for changeset 902554884335

Added tag 3.2.2 for changeset 902554884335

merge: before cd/dc prompt, check that changed side really changed Before, merging would in some cases ask "wrong" questions about "changed/deleted" conflicts ... and even do it before the resolve phase where they can be postponed, re"resolved" or answered in bulk operations. Instead, check that the content of the changed file really did change. Reading and comparing file content is expensive and should be avoided before the resolve phase. Prompting the user is however even more expensive. Checking the content here is thus better. The 'f in ancestors[0]' should not be necessary but is included to be extra safe.

largefiles: don't show largefile/normal prompts if one side is unchanged

tests: add test-issue3084.t cases for 'changed but same' as for 'unchanged' Use suffix -same for cases where file changed but content is the same - that is the case where manifestmerge doesn't detect that a file is unchanged. (The suffix -id is already used for cases where the file didn't change - that is the trivial case where manifestmerge detects that the file is unchanged.) These new tests are good but the results are bad. There shouldn't be any merge conflicts or prompts when one side didn't change.

tests: clean-up of largefiles tests in test-issue3084.t Prepare for adding more test cases to the systematic testing, moving the test from ac3b3a2d976d to another section.

pushkey: gracefully handle prepushkey hook failure (issue4455) This allow to gracefully report the failure of the bookmark push and carry on. Before this change set. Local push would plain quit and wireprotocol would failed in various ungraceful way.

hook: raise a more specialized HookAbort exception when a hook fails We need to gracefully handle some aborts for pushkey, especially because it leads to a user-facing crash over the wireprotocols. So we need a more specialized exception to catch.

help: fix typo in files help

revset: fix first and last for generatorset (issue4465) The code was just plain wrong.

mq: fix update of headers that occur in the "wrong" order Mq tried to insert headers in the right order. Sometimes it would stop searching before checking all headers and it could thus duplicate a header instead of replacing it.

subrepo: add status support for ignored files in git subrepos Retrieving the status of a git subrepo did not show ignored files. Using 'git ls-files', we can retrieve these files and display the correct status.

manifest: document the extra letter in working copy manifest node As the second developer to get confused by this in November, I'm adding some documentation for the next poor soul.

hgweb: send proper HTTP response after uncaught exception This patch fixes a bug where hgweb would send an incomplete HTTP response. If an uncaught exception is raised when hgweb is processing a request, hgweb attempts to send a generic error response and log that exception. The server defaults to chunked transfer coding. If an uncaught exception occurred, it was sending the error response string / chunk properly. However, RFC 7230 Section 4.1 mandates a 0 size last chunk be sent to indicate end of the entity body. hgweb was failing to send this last chunk. As a result, properly written HTTP clients would assume more data was coming and they would likely time out waiting for another chunk to arrive. Mercurial's own test harness was paving over the improper HTTP behavior by not attempting to read the response body if the status code was 500. This incorrect workaround was added in ba6577a19656 and has been removed with this patch.

tests: test rebase of merge of target ancestors

tests: test coverage of parent order with p1 outside rebase set

tests: add test for rebasing merges with ancestors of the rebase destination This shows sub-optimal behaviour. The user gets a merge prompt that it is very hard to justify.

merge: 0 is a valid ancestor different from None Most internal functions can take either a hash or an integer. Merge did however not handle 0 as revision 0. Now it does.

merge with stable