scmutil: pass a matcher to scmutil.addremove() instead of a list of patterns This will make it easier to support subrepository operations.

tests: fix a typo in test-walkrepos.py

merge: extract _resolvetrivial() function We would eventually like to move the resolution of modify/delete and delete/modify conflicts to the resolve phase. However, we don't want to move the checks for identical content that were added in 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01). Let's instead move these out to a new _resolvetrivial() function that processes the actions from manifestmerge() and replaces any false cd/dc conflicts. The function will also provide a natural place for us to later add code for resolving false 'm' conflicts.

largefiles: start by finding files of interest Instead of iterating over 'g' action, first find the set of all files that are largefiles in p1. Then iterate over these files. This prepares for considering actions other than 'g'.

largefiles: rewrite merge code using dictionary with entry per file In overridecalculateupdates(), we currently only deal with conflicts that result in a 'g' action for either the largefile or a standin. We will soon want to deal cases with 'cd' and 'dc' actions here. It will be easier to reason about such cases if we rewrite it using a dict from filename to action. A side-effect of this change is that the output can only have one action per file (which should be a good change). Before this change, when one of the tests in test-issue3084 received this input (the 'a' in the input was a result of 'cd' conflict resolved in favor of the modified file): 'g': [('.hglf/f', ('',), 'remote created')], 'a': [('f', None, 'prompt keep')], and the user chose to keep the local largefile, it produced this output: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], 'a': [('f', None, 'prompt keep')], Although 'a' actions are processed after 'r' actions by recordupdates(), it still worked because 'a' actions have no effect on merges (only on updates). After this change, the output is: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], Similarly, there are several tests in test-largefiles-update that get inputs like: 'a': [('.hglf/large2', None, 'prompt keep')], 'g': [('large2', ('',), 'remote created')], and when the user chooses to keep the local largefile, they produce this output: 'a': [('.hglf/large2', None, 'prompt keep'), ('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')], In this case, it was not a merge but an update, so the 'a' action does have an effect. However, since dirstate.add() is idempotent, it still has no obserable effect. After this change, the output is: 'a': [('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')],

largefiles: put same 'action' object back in 'newglist' The items we put in 'newglist' are always the same as what we found in actions['g'], so let's just put the same item into the list instead of creating a new one.

largefiles: don't unnecessarily sort merge action lists The action lists returned from calculateupdates() (in merge.py) are not required to be sorted. In fact, since they result from iteration over the unordered manifest, they are unlikely to be sorted. Moreover, some of the lists are appended to after they are returned from manifestmerge(). The lists are instead sorted in applyupdates(). Therefore, let's not sort the lists generated in largefiles' overridecalculateupdates().

merge: don't treat 'diverge' and 'renamedelete' like actions See earlier patch for motivation.

merge: move dr/rd warning messages out of applyupdates() As preparation for making 'dr' and 'rd' actions no longer actions, move the reporting from applyupdates() to its caller update(). This way we won't have to pass additonal arguments to applyupdates() when they are no longer actions. Also, the warnings are equally unrelated to applyupdates() as they are to recordupdates(), as they don't result in any changes to either the working copy or the dirstate. See earlier patch for additional motivation.

merge: don't report progress for dr/rd actions It is easier to reason about certain algorithms in terms of a file->action mapping than the current action->list-of-files. Bid merge is already written this way (but with a list of actions per file), and largefiles' overridecalculateupdates() will also benefit. However, that requires us to have at most one action per file. That requirement is currently violated by 'dr' (divergent rename) and 'rd' (rename and delete) actions, which can exist for the same file as some other action. These actions are only used for displaying warnings to the user; they don't change anything in the working copy or the dirstate. In this way, they are similar to the 'k' (keep) action. However, they are even less action-like than 'k' is: 'k' at least describes what to do with the file ("do nothing"), while 'dr' and 'rd' or only annotations for files for which there may exist other, "real" actions. As a first step towards separating these acitons out, stop including them in the progress output, just like we already exclude the 'k' action.

subrepo: add partial diff support for git subrepos So far, git subrepositories were silently ignored for diffs. This patch adds support for git subrepositories, with the remark that --include and --exclude are not supported. If --include or --exclude are used, the subrepo is ignored.

subrepo: extend git version check to 3 digits This allows more flexibility when a version check is required. Some git features are introduced in a version where only the 3rd digit changes.

subrepo: move git version check into a separate method This allows checking the git version in other methods, instead of only being able to check if the version is ok or not.

rebase: show a note for updated mq patches It deserves more than a debug message. Show a note like: updating mq patch p0.patch to 5:9ecc820b1737 The message could also refer to "qrefresh" instead. Same same.

rebase: improve debug messages while processing the list of rebases

rebase: show warning when rebase creates no changes to commit Similar to graft: note: rebase of 6:eea13746799a created no changes to commit

rebase: show more useful status information while rebasing Show status messages while rebasing, similar to what graft do: rebasing 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: make 'saved backup' globbing less narrow in rebase tests Globbing the hash made it harder to maintain tests with run-tests -i when it was so far by the generated test output. The hashes are stable and we just need to add a (glob).

tests: stabilise mq rebase tests Prepare for including hashes in output ... and less globbing make the tests easier to update.

test-graft: use strip extension instead of mq extension It only needs strip, no reason to load all of mq.

transaction: remove the 'onabort' mechanism It has no known users. If someones needs similar functionality, a new 'addabort' method similar to 'addfinalize' should be added.

transaction: remove the redundant 'onclose' mechanism It is superseded by the 'addfinalize' function and all its user have been migrated.

fncache: document the fact fncache is outdate at hook run time Using 'addfinalize' to generate 'fncache' means that no pending version of the file will be generated for the hooks. We would have to use the 'addfilegenerator' method to get such result. However the 'fncachevfs' (who decide that a write is necessary) have no access to the transaction to register such file generation at add time. Having the transaction accessible to the 'vfs' is too much trouble for no benefit. This outdated 'fncache' file at hook time is not expected to be an issue. The previous move from 'onclose' to 'addfinalize' had no impact on this timing. I'm documenting it now because I looked at it.

fncache: drop dedicated 'onclose' function in favor of 'tr.addfinalize' Now that we have a shiny generic mechanism, we can use it.

merge with stable

graft: drop cset description from empty commit message This is either already redundant in the output or too verbose in quiet mode.

graft: show hashes in user-facing messages Graft was in various places just showing the revision number in status messges. Instead, also show the stable and more useful short hash.

graft: give helpful warning for empty grafts It was just showing a status message with the internal revision number. Instead, show a warning like note: graft of 27:3aaa8b6725f0 "28" created no changes to commit (message tweaked in-flight by mpm)

graft: show more useful status information while grafting Show status messages with first line of commit description and names, like grafting 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: test coverage for empty graft A future change will add a warning to the quiet rebase.

largefiles: drop the unfiltered repo usage in overridepurge() Now that repoview supports replacing methods, we don't need this hack.

repoview: allow methods on the proxy class to be replaced It doesn't seem to be a common idiom for repo instances, but the status() method is replaced in largefiles' purge() override. Since __setattr__ is implemented in repoview to setattr() on the unfiltered repo, the replacement method wouldn't get called unless it was invoked with the unfiltered repo, because the filtered repo remains unchanged. Since this doesn't seem to be commonly used, I didn't bother to filter out methods that perhaps shouldn't be replaced, such as changelog().

log: fix log revset instability The log/graphlog revset was not producing stable results since it was iterating over a dict. Now we sort before iterating to guarantee a fixed order. This fixes some potential flakiness in the tests.

log: fix log -f slow path to actually follow history The revset created when -f was used with a slow path (for patterns and directories) did not actually contain any logic to enforce follow. Instead it was depending on the passed in subset to already be limited (which was limited to :. but not ::.). This fixes it by adding a '& ::.' to any -f log revset. hg log -f <file> is still broken, in that it can return results that aren't actually ancestors of the current file, but fixing that has major perf implications, so we'll deal with it later.

obsstore: cache size computation for fm1 node We have two different types of node type (sha1 and sha256, only sha1 is used now) and therefor different sizes for them. We now compute the value once instead of redoing the computation every loop. This has no visible performance impact.

obsstore: prefetch struct.calcsize This function is widely used and worth but be at module level. No specific performance boost is visible, but this is more consistent.

obsstore: disable garbage collection during initialization (issue4456) Python garbage collection is triggered by container creation. So code that creates a lot of tuples tends to trigger GC a lot. We disable the gc during obsolescence marker parsing and associated initialization. This provides an interesting speedup (25%). Load marker function on my 58758 markers repo: before: 0.468247 seconds after: 0.344362 seconds The benefit is a bit less visible overall. With python2.6 on my system I see: after: 0.60 before: 0.53 The difference is probably explained by the delaying of a costly GC. (but there is still a win). Marking involved tuples, lists and dicts as ignorable by the garbage collector should give us more benefit. But this is another adventure. Thanks goes to Siddharth Agarwal for the lead.

dirstate: use the 'nogc' decorator Now that we have a generic way to disable the gc, we use it. however, we have too use it in a baroque way. See inline comment for details.

util: add a 'nogc' decorator to disable the garbage collection Garbage collection behave pathologically when creating a lot of containers. As we do that more than once it become sensible to have a decorator for it. See inline documentation for details.

generate-working-copy-states: open() in binary mode when writing content This avoids changes to the hashes on Windows in test-merge-force.t like so: @@ -594,12 +594,12 @@ content2 M missing_content2_content3_content4-tracked - <<<<<<< local: 443153eb5b88 - test: local\r (esc) - content4 - ||||||| base - ======= - content2 - >>>>>>> other: 9b2ccd328a08 - test: remote\r (esc) + <<<<<<< local: 0447570f1af6 - test: local + content4 + ||||||| base + ======= + content2 + >>>>>>> other: 85100b8c675b - test: remote missing_content2_content3_content4-tracked.orig: content4

largefiles: don't use 'r' action for standin that doesn't exist When merging and the remote has turned a normal file into a largefile and the user chooses to keep the local largefile, we use the 'r' action for the remote largefile standin. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

largefiles: don't use 'r' action for normal file that doesn't exist When merging and the remote has turned a largefile into a normal file and the user chooses to keep the local largefile, we use the 'r' action for the remote normal file. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

rebase: handle revtodo as a special value when storing/restoring state Revtodo happens to share its value with nullrev, but this is an implementation details, so we move away from it. After this changeset one can successfully change the values for all the constants and the tests still pass, but doing so would require more refactoring if we want to avoid breaking backward compatibility on the state file.

rebase: add a 'revtodo' constant The state mapping is using '-1' to mark revisions that have not been rebased yet. We introduce and use a constant for that purpose. This will help emphasize the fact the value means something other than nullrev.

rebase: use '>= 0' to know is a revision was rebased The fact that the state for "not yet rebased" is -1 is an implementation details. So we change the comparisons to some semantically more correct.

patchbomb: introduce a 'patchbomb.confirm' option When set to true, this option will make patchbomb always ask for confirmation before sending the email. Confirmation is a powerful way to prevent stupid mistakes when the sending patches. This should let me get rid of my global alias adding --confirm to hg email. I know that some people may get bitten when moving from a machine with confirm configured to a machine where it is not, but I think it is worth the risk.

patchbomb: add a 'patchbomb.intro' option This option allows the user to control the default behavior for including an introduction message. This avoids having to tirelessly skip the intro for people contributing to Mercurial. The three possibles values are: - always, - auto (default, current behavior), - never. I was thinking of ("true", "false", "") (empty value being auto) but I ruled it out as too confusing. This new config option reuses the pre-existing 'patchbomb' section.

patchbomb: extract 'getoutgoing' closure into its own function This continues my crusade against closure complication.

perf: add a perfloadmarkers command It is used to benchmark the obsstore initialization time.

rebase: move base calculation from rebasenode() to defineparents() We want to collect all calculation in one place.

largefiles: remove redundant checks for false modify/delete conflicts In a34a99181f36 (largefiles: don't show largefile/normal prompts if one side is unchanged, 2014-12-01), overridecalculateupdates() started checking for false modify/delete conflicts in large files and their standins. Then, in the very next changeset, 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01), calculateupdates() itself started checking for false modify/delete conflicts in all files. Since "large files and their standins" is a subset of "all files", we can now drop the checks in overridecalculateupdates().

merge: make 'keep' message more descriptive Most merge action messages don't describe the action itself, they describe the reason the action was taken. The only exeption is the 'k' action, for which the message is just "keep" and instead there is a code comment folling it that says "remote unchanged". Let's move that comment into the merge action message.

addremove: restore the relative path printing when files are named This fixes the previously mentioned issue with 3778884197f0, and undoes its corresponding test change. The test change demonstrates the correctness when a file is specified (i.e. the glob is required on Windows because relative paths use '\' and absolute paths use '/'). It is admittedly very subtle, but there will be a more robust test in the addremove -S v3 series.

match: introduce uipath() to properly style a file path Several methods print files relative to the repo root, unless files are named on the command line, in which case they are printed relative to cwd. Since the check relies on the 'pats' parameter, which needs to be replaced by a matcher when adding subrepo support, this logic gets folded into the matcher to tidy up the callers. Prior to 3778884197f0, this style decision was based off of whether or not the 'pats' list was empty. That change altered the check to test match.anypats() instead, in order to make paths printed consistent when -I/-X is specified. That however, changed the style when a file is given to the command. So now we test the pattern list to get the old behavior for files, as well as test -I/-X to get the consistency for patterns.

merge with stable

update: don't overwrite untracked ignored files on update When looking for untracked files that would conflict with a tracked file in the target revision (or the remote side of a merge), we explcitly exclude ignored files. The code was added in 7e30f5f2285f (merge: refactor unknown file conflict checking, 2012-02-09), but it seems like only unknown, not ignored, files were considered since the beginning of time. Although ignored files are mostly build outputs and backup files, we should still not overwrite them. Fix by simply removing the explicit check.

update: add tests for untracked local file We don't seem to have any tests for updating to another revision when there are untracked files on the local side that conflict with the those on the remote side, so let's add tests. This shows how we overwrite untracked ignored files when updating to a revision that tracks the file.

merge: don't overwrite conflicting file in locally renamed directory When the local side has renamed a directory from a/ to b/ and added a file b/c in it, and the remote side has added a file a/c, we end up overwriting the local file b/c with the contents of remote file a/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename get ('dg') action.

merge: don't ignore conflicting file in remote renamed directory When the remote side has renamed a directory from a/ to b/ and added a file b/c in it, and the local side has added a file a/c, we end up moving a/c to b/c without considering the remote version of b/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename ('dm') action.

merge: duplicate 'if f in copied' into each branch

merge: branch code into {n1 and n2, n1, n2} top-level cases There are three high-level cases that are of interest in manifestmerge(): 1) The file exists on both sides, 2) The file exists only on the local side, and 3) The file exists only on the remote side. Let's make this clearer in the code. The 'if f in copied' case will be broken up into the two applicable branches in the next patch.

update: remove redundant and misplaced merge tests The "nothing to merge" case is covered by test-merge-default.t. The "uncommitted changes" case is covered by test-merge1.t (and others). The "merge -f" case is covered by test-merge-force.t.

largefiles: use clearer debug messages in actions In overridecalculateupdates(), 'g' (get) actions may be converted into other actions. In most of these cases, it does not make sense to keep the action's message. For example, 'remote created' does not make sense for an 'r' (remove) action.

largefiles: don't clobber merge action message with user message The message in the action is used for debugging and should not be the same as the question presented to the user. Use a different variable for the user message, so the 'msg' variable already in scope does not get overwritten.

bookmarks: factor out repository lookup from writing bookmarks file This will allow the share extension to extend bookmarks functionality to share bookmarks between repositories.

parsers: ensure revlog index node tree is initialized before insertion Currently, the revlog index C implementation assumes its node tree will be initialized before a new element is inserted by revnum. For example, revlog.py executes 'self.index.insert(-1, e)' in _addrevision(). This is only safe because the node tree has been initialized by a "node in self.nodemap" check made in addrevision(). (For context, this was discovered while developing an experimental revlog mixin which stores "elided nodes" via a separate code path from _addrevision(); that new code path segfaults without this patch.)

revset: add 'only' to DoS-safe symbols There is no reason to disable it in hgweb because the same query can be written without using 'only()'.

revset: move 'only' so that functions are sorted alphabetically

tests: drop unnecessary glob that causes a warning on Windows This was triggered by 3778884197f0.

merge: add test with conflicting file and remote directory rename Changset 88629daa727b (merge: demonstrate that directory renames can lose local file content, 2014-12-02) should clearly have added the reverse version of the test: where the remote side renamed a directory, added a new file in that directory, and the local directory added a conflicting file in the source directory. Add such a test now, and also touch up the ones already added slightly (e.g. 'local' was a stupid value for content that can be on either side of a merge).

graft: use a real conflict for the tests One of the graft tests grafts a changeset that changes a file's content from 'a' to 'b' onto a branch that has changed the file's content from 'a', via 'b', and then back to 'a' again. To prepare for not considering this a file in need of merging, let's use 'c' as the file's new content to make sure it has to be considered conflicting. There's a second similar case further down where an ancestor is grafted. Make sure that is also considered a conflict.

add: use lexists so that broken symbolic links are added This restores the add behavior prior to d8cdd46f426d and matches the behavior of addremove.

rebase: avoid redundant repo[rev].rev() - just keep working in rev space

rebase: always store rev in state, also when collapsing

rebase: clarify naming of variables holding node hashes - don't call them rev

bookmarks: factor out bookmark file opening for easier extensibility

merge with stable

diff: explicitly honor all diffopts 'hg diff' should naturally honor all diffopts.

annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.

notify: explicitly honor all diffopts The notify output doesn't seem to be parseable anyway, what with the maxdiff config option. Plus it is designed mainly for servers where hopefully the admins are doing sensible things.

jsonchangeset: don't honor whitespace and format-changing diffopts JSON is meant to be parsed by computers, and format changes can break them.

transplant: don't honor whitespace and format-changing diffopts transplant uses diff purely internally.

hgk: don't honor whitespace and format-changing diffopts The diff-tree command is a plumbing command in Git, which means it is expected to have a stable output format. Ensure that our output is stable too.

patchbomb: don't honor whitespace and format-changing diffopts (BC) The whitespace diffopts break lossless transmission, and the format-changing ones make import harder. We expect parsers to be able to read git-style diffs, though.

merge: add more thorough tests for --force With generate-working-copy-states.py generalized to support arbitrarily many changesets, we can use it for generating test cases for merge: use one changeset each for base, remote and local. With the various working copy states, this is a total of 104 cases. The first candidate for additional testing is 'hg merge --force'. Even though the force option is deprecated, it is convenient for testing because it can be tested without first needing to revert any changes. Except for the lack of checking for uncommitted changes, it differs in only a few cases from unforced merge. The new tests cover all the cases in the existing test-merge-force.t, except for the unforced merge case, which is covered in several other files anyway, so nothing remains of the file after this patch.

merge: display modify/delete conflict prompts in sorted order The order is determined by manifest.diff(), which currently is not sorted. There are currently no tests for this, but we will soon add some that would be flaky without this patch.

generate-working-copy-states: accept depth arguments on command line Add a parameter to generate-working-copy-states.py that indicates how many changesets are wanted. This number impacts all the subcommands. The current 'filelist' subcommand becomes 'filelist 2', the current 'base' and 'parent' subcommands become 'state 2 1' and 'state 2 2' respectively, while 'wc' becomes 'state 2 wc'. See earlier patch for motivation.

generate-working-copy-states: generalize for depth The script can currently generate filenames and contents for exactly two changesets plus the working copy. For some tests (e.g. of plain dirstate status), only one changeset is needed, while for others (e.g. merge), three changesets are needed. Let's prepare for such tests by generalizing the code for any number of changesets.

generate-working-copy-states: make 'tracked' a string The only places the 'tracked' variable is only used is where it's converted to a string, so let's simplify by using the string value directly.

merge: demonstrate that directory renames can lose local file content When a directory has been renamed on the local branch and a file has been added in the old location on a remote branch, we move that new file to the new location. Unfortunately, if there is already a file there, we overwrite it with the contents from the remote branch. For untracked local files, we should probably abort, and for tracked local files, we should merge the contents. To start with, let's add a test to demonstrate the breakage. Also note that while files merged in from a remote branch are normally (and unintuitively) reported as modified, these files are reported as added.

mq: drop updateheader - inserthgheader and insertplainheader is enough

mq: smarter handling of plain headers 6333412245ec and 5ccced6eab0b fixed issue4453 with a simple insertplainheader function that fixed the regression but didn't make the implementation more stable. Now we introduce plain header handling similar to how we handle hg patches. The whole header is scanned for fields to update while determining the best position for inserting the field if it is missing. It also makes sure there is an empty line between headers and body.

largefiles: drop the override for 'fetch' The fetch extension has been calling cmdutil.bailifchanged() since a014fdc97154, so this is redundant. Add test coverage to prevent regression. It doesn't look like there is any testing for fetch with largefiles.

rebase: ignore negative state when updating back to original wc parent The state mapping also contains some magic negative values (detached parent, ignored revision). Blindly reading the state thus lead to unfortunate usage of the negative value as an update destination. We now filter them out. We do a minor alteration of the test to catch this.

bundle2-push: provide transaction to reply unbundler This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. This diff adds an experimental config option "bundle2.pushback" which provides a transaction to the reply unbundler during a push operation. This behavior is opt-in because of potential security issues: the response can contain any part type that has a handler defined, allowing the server to make arbitrary changes to the local repository.

bundle2.processbundle: let callers request default behavior This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. The default transaction getter for processbundle is a private function that raises an exception; this diff lets calling code pass None as the transaction getter to explicitly request this default behavior. The next diff will check a config option to determine whether to provide a transaction to the reply bundle processor. If one shouldn't be provided, the code needs a way to specify that the default behavior should be used.