largefiles: get function to write status messages via "getstatuswriter()" This patch makes "updatelfiles()" get appropriate function to write largefiles specific status messages via "getstatuswriter()". This patch introduces None as "print messages if needed", because True (forcibly writing) and False (forcibly ignoring) are already used for "printmessage" of "updatelfiles". Subsequent patch will move "avoid printing messages only while automated committing" decision from caller of "updatelfiles()" into "getstatuswriter()".

largefiles: introduce "_lfstatuswriters" to customize status reporting "lfutil.getstatuswriter" is the utility to get appropriate function to write largefiles specific status out from "repo._lfstatuswriters". This patch uses "stack" with an element instead of flag like "_isXXXXing" or so, because: - the former works correctly even when customizations are nested, and - ensuring at least one element can ignore empty check

largefiles: update standins only at the 1st commit of "hg rebase --continue" Before this patch, "hg rebase --continue" may record incorrect standins, because largefiles extension always avoid updating standins while rebasing, even though largefiles in the working directory may be modified manually at the 1st commit of "hg rebase --continue". But, on the other hand, updating standins should be avoided at subsequent commits for efficiency reason. To update standins only at the 1st commit of "hg rebase --continue", this patch introduces state-full callable object "automatedcommithook", which updates standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of resuming. Even after this patch, "repo._isrebasing = True" is still needed to avoid some status report while updating largefiles in "lfcommands.updatelfiles()". This is reason why this patch omits not "repo._isrebasing = True" in "overriderebase" but examination of "getattr(repo, "_isrebasing", False)" in "updatestandinsbymatch".

largefiles: introduce "_lfcommithooks" to abstract pre-committing procedures This changes allows to customize pre-committing procedures according to conditions. This patch uses "stack" with an element instead of flag like "_isXXXXing" or so, because: - the former works correctly even when customizations are nested, and - ensuring at least one element can ignore empty check

largefiles: factor out procedures to update standins for pre-committing This patch factors out procedures to update standins for pre-committing. This is one of preparations to avoid execution of such procedures according to invocation context. For example, resuming automated committing (e.g. "hg rebase --continue") should update standins at the 1st commit, because largefiles in the working directory may be modified manually. But on the other hand, it should avoid updating standins at subsequent committings for efficiency reason. For simplicity, this patch just moves procedures mechanically only with replacing below. - "self" => "repo" - "lfutil." => (none) - "orig" invocation => returning "match" Using "fstandin" instead "standin" as the name of local variable for the loop below is the only special care, because the latter shadows the same name function in "lfutil.py". [before] for standin in standins: lfile = lfutil.splitstandin(standin) if lfdirstate[lfile] != 'r': lfutil.updatestandin(self, standin) [after] for fstandin in standins: lfile = splitstandin(fstandin) if lfdirstate[lfile] != 'r': updatestandin(repo, fstandin)

largefiles: factor out procedures to update lfdirstate for post-committing Before this patch, procedures to update lfdirstate for post-committing are scattered in "lfilesrepo.commit". In the case of "hg commit" with patterns for target files ("Case 2"), lfdirstate is updated BEFORE real committing. This patch factors out procedures to update lfdirstate for post-committing into "lfutil.markcommitted", and makes it callable via "markcommitted" of the context passed to "lfilesrepo.commitctx". "markcommitted" of the context is called, only when it is committed successfully. Passing original "markcommitted" of the context is meaningless in this patch, but required in subsequent one to prepare something before invocation of it.

largefiles: remove meaningless code path for "hg pull --rebase" This patch removes "--rebase" specific code path for "hg pull" in "overridepull", because previous patch makes it meaningless: now, "rebase.rebase" ("orig" invocation in this patch) can update/commit largefiles safely without "repo._isrebasing = True". As a side effect of removing "rebase.rebase" invocation in "overridepull", this patch removes "nothing to rebase ..." message in "test-largefiles.t", which is shown only when rebase extension is enabled AFTER largefiles: before this patch: 1. "dispatch" invokes "pullrebase" of rebase as "hg pull" at first, because rebase wraps "hg pull" later 2. "pullrebase" invokes "overridepull" of largefiles as "orig", even though rebase assumes that "orig" is "pull" of commands 3. "overridepull" executes "pull" and "rebase" directly 3.1 "pull" pulls changesets and creates new head "X" 3.2 "rebase" rebases current working parent "Y" on "X" 4. "overridepull" returns to "pullrebase" 5. "pullrebase" tries to rebase, but there is nothing to be done, because "Y" is already rebased on "X". then, it shows "nothing to rebase ..." after this patch: 1. "dispatch" invokes "pullrebase" of rebase as "hg pull" 2. "pullrebase" invokes "overridepull" of largefiles as "orig" 3. "overridepull" executes "pull" as "orig" 4. "overridepull" returns to "pullrebase" 5. revision "Y" is not yet rebased, so "pullrebase" doesn't shows "nothing to rebase ..." As another side effect of removing "rebase.rebase" invocation, this patch fixes issue3861, which occurs only when rebase extension is enabled BEFORE largefiles: before this patch: 1. "dispatch" invokes "overridepull" of largefiles at first, because largefiles wrap "hg pull" later 2. "overridepull" executes "pull" and "rebase" explicitly 2.1 "pull" pulls changesets and creates new head "X" 2.2 "rebase" rebases current working parent, but fails because no revision is checked out in issue3861 case 3. "overridepull" returns to "dispatch" with exit code 1 returned from "rebase" at (2.2) 4. "hg pull" terminates with exit code 1 unexpectedly after this patch: 1. "dispatch" invokes "overridepull" of largefiles at first 2. "overridepull" invokes "pullrebase" of rebase as "orig" 3. "pullrebase" invokes "pull" as "orig" 4. "pullrebase" invokes "rebase", and it fails 5. "pullrebase" returns to "overridepull" with exit code 0 (because "pullrebase" ignores result of "pull" and "rebase") 6. "overridepull" returns to "dispatch" with exit code 0 returned from "rebase" at (5) 7. "hg pull" terminates with exit code 0

largefiles: wrap "rebase.rebase" for functions using it directly Before this patch, largefiles extension wraps only "rebase" in the command table by "extensions.wrapcommand". But there are some functions using "rebase.rebase" directly. Without special care for them, largefiles extension can't work correctly with such functions. In addition to it, "special care" often becomes complicated and awkward. For example: - "unshelve" can't get correct result of "rebase.rebase", because of lack of special care - special care for "hg pull --rebase" causes issue3861 This patch wraps "rebase.rebase" for functions using it directly. For simplicity, this patch keeps 'special care for "hg pull --rebase"'. It is removed in the subsequent patch.

changegroup: introduce cg2packer/unpacker cg2 supports generaldelta in changegroups, to be used in bundle2. Since generaldelta is handled directly in cg2, reordering is switched off by default.

push: send highest changegroup format supported by both side When using bundle2, we find the common subset of supported changegroup-packers and we pick the max of them. This allow to use generaldelta aware changegroups through bundle2.

getbundle: send highest changegroup format supported by both side When using bundle2, we find the common subset of supported changegroup-packers and we pick the max of them. This allow to use generaldelta aware changegroup through bundle2.

changegroup: allow use of different cg#packer in getchangegroupraw This will allow the use of general delta aware changegroup formats.

changegroup: introduce "raw" versions of some commands The commands getchangegroup, getlocalchangegroup and getsubset now each have a version ending in -raw. The raw versions return the chunk generator from the changegroup packer directly, without wrapping it in a chunkbuffer and unpacker. This avoids extra chunkbuffers in the bundle2 code path. Also, the raw versions can be extended to support alternative packers in the future, to be used from bundle2.

merge with stable

win32: remove Mercurial.ini file from Inno Setup installer (issue4435)

win32: fix win32 installers generation cd0c51c07e5f introduced a typo preventing a proper generation of the installers. Also remove ConcatenateFiles() function as it is no more required.

patchbomb: add label and color to the confirm output The current output is mostly a wall of text. This makes it hard to actually check something for people with lazy eyes. We use labels and colors to make it more joyful (and get the patch summaries to stand out). The colors have been arbitrarily choosen. They can be changed later if someone has a more scientific choice.

tests: write hgrc of more than two lines by using shell heredoc Here document should be readable than repeating echo commands.

perf: use a formatter for output We use a `formatter` object in the perf extensions. This allow the use of formatted output like json. To avoid adding logic to create a formatter and pass it around to the timer function in every command, we add a `gettimer` function in charge of returning a `timer` function as simple as before but embedding an appropriate formatter. This new `gettimer` function also return the formatter as it needs to be explicitly closed at the end of the command. example output: $ hg --config ui.formatjson=True perfvolatilesets visible obsolete [ { "comb": 0.02, "count": 126, "sys": 0.0, "title": "obsolete", "user": 0.02, "wall": 0.0199398994446 }, { "comb": 0.02, "count": 117, "sys": 0.0, "title": "visible", "user": 0.02, "wall": 0.0250301361084 } ]

bundle2: support a "version" argument in `changegroup` part When included, this mandatory parameter (mandatory == cannot be ignored) lets the part handler select the right cgunpacker class.

bundle2caps: advertise the available versions for changegroup packer This will let the bundle2 client and server detect what packer they should be using. This detection part is not done. I expect it to be done with the addition of the second packer (with generaldelta support).

changegroup: add a "packermap" dictionary to track different packer versions We only have "01" right now, but we should get general delta in soon. Bundle2 is expected to make use of this to advertise and select the right packer to use on both sides.

templater: don't overwrite the keyword mapping in runsymbol() (issue4362) This keyword remapping was introduced in e06e9fd2d99f as part of converting generator based iterators into list based iterators, mentioning "undesired behavior in template" when a generator is exhausted, but doesn't say what and introduces no tests. The problem with the remapping was that it corrupted the output for keywords like 'extras', 'file_copies' and 'file_copies_switch' in templates such as: $ hg log -r 142b5d5ec9cc --template "{file_copies % ' File: {file_copy}\n'}" File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) What was happening was that in the first call to runtemplate() inside runmap(), 'lm' mapped the keyword (e.g. file_copies) to the appropriate showxxx() method. On each subsequent call to runtemplate() in that loop however, the keyword was mapped to a list of the first item's pieces, e.g.: 'file_copy': ['mercurial/changelog.py', ' (', 'mercurial/hg.py', ')'] Therefore, the dict for the second and any subsequent items were not processed through the corresponding showxxx() method, and the first item's data was reused. The 'extras' keyword regressed in de7e6c489412, and 'file_copies' regressed in 0b241d7a8c62 for other reasons. The common thread of things fixed by this seems to be when a list of dicts are passed to the templatekw._hybrid class.

revset-matching: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet but it is less visible because the matching process itself is very expensive: revset) matching(100) before) wall 6.413281 comb 6.420000 user 5.910000 sys 0.510000 (best of 3) after) wall 6.173608 comb 6.170000 user 5.750000 sys 0.420000 (best of 3) However for some complex list, this provide a massive speedup revset) matching(parents(100)) before) wall 23.890740 comb 23.890000 user 23.450000 sys 0.440000 (best of 3) after) wall 6.382280 comb 6.390000 user 5.930000 sys 0.460000 (best of 3)

revset-parentspec: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) 100^1 before) wall 0.002694 comb 0.000000 user 0.000000 sys 0.000000 (best of 897) after) wall 0.000997 comb 0.000000 user 0.000000 sys 0.000000 (best of 2324) revset) parents(100)^1 before) wall 0.003832 comb 0.000000 user 0.000000 sys 0.000000 (best of 587) after) wall 0.001034 comb 0.000000 user 0.000000 sys 0.000000 (best of 2309) revset) (100^1)^1 before) wall 0.005616 comb 0.000000 user 0.000000 sys 0.000000 (best of 405) after) wall 0.001030 comb 0.000000 user 0.000000 sys 0.000000 (best of 2258)

revset-children: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) children(tip~100) before) wall 0.007469 comb 0.010000 user 0.010000 sys 0.000000 (best of 338) after) wall 0.003356 comb 0.000000 user 0.000000 sys 0.000000 (best of 755)

revset-ancestorspec: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) 100~5 before) wall 0.002712 comb 0.000000 user 0.000000 sys 0.000000 (best of 918) after) wall 0.000996 comb 0.000000 user 0.000000 sys 0.000000 (best of 2493) revset) parents(100)~5 before) wall 0.003812 comb 0.010000 user 0.010000 sys 0.000000 (best of 667) after) wall 0.001038 comb 0.000000 user 0.000000 sys 0.000000 (best of 2361) revset) (100~5)~5 before) wall 0.005614 comb 0.000000 user 0.000000 sys 0.000000 (best of 446) after) wall 0.001035 comb 0.000000 user 0.000000 sys 0.000000 (best of 2424)

revset-rangeset: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefit ofs the fullreposet: revset) 10:100 before) wall 0.002774 comb 0.000000 user 0.000000 sys 0.000000 (best of 797) after) wall 0.001977 comb 0.000000 user 0.000000 sys 0.000000 (best of 1244) revset) parents(10):parents(100) before) wall 0.005054 comb 0.000000 user 0.000000 sys 0.000000 (best of 481) after) wall 0.002060 comb 0.000000 user 0.000000 sys 0.000000 (best of 1056)

test-revert: make sure all 'tracked' files are really tracked When a file is missing in the 'parent' version and is tracked but missing in the working directory, which happens by the 'missing' or 'removed' types, and the 'clean' type in the working directory, the file does not exist in the working directory (unlike it would had the 'deleted' type been used). Thus, the *_missing_missing_tracked are not actually tracked and they end up testing the same state as *_missing_missing_untracked. To make them tracked, add a temporary file, just like we do for the delete case. For simplicity's sake, let's make sure the gen-revert-cases.py script always puts a file in the working directory, whether or not it's going to be deleted.

test-revert: sort by output filename again Future patches will change how the output of 'gen-revert-cases.py filelist' is generated, so now we want the order to depend on just the filename again.

test-revert: name files by state, not by state transition This is the main patch in a series. See motivation in earlier patch. In this patch, we actually change the names of the generated files. For example, the file that is currently called missing_clean becomes missing_missing_missing-tracked and it's clearer that it should be tracked. It turns out that since the state was not previously clear, it ended up testing an untracked state, which was the same as for missing_clean. We'll fix this in a later patch. Let's also change the content from (base,parent,wc) to (content1,content2,content3) to make them all the same length so they line up when displayed.

test-revert: temporarily sort by input states instead of output filename The next patch will change the names of the files produced by the script in test-revert. In order to reduce the size and increase the clarity of the next patch, make the order produced by the internal 'gen-revert-cases.py filelist' command independent of the filenames.

test-revert: put content, not keys, into 'combination' By putting the file content rather than keys in the 'combination' list, we restrict the knowledge of 'ctxcontent' and 'wccontent' to the loop generating the combinations. That will make it easier to replace the generation code.

test-revert: replace 'removed' in working copy with 'untracked-deleted' The 'wccontent' variable has eight different states, four of them tracked, and the other four untracked (at least when the file existed in the parent revision). Among these eight states, 'removed' sticks out by lacking the 'untracked-' prefix despite resulting in an untracked state. To make the symmetry clearer, and to prepare for future patches, rename 'removed' to 'untracked-deleted', which is exactly what it is. Note that, unlike 'remove', 'deleted' is configured in gen-revert-cases.py to have content in the working directory and that that content is instead expected to be removed in the test script. However, no changes are needed to the test script, since it already contains 'hg forget *untracked*' and 'rm *deleted*', which together have the same effect as 'hg remove'. See additional motivation in earlier patch.

test-revert: removing a missing file has no effect The tests for removed_deleted and removed_removed test the same state as removed_clean and removed_untracked-clean, respectively. Drop the duplicate tests. See additional motivation in earlier patch.

test-revert: reverting an addition is the same as removing The tests for added_revert and added_untracked-revert test the same state as added_deleted and added_removed, respectively. Drop the duplicate tests. See additional motivation in earlier patch.

test-revert: reverting no change means it's clean This is the first step in a series that aims to put the state, not the state transitions, in the filenames of the files generated by the gen-revert-cases.py script. The possible state of a file in a revision and in the working copy is only whether it exists and what its content is (the tests don't care check flags). In the dirstate, the only state is whether it's tracked or not. With the new naming, the file that is currently called modified_untracked-clean now becomes content1_content2_content2-untracked, for example. By putting these states in the filename, it becomes easier to see that we're not missing or duplicating any state, and to check that the state is what we think it is. For example, the file that is currently called missing_clean becomes missing_missing_missing-tracked and it's clearer that it should be tracked. Putting the content in the filename will also make the tests of file content (e.g. "cat ../content-parent.txt") very obvious. When we put the state in the filename, the filenames clearly need to be unique. However, it turns out that some states are currently tested multiple times. The 'revert' transition in the script means to take the content from the grandparent. If the parent is the same as the grandparent, there is no change compared to the parent, which is exactly what 'clean' means. Avoid testing the same state twice.

merge with stable

extdiff: drop the command alias without options example in the help text In the dropped example, the extension would look for 'vdiff.diffargs' in the configuration, and not finding it, would run kdiff3 without the configured options. That's not obvious to a new user who sees a kdiff3 configuration in the prepackaged mergetools.rc file, and sees that kdiff3 still runs. While it is conceivable that the user wants a kdiff3 command that runs without the preconfigured options, it is more likely what they want is this, which uses the canned options: [alias] vdiff = kdiff3 [extdiff] kdiff3 = We could mention alias here, but that seems like it belongs elswhere.

extdiff: allow a preconfigured merge-tool to be invoked There are three ways to configure an extdiff tool: 1) cmd.tool = (/path/to/exe optional) 2) tool = (path/to/exe optional) 3) tool = sometool someargs Previously, if no executable is specified in the first two forms, the named tool must be in $PATH, or the invocation fails. Since the [merge-tools] section already has the path to the diff executable, and/or the registry keys to find the executable on Windows, reuse that configuration for forms 1 and 2 instead of failing. We already fallback to [diff-tools] and then [merge-tools] for program arguments if they aren't specified in the [extdiff] section. Since this additional lookup only occurs if an executable is not on the $PATH for the named tool, this is backwards compatible. For now, we assume the user knows what he is doing if a path is provided. This change allows a configuration file like this (assuming beyondcompare3 is configured in merge-tools), instead of hardcoding system specific a path: [extdiff] beyondcompare3 =

extdiff: sort files when snapshotting This fixes output stability and is generally filesystem-performance-friendly.

filemerge: split the logic for finding an external tool to its own function This will be used by extdiff in an subsequent patch.

largefiles: simplify check for lack of path arguments Instead of checking for a partial merge by checking that the matches has no files and no patterns, check that it's not an always-matcher. Except for being shorter, it also catches the rare case of an exact-matcher with no files.

largefiles: shortcircuit status code also for non-matching patterns We currently shortcircuit the checking for large file standins if only patterns of type 'path' are given on the command line. That makes e.g. "hg st 'glob:foo/**'" unnecessarily slow when the only large files are in a sibling directory. Relax the check to be that it is not an always-matcher and that no large files match the patterns given on the command line. Note that before this change, only the latter of the following two would show the status of files in .hglf (since the -I makes match.anypats() true). After this change, they both display the status. This behavior doesn't seem correct, but it would be a separate change to explicitly filter out .hglf even in the shortcircuit case. hg st .hglf/$file hg st .hglf/$file -I .

largefiles: remove confusing 'or None' from predicate The match function that is overriden returns a boolean value, so adding 'or None' is both unnecessary and confusing.

largefiles: drop unnecessary setting of matcher._always In two very similar segments of code, an existing matcher is modified by changing its _files attribute through a map and a filter operation. Neither operation can cause an empty list to become non-empty, so a matcher that always matches can not stop always matching. Drop the setting of the attribute, so we don't unnecessarily prevent the fast paths to be taken where these matchers end up being used.

config: move mergetools configuration from contrib to default configuration The merge tool configuration is an essential part of a good initial user experience. 'make osx' installers and direct 'make' installation did not have merge tool configuration. Now they have. Note: The installer fixes for windows have been done blindly and might require additional changes.

config: introduce "built-in" default configuration settings in default.d This helps providing a more consistent user experience on all platforms and with all packaging. The exact location of default.d depends on how Mercurial is installed and whether it is 'frozen'. The exact location should never be relevant to users and is intentionally not explained in details in the documentation. It will however always be next to the help and templates files. Note that setting HGRCPATH also disables these defaults. I don't know if that should be considered a bug or a feature.

contrib: buildrpm checking of md5 checksums of downloaded Python and Docutils

contrib: update build defaults to latest Python and docutils versions Use Python 2.7.8 and Docutils 0.12 for Makefile.python and buildrpm (CentOS 5 rpms).

spelling: fixes from proofreading of spell checker issues

extdiff: quote user-supplied options passed to shell $ hg extdiff -p cmd -o "name <user@example.com>" resulted in a shell redirection error (due to the less-than sign), rather than passing the single option to cmd. This was due to options not being quoted for passing to the shell, via util.system(). Apply util.shellquote() to each of the user-specified options (-o) to the comparison program before they are concatenated and passed to util.system(). The requested external diff command (-p) and the files/directories being compared are already quoted correctly. The discussion at the time of changeset be98c5ce4022 correctly noted that this course of action breaks whitespace-separated options specified for external diff commands in the configuration. The lower part of the patch corrects this by lexing options read from the configuration file into separate options rather than reading them all into the first option. Update test to cover these conditions. Related changesets (reverse-chronological): - be98c5ce4022 (fix reverted to make configuration file options work) - 453097750fbf (issue fixed but without fix for configuration file)

serve: correct meta variable of --daemon-pipefds option It was changed to lock path at e22695b4472f.

tests: have dumbhttp.py use cmdutil.service() to wait for child to listen() Because the original dumbhttp.py exited without waiting for listen(), several tests could fail with "abort: error: Connection refused" if subsequent hg command is fast enough.

test-revert: remove obsolete comment about known misbehavior It seems like the last known misbehvior that the comment was referring to was dealt with in 2ff28e07d7d6 (revert: properly back up added files with local modification, 2014-08-31).

test-revert.t: fix wc check-code false positive

merge with stable

Added signature for changeset ced632394371

Added tag 3.2 for changeset ced632394371

setdiscovery: limit the size of all sample (issue4411) Further digging on this issue show that the limit on the sample size used in discovery never works for heads. Here is a quote from the code itself: desiredlen = size - len(always) if desiredlen <= 0: # This could be bad if there are very many heads, all unknown to the # server. We're counting on long request support here. The long request support never landed and evolution make the "very many heads, all unknown to the server" case quite common. We implement a simple and stupid hard limit of sample size for all query. This should prevent HTTP 414 error with the current state of the code.