test-status: add test for removed-and-untracked state (BC) In Mercurial 3.0, "hg status" can list the same file twice if it was removed but still exists in working directory, i.e. removed by "hg forget": $ hg status --rev 0 removed R removed ? removed But since 65cdc6bab91e, untracked state, "?", is no longer displayed in this example. I think the new behavior is correct since a file should have single state, but if it is a bug, this patch should be dropped.

bundle2: only use callable return as reply handler When a bundle2 parts generator returns a non callable value, it should not be used as a reply handler. The changegroup part generator is already having this case of behavior when there is no changegroup to push. This changeset prevent a crash for user of the experimentable bundle2 feature.

resolve: report no argument warning using a hint With this change resolve and revert produce consistent output when run with no arguments: $ hg resolve abort: no files or directories specified (use --all to remerge all files) $ hg revert abort: no files or directories specified (use --all to revert all files)

revset: optimize baseset.__sub__ (issue4313) dd716807fd23 regressed performance of baseset.__sub__ by introducing a lazyset. This patch restores that lost performance by eagerly evaluating baseset.__sub__ if the other set is a baseset. revsetbenchmark.py results impacted by this change: revset #6: roots(0::tip) 0) wall 2.923473 comb 2.920000 user 2.920000 sys 0.000000 (best of 4) 1) wall 0.077614 comb 0.080000 user 0.080000 sys 0.000000 (best of 100) revset #23: roots((0:tip)::) 0) wall 2.875178 comb 2.880000 user 2.880000 sys 0.000000 (best of 4) 1) wall 0.154519 comb 0.150000 user 0.150000 sys 0.000000 (best of 61) On the author's machine, this slowdown manifested during evaluation of 'roots(%ln::)' in phases.retractboundary after unbundling the Firefox repository. Using `time hg unbundle firefox.hg` as a benchmark: Before: 8:00 After: 4:28 Delta: -3:32 For reference, the subset and cs baseset instances impacted by this change were of lengths 193634 and 193627, respectively. Explicit test coverage of roots(%ln::), while similar to the existing roots(0::tip) benchmark, has been added.

memctx: substate needs to be {} instead of None Setting substate to None was an oversight in 7cfd94ec5d30 and this patch corrects it by setting substate to an empty dictionary which matches what subrepo code expects.

version: don't traceback if no extensions to list (issue4312)

merge with i18n

i18n-pt_BR: synchronized with 6c36dc6cd61a

largefiles: use "normallookup" on "lfdirstate" while reverting Before this patch, largefiles gotten from revisions other than the parent of the working directory at "hg revert" become "clean" unexpectedly in steps below: 1. "repo.status()" is invoked (for status check before reverting) 1-1 "dirstate" entry for standinfile SF is "normal"-ed (1-2 "lfdirstate" entry of largefile LF (for SF) is "normal"-ed) 2. "cmdutil.revert()" is invoked 2-1 standinfile SF is updated in the working directory 2-2 "dirstate" entry for SF is NOT updated 3. "lfcommands.updatelfiles()" is invoked (by "overrides.overriderevert()") 3-1 largefile LF (for SF) is updated in the working directory 3-2 "dirstate" returns "n" and valid timestamp for SF (by 1-1 and 2-2) 3-3 "lfdirstate" entry for LF is "normal"-ed 3-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and timestamp of LF is stored into "lfdirstate" file (by 3-3) (ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file) Then, "hs status" treats LF as "clean", even though LF is updated by "other" revision (by 3-1), because "lfilesrepo.status()" always treats "normal"-ed files (by 3-3 and 3-4) as "clean". When largefiles are reverted, they should be "normallookup"-ed forcibly. This patch uses "normallookup" on "lfdirstate" while reverting, by passing "True" to newly added argument "normallookup". Forcible "normallookup"-ing is not so expensive, because list of target largefiles is explicitly specified in this case. This patch uses "[debug] dirstate.delaywrite" feature in the test, to ensure that timestamp of the largefile gotten from "other" revision is stored into ".hg/largefiles/dirstate" (for ASSUMPTION at 3-4)

largefiles: invoke "normallookup" on "lfdirstate" for merged files Before this patch, largefiles gotten from "other" revision (with conflict) at "hg merge" become "clean" unexpectedly in steps below: 1. "repo.status()" is invoked (for status check before merging) 1-1 "dirstate" entry for standinfile SF is "normal"-ed 1-2 "lfdirstate" entry of largefile LF (for SF) is "normal"-ed 2. "merge.update()" is invoked 2-1 SF is updated in the working directory (ASSUMPTION: user choice "other" at conflict) 2-2 "dirstate" entry for SF is "merge"-ed 3. "lfcommands.updatelfiles()" is invoked (by "overrides.hgmerge()") 3-1 largefile LF (for SF) is updated in the working directory 3-2 "dirstate" returns "m" for SF (by 2-2) 3-3 "lfdirstate" entry for LF is left as it is 3-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and timestamp of LF is stored into "lfdirstate" file (by 1-2) (ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file) Then, "hs status" treats LF as "clean", even though LF is updated by "other" revision (by 3-1), because "lfilesrepo.status()" always treats "normal"-ed files (by 1-2 and 3-4) as "clean". When state of standinfile in "dirstate" is "m", largefile should be "normallookup"-ed. This patch invokes "normallookup" on "lfdirstate" for merged files. This patch uses "[debug] dirstate.delaywrite" feature in the test, to ensure that timestamp of the largefile gotten from "other" revision is stored into ".hg/largefiles/dirstate". (for ASSUMPTION at 3-4)

largefiles: use "normallookup", if "mtime" of standin is unset Before this patch, largefiles gotten from "other" revision (without conflict) at "hg merge" become "clean" unexpectedly in steps below: 1. "merge.update()" is invoked 1-1 standinfile SF is updated in the working directory 1-2 "dirstate" entry for SF is "normallookup"-ed 2. "lfcommands.updatelfiles()" is invoked (by "overrides.hgmerge()") 2-1 largefile LF (for SF) is updated in the working directory 2-2 "dirstate" returns "n" for SF (by 1-2) 2-3 "lfdirstate" entry for LF is "normal"-ed 2-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and timestamp of LF is stored into "lfdirstate" file (ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file) Then, "hs status" treats LF as "clean", even though LF is updated by "other" revision (by 2-1), because "lfilesrepo.status()" always treats "normal"-ed files (by 2-3 and 2-4) as "clean". When timestamp is not set (= negative value) for standinfile in "dirstate", largefile should be "normallookup"-ed regardless of rebasing or not, because "n" state in "dirstate" doesn't ensure "clean"-ness of a standinfile at that time. This patch uses "normallookup" instead of "normal", if "mtime" of standin is unset This is a temporary way to fix with less changes. For fundamental resolution of this kind of problems in the future, "lfdirstate" should be updated with "dirstate" simultaneously while "merge.update" execution: maybe by hooking "recordupdates" It is also why this patch (temporarily) uses internal field "_map" of "dirstate" directly. This patch uses "[debug] dirstate.delaywrite" feature in the test, to ensure that timestamp of the largefile gotten from "other" revision is stored into ".hg/largefiles/dirstate". (for ASSUMPTION at 2-4) This patch newly adds "test-largefiles-update.t", to avoid increasing cost to run other tests for largefiles by subsequent patches (especially, "[debug] dirstate.delaywrite" causes so).

dirstate: delay writing out to ensure timestamp of each entries explicitly Even though "dirstate.write()" is invoked explicitly after "normal" invocations, timestamp field of entries may be still "unset" in the "dirstate" file itself , because "pack_dirstate" drops it when it is equal to the timestamp of "dirstate" file itself. This can avoid overlooking modification of files, which are updated at same time in the second. But on the other hand, this may hide timing critical problems. For example, incorrect "normal"-ing (or lack of "normallookup"-ing on the already "normal"-ed entry) is visible only when: - the target file is modified in the working directory at T1, and - "dirstate" file is written out at T2 (!= T1) Otherwise, T1 is dropped by "pack_dirstate" in "dirstate.write()" invocation, and "unset" is stored into "dirstate" file. It often fails to reproduce problems from incorrect "normal"-ing by Mercurial testset, because automated actions in the small repository almost always causes that T1 and T2 are same. This patch adds the debug feature to delay writing out to ensure timestamp of each entries explicitly. This feature is used to make timing critical "dirstate" problems reproducable in subsequent patches.

tests: cat error messages are different on Solaris

commands: fix typo in import documentation

Added signature for changeset 6c36dc6cd61a

Added tag 3.1-rc for changeset 6c36dc6cd61a

merge default into stable for 3.1 code freeze

revset: avoid a ValueError when 'only()' is given an empty set This previously died in _revdescendants() taking the min() of the first set to only(), when it was empty. An empty second set already worked. Likewise, descendants() already handled an empty set.

cmdutil: make commit message shown in text editor customizable by template This patch makes commit message shown in text editor customizable by template. For example, this can advertise: - sample commit messages for routine works, - points to call attention before commit, - message of the day, and so on To show commit message correctly even in problematic encoding, this patch chooses the latter below: - replace "buildcommittext" with "buildcommittemplate" completely - invoke "buildcommittemplate" only if '[committemplate] changeset' is configured explicitly For example, if multibyte character ending with backslash (0x5c) is followed by ASCII character 'n' in the customized template, sequence of backslash and 'n' is treated as line-feed unexpectedly (and multibyte character is broken, too). This corruption occurs in 'decode("string-escape")' while parsing template string.

commiteditor: refactor default extramsg

filemerge: add internal:tagmerge merge tool Add a new internal:tagmerge merge tool which implements an automatic merge algorithm for mercurial's tag files The tagmerge algorithm is able to resolve most merge conflicts that currently would trigger a .hgtags merge conflict. The only case that it does not (and cannot) handle is that in which two tags point to different revisions on each merge parent _and_ their corresponding tag histories have the same rank (i.e. the same length). In all other cases the merge algorithm will choose the revision belonging to the parent with the highest ranked tag history. The merged tag history is the combination of both tag histories (special care is taken to try to combine common tag histories where possible). The algorithm also handles cases in which tags have been manually removed from the .hgtags file and other similar corner cases. In addition to actually merging the tags from two parents, taking into account the base, the algorithm also tries to minimize the difference between the merged tag file and the first parent's tag file (i.e. it tries to make the merged tag order as as similar as possible to the first parent's tag file order). The algorithm works as follows: 1. read the tags from p1, p2 and the base - when reading the p1 tags, also get the line numbers associated to each tag node (these will be used to sort the merged tags in a way that minimizes the diff to p1). Ignore the file numbers when reading p2 and the base 2. recover the "lost tags" (i.e. those that are found in the base but not on p1 or p2) and add them back to p1 and/or p2 - at this point the only tags that are on p1 but not on p2 are those new tags that were introduced in p1. Same thing for the tags that are on p2 but not on p2 3. take all tags that are only on p1 or only on p2 (but not on the base) - Note that these are the tags that were introduced between base and p1 and between base and p2, possibly on separate clones 4. for each tag found both on p1 and p2 perform the following merge algorithm: - the tags conflict if their tag "histories" have the same "rank" (i.e. length) _AND_ the last (current) tag is _NOT_ the same - for non conflicting tags: - choose which are the high and the low ranking nodes - the high ranking list of nodes is the one that is longer. In case of draw favor p1 - the merged node list is made of 3 parts: - first the nodes that are common to the beginning of both the low and the high ranking nodes - second the non common low ranking nodes - finally the non common high ranking nodes (with the last one being the merged tag node) - note that this is equivalent to putting the whole low ranking node list first, followed by the non common high ranking nodes - note that during the merge we keep the "node line numbers", which will be used when writing the merged tags to the tag file 5. write the merged tags taking into account to their positions in the first parent (i.e. try to keep the relative ordering of the nodes that come from p1). This minimizes the diff between the merged and the p1 tag files This is done by using the following algorithm - group the nodes for a given tag that must be written next to each other - A: nodes that come from consecutive lines on p1 - B: nodes that come from p2 (i.e. whose associated line number is None) and are next to one of the a nodes in A - each group is associated with a line number coming from p1 - generate a "tag block" for each of the groups - a tag block is a set of consecutive "node tag" lines belonging to the same tag and which will be written next to each other on the merged tags file - sort the "tag blocks" according to their associated number line - put blocks whose nodes come all from p2 first - write the tag blocks in the sorted order Notes: - A few tests have been added to test-tag.t. These tests are very specific to the new internal:tagmerge tool, so perhaps they should be moved to their own test file. - The merge algorithm was discussed in a thread on the mercurial mailing list. In http://markmail.org/message/anqaxldup4tmgyrx a slightly different algorithm was suggested. In it the p1 and p2 tags would have been interleaved instead of put one before the other. It would be possible to implement that but my tests suggest that the merge result would be more confusing and harder to understand.

filemerge: use non-minimal conflict marker regions (BC) As extensively detailed by Pierre-Yves[1], simplemerge's minimal markers can result in hopeless confusion for many common merges. As it happens, we accidentally inherited this behavior when we borrowed simplemerge from bzr; it is not the behavior used by RCS's merge(1), Since merge(1) (and not bzr) is what we aim to emulate when emulating RCS's merge markers, we simply turn this feature off. This brings us in line with the behavior of CVS, SVN, and Git as a bonus. (NB: using conflict markers with Mercurial is discouraged.) [1] http://markmail.org/message/wj5mh3lc46czlvld convert glob tessa

test: use more elaborated content in ``test-conflict.t`` We are going to introduce a setting to control the "minimisation" feature of ``internal:merge``. So we need more interesting conflicting content. We change the content in an independent changeset to make sure the coming code change leave the output unchanged.

run-tests: make --view work again

filemerge: use 'basic' as the default of '[ui] mergemarkers' for safety Before this patch, 'detailed' is used as the default of '[ui] mergemarkers'. This embeds non-ASCII characters in tags, branches, bookmarks, author and/or commit descriptions into merged files in the encoding specified by '--encoding' global option, 'HGENCODING' or other locale setting environment variables. But, if files to be merged use another encoding, this behavior breaks consistency of encoding in merged files. For example, ISO-2022-JP or EUC-JP are sometimes used as the file encoding for Japanese characters, because of historical and/or environmental reasons, even though UTF-8 or Shift-JIS are ordinarily used as the terminal encoding. This can't be resolved automatically, because Mercurial doesn't aware encoding of managed files. This patch uses 'basic' as the default of '[ui] mergemarkers' to avoid embedding encoding sensitive characters for safety. This patch puts '[ui] mergemarkers = detailed' into default hgrc file for tests, to reduce changes for tests in this patch.

largefiles: avoid unnecessary creation of .hg/largefiles when opening lfdirstate Previously, the directory '.hg/largefiles' would always be created if it didn't exist when the lfdirstate was opened. If there were no standin files, no dirstate file would be created in the directory. The end result was that enabling the largefiles extension globally, but not explicitly adding a largefile would result in the repository eventually sprouting this directory. Creation of this directory effectively changes readonly operations like summary and status into operations that require write access. Without write access, commands that would succeed without the extension loaded would abort with a surprising error when the extension is loaded, but not actively used: $ hg sum -R /tmp/thg --config extensions.largefiles= parent: 16541:00dc703d5aed repowidget: specify incoming bundle by plain file path to avoid url parsing branch: default abort: Permission denied: '/tmp/thg/.hg/largefiles' This change is simpler than changing the callers of openlfdirstate() to use the 'create' parameter that was introduced in ae57920ac188, and probably how that should have been implemented in the first place.

run-tests: write out scripts in binary mode Caught because Python 3 refuses to write bytes to a non-binary fd.

match: use util.re.escape instead of re.escape For a pathological .hgignore with over 2500 glob lines and over 200000 calls to re.escape, and with re2 available, this speeds up parsing the .hgignore from 0.75 seconds to 0.20 seconds. This causes e.g. 'hg status' with hgwatchman enabled to go from 1.02 seconds to 0.47 seconds.

util.re: add an escape method The escape method in at least one of the modules called 're2' is in C. This means it is significantly faster than the Python code written in 're'. An upcoming patch will have benchmarks.

util.re: move check for re2 into a separate method We're going to use the same check for another method in an upcoming patch.