linkrev-filelog: handle filtered linkrev with no visible children (issue4307) If the file revision with a filtered linkrev does not have any (unfiltered) children, we cannot use it to bound the search for another introduction. Instead, we have to look at the file revision used by each head changeset. If one of them uses this file revision, we know there is another occurrence and we have a starting point. See inline comments for details. Adding some kind of permanent reference of all the introductions of a file revision instead of just the first one would be much better. But this is more difficult. I hope to take that into account in the next repository format.

linkrev: work around linkrev to filtered entry in 'filelog' revset This revset is used by 'hg log FILENAME'. This prevent bugs when used on a repository with hidden revisions. Instead of just discarding file revisions whose linkrevs point to filtered revisions, we put them aside and post-process them trying to find a non-filtered introduction. See inline documentation for details about how it works. This only fixes some of the problems. Once again, more will be needed when we can cannot rely on child revisions of a file to find linkrev-shadowned revisions. A test is added for 'hg log' catching such cases.

namespaces: update documentation and code indentation The previous patch changed just the functionality, while this patch focuses on the documentation and indentation to keep review simple.

namespaces: use namespace object instead of dictionary This isn't as bad as the diff seems, it only looks like scary. In this patch, we use the 'namespace' object instead of accessing keys in a dictionary. This required the 'templatename' and 'names' method to change their implementation. Later, we will remove these functions entirely due to a better api.

namespaces: copy implementation to new namespace object

namespaces: add a namespace object Currently, we use a dictionary object to store the namespace properties. This is python so let's use an object. This will allow us to be more flexible in our method signatures in the future.

localrepo: use vfs.reljoin rather than os.path.join in the localrepository class The removes the last remaining usage of os.path.join inside teh localrepository class.

largefiles: drop a redundant check for patterns when removing without --after This is in the way of passing a matcher to removelargefiles(). This method is called in exactly two places- first in overrides.addremove() (but only if the pattern list passed to it is not empty), and second in the commands.remove() override. But since the latter calls commands.remove() first, which also does this check, it isn't needed here.

context: override _dirstatestatus in workingcommitctx for correct matching Before this patch, the result of "status()" on "workingcommitctx" may incorrectly contain files other than ones to be committed, because "workingctx._dirstatestatus()" returns the result of "dirstate.status()" directly. For correct matching, this patch overrides "_dirstatestatus" in "workingcommitctx" and makes it return matched files only in "self._status". This patch uses empty list for "deleted", "unknown" and "ignored" of status, because status between "changectx"s also makes them empty.

context: avoid breaking already fixed self._status at ctx.status() Before this patch, "status()" on "workingcommitctx" with "always match" object causes breaking "self._status" in "workingctx._buildstatus()", because "workingctx._buildstatus()" caches the result of "dirstate.status()" into "self._status" for efficiency, even though it should be fixed at construction for committing. For example, template function "diff()" without any patterns in "committemplate" implies "status()" on "workingcommitctx" with "always match" object, via "basectx.diff()" and "patch.diff()". Then, broken "self._status" causes committing unexpected files. To avoid breaking already fixed "self._status" at "ctx.status()", this patch overrides "_buildstatus" in "workingcommitctx". This patch doesn't write out the result of template function "diff()" in "committemplate" in "test-commit.t", because matching against files to be committed still has an issue fixed in subsequent patch.

context: add workingcommitctx for exact context to be committed Before this patch, "workingctx" is also used for the context to be committed. But "workingctx" works incorrectly in some cases. For example, even when only some of changed files in the working directory are committed, "status()" on "workingctx" object for committing recognizes files not to be committed as changed, too. As the preparation for fixing these issues, this patch chooses adding new class "workingcommitctx" for exact context to be committed, because switching by the flag (like "self._fixedstatus" or so) in some code paths of "workingctx" is less readable and maintenancable.

context: make unknown/ignored/clean of cached status empty for equivalence Before this patch, "workingctx.status" caches the result of "dirstate.status" directly into "self._status". But "dirstate.status" is invoked with False "list*" arguments in normal "self._status" accessing route, and this makes "unknown"/"ignored"/"clean" of status empty. This may cause unexpected result of code paths internally accessing to them (accessors for external usage are already removed by previous patch). This patch makes "unknown"/"ignored"/"clean" of cached status empty for equivalence. Making them empty is executed only when at least one of "unknown", "ignored" or "clean" has files, for efficiency.

templatefilters.json: stabilize output The json filter was previously iterating over keys in an object in an undefined order. Let's throw a sorted() in there so output is consistent. It's somewhat frightening that there are no tests for the json filter. Subsequent commits will add them, so we pass on the opportunity to add them here.

templatefilters.json: call functions The "changeset" template from hgweb is using a lambda in the "diffsummary" key. In preparation for enabling JSON output from hgweb, teach the json filter how to call functions.

merge with stable

linkrev: also adjust linkrev when bootstrapping annotate (issue4305) The annotate logic now use the new 'introrev' method to bootstrap its traversal. This catches issues from linkrev-shadowing of the changeset introducing the version of a file in source changeset. More tests have been added to display pathological cases.

linkrev: also adjust linkrev when bootstrapping 'follow' revset The follow revset (used by `hg log --follow`) now uses the new 'introrev' method to bootstrap its traversal. This catches issues from linkrev-shadowing of the changesets introducing the version of a file in source changeset. A new test has been added to display pathological cases. Another test is affected because it was meant to test this behavior but actually failed to do so for half of the output. The output are now similar.

linkrev: introduce an 'introrev' method on filectx The previous changeset properly fixed the ancestors computation, but we need to ensure that the initial filectx is also using the right changeset. When asking for log or annotation from a certain point, the first step is to define the changeset that introduced the current file version. We cannot just pick the "starting point" changesets as it may just "use" the file revision, unchanged. Currently, we were using 'linkrev' for this purpose, but this exposes us to unexpected branch-jumping when the revision introducing the starting point version is itself linkrev-shadowed. So we need to take the topology into account again. Therefore, we introduce an 'introrev' function, returning the changeset which introduced the file change in the current changeset. This function will be used to fix linkrev-related issues when bootstrapping 'hg log --follow' and 'hg annotate'. It reuses the '_adjustlinkrev' function, extending it to allow introspection of the initial changeset too. In the previous usage of the '_adjustlinkrev' the starting rev was always using a children file revisions, so it could be safely ignored in the search. In this case, the starting point is using the revision of the file we are looking, and may be the changeset we are looking for.

filectx.parents: enforce changeid of parent to be in own changectx ancestors Because of the way filenodes are computed, you can have multiple changesets "introducing" the same file revision. For example, in the changeset graph below, changeset 2 and 3 both change a file -to- and -from- the same content. o 3: content = new | | o 2: content = new |/ o 1: content = old In such cases, the file revision is create once, when 2 is added, and just reused for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for optimization purposes when walking a file history, the linkrev-shadowing results in an unexpected jump to another branch during such a walk.. This leads to multiple bugs with log, annotate and rename detection. One element to fix such bugs is to ensure that walking the file history sticks on the same topology as the changeset's history. For this purpose, we extend the logic in 'basefilectx.parents' so that it always defines the proper changeset to associate the parent file revision with. This "proper" changeset has to be an ancestor of the changeset associated with the child file revision. This logic is performed in the '_adjustlinkrev' function. This function is given the starting changeset and all the information regarding the parent file revision. If the linkrev for the file revision is an ancestor of the starting changeset, the linkrev is valid and will be used. If it is not, we detected a topological jump caused by linkrev shadowing, we are going to walk the ancestors of the starting changeset until we find one setting the file to the revision we are trying to create. The performance impact appears acceptable: - We are walking the changelog once for each filelog traversal (as there should be no overlap between searches), - changelog traversal itself is fairly cheap, compared to what is likely going to be perform on the result on the filelog traversal, - We only touch the manifest for ancestors touching the file, And such changesets are likely to be the one introducing the file. (except in pathological cases involving merge), - We use manifest diff instead of full manifest unpacking to check manifest content, so it does not involve applying multiple diffs in most case. - linkrev shadowing is not the common case. Tests for fixed issues in log, annotate and rename detection have been added. But this changeset does not solve all problems. It fixes -ancestry- computation, but if the linkrev-shadowed changesets is the starting one, we'll still get things wrong. We'll have to fix the bootstrapping of such operations in a later changeset. Also, the usage of `hg log FILE` without --follow still has issues with linkrev pointing to hidden changesets, because it relies on the `filelog` revset which implement its own traversal logic that is still to be fixed. Thanks goes to: - Matt Mackall: for nudging me in the right direction - Julien Cristau and Rémi Cardona: for keep telling me linkrev bug were an evolution show stopper for 3 years. - Durham Goode: for finding a new linkrev issue every few weeks - Mads Kiilerich: for that last rename bug who raise this topic over my anoyance limit.

context: remove unreliable accessor methods from committablectx There are two caching routes for (propertycache-ed) "_status" below in committablectx: - invoking "status()": "dirstate.status()" is invoked, and the result of it is cached into "_status". In this case, any of "listignored", "listclean" and "listunknown" may be True. - accessing "_status" directly before "status()": Own "status()" is invoked, but all of "listignored", "listclean" and "listunknown" arguments are False, in this case. "ignored"/"clean"/"unknown" accessor methods of "committablectx" use corresponded fields of "_status", but these fields aren't reliable, because these fields are empty when: - "_status" method is executed before accessors, or - "status()" is executed with "list*=False" before accessors In addition to it, these accessors aren't used in the recent Mercurial implementation. At least, removing them doesn't cause any test failures.

context: cache self._status correctly at workingctx.status Before this patch, "workingctx.status" always replaces "self._status" by the recent result, even though: - status isn't calculated against the parent of the working directory, or - specified "match" isn't "always" one (status is only visible partially) If "workingctx" object is shared between some procedures indirectly referring "ctx._status", this incorrect caching may cause unexpected result: for example, "ctx._status" is used via "manifest()", "files()" and so on. To cache "self._status" correctly at "workingctx.status", this patch overwrites "self._status" in "workingctx._buildstatus" only when: - status is calculated against the parent of the working directory, and - specified "match" is "always" one This patch can be applied (and effective) only on default branch, because procedure around "basectx.status" is much different between stable and default: for example, overwriting "self._status" itself is executed not in "workingctx._buildstatus" but in "workingctx._poststatus", on stable branch.

filectx.parents: also fetch the filelog of rename source too we are going to need this filelog for the linkrev adjustment, so we better normalise the list and have the filelog in all case. This is done in a previous changeset to help readability.

Added signature for changeset 1265a3a71d75

Added tag 3.2.4 for changeset 1265a3a71d75

largefiles: backout f72d73937853 - linear updates handle m -> a differently f72d73937853 introduced a significant performance regression: All largefiles were marked 'normallookup' by linear (or noop) updates and had to be rehashed by the next command. The previous change introduced a different solution to the problem f72d73937853 solved and we can thus back it out again.

largefiles: mark lfile as added in lfdirstate when the standin is added This is an alternative solution to the problem addressed by f72d73937853. This implementation has the advantage that it doesn't mark clean largefiles as normallookup. We can thus avoid repeated rehashing of all largefiles when f72d73937853 is backed out. This implementation use the existing 'lfmr' actions that 23fe278bde43 introduced for handling another part of the same cases.

tests: add test coverage for lfdirstate invalidation of linear update f72d73937853 introduced a significant performance regression: All largefiles are marked 'normallookup' in lfdirstate by linear (or noop) updates and has to be rehashed by the next command. To avoid such regressions, keep an eye on the dirstate content after a plain 'hg up'.

test-subrepo-git: ignore global git config This was causing a test failure for people with company-wide settings. Still need a way to ignore local config.

tests/autodiff.py: explicitly only honor feature diffopts This test extension manages the opts it cares about on its own anyway.

cmdutil.changeset_printer: explicitly honor all diffopts This is used in hg log -p so the output is expected to be the same as that of hg diff.

export: explicitly honor all diffopts This is slightly more controversial than diff, but we hope that HGPLAIN=1 covers all the format-breaking ones. A possible alternative here that breaks BC is to honor all opts except the whitespace ones.

webcommands.annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.