FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 31 Dec 2014 17:55:43 +0900] rev 23709
context: make unknown/ignored/clean of cached status empty for equivalence
Before this patch, "workingctx.status" caches the result of
"dirstate.status" directly into "self._status".
But "dirstate.status" is invoked with False "list*" arguments in
normal "self._status" accessing route, and this makes
"unknown"/"ignored"/"clean" of status empty.
This may cause unexpected result of code paths internally accessing to
them (accessors for external usage are already removed by previous patch).
This patch makes "unknown"/"ignored"/"clean" of cached status empty
for equivalence. Making them empty is executed only when at least one
of "unknown", "ignored" or "clean" has files, for efficiency.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Dec 2014 13:48:55 -0800] rev 23708
templatefilters.json: stabilize output
The json filter was previously iterating over keys in an object in an
undefined order. Let's throw a sorted() in there so output is
consistent.
It's somewhat frightening that there are no tests for the json filter.
Subsequent commits will add them, so we pass on the opportunity to add
them here.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Dec 2014 11:22:17 -0800] rev 23707
templatefilters.json: call functions
The "changeset" template from hgweb is using a lambda in the
"diffsummary" key. In preparation for enabling JSON output from hgweb,
teach the json filter how to call functions.
Matt Mackall <mpm@selenic.com> [Thu, 01 Jan 2015 16:47:14 -0600] rev 23706
merge with stable
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 24 Dec 2014 03:26:48 -0800] rev 23705
linkrev: also adjust linkrev when bootstrapping annotate (issue4305)
The annotate logic now use the new 'introrev' method to bootstrap its traversal.
This catches issues from linkrev-shadowing of the changeset introducing the
version of a file in source changeset.
More tests have been added to display pathological cases.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 29 Dec 2014 23:40:24 -0800] rev 23704
linkrev: also adjust linkrev when bootstrapping 'follow' revset
The follow revset (used by `hg log --follow`) now uses the new 'introrev'
method to bootstrap its traversal. This catches issues from linkrev-shadowing of
the changesets introducing the version of a file in source changeset.
A new test has been added to display pathological cases.
Another test is affected because it was meant to test this behavior but actually
failed to do so for half of the output. The output are now similar.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 23 Dec 2014 16:14:39 -0800] rev 23703
linkrev: introduce an 'introrev' method on filectx
The previous changeset properly fixed the ancestors computation, but we need to
ensure that the initial filectx is also using the right changeset.
When asking for log or annotation from a certain point, the first step is to
define the changeset that introduced the current file version. We cannot just
pick the "starting point" changesets as it may just "use" the file revision,
unchanged.
Currently, we were using 'linkrev' for this purpose, but this exposes us to
unexpected branch-jumping when the revision introducing the starting point
version is itself linkrev-shadowed. So we need to take the topology into
account again. Therefore, we introduce an 'introrev' function, returning the
changeset which introduced the file change in the current changeset.
This function will be used to fix linkrev-related issues when bootstrapping 'hg
log --follow' and 'hg annotate'.
It reuses the '_adjustlinkrev' function, extending it to allow introspection of
the initial changeset too. In the previous usage of the '_adjustlinkrev' the
starting rev was always using a children file revisions, so it could be safely
ignored in the search. In this case, the starting point is using the revision
of the file we are looking, and may be the changeset we are looking for.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 23 Dec 2014 15:30:38 -0800] rev 23702
filectx.parents: enforce changeid of parent to be in own changectx ancestors
Because of the way filenodes are computed, you can have multiple changesets
"introducing" the same file revision. For example, in the changeset graph
below, changeset 2 and 3 both change a file -to- and -from- the same content.
o 3: content = new
|
| o 2: content = new
|/
o 1: content = old
In such cases, the file revision is create once, when 2 is added, and just reused
for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing
to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for
optimization purposes when walking a file history, the linkrev-shadowing
results in an unexpected jump to another branch during such a walk.. This leads to
multiple bugs with log, annotate and rename detection.
One element to fix such bugs is to ensure that walking the file history sticks on
the same topology as the changeset's history. For this purpose, we extend the
logic in 'basefilectx.parents' so that it always defines the proper changeset
to associate the parent file revision with. This "proper" changeset has to be an
ancestor of the changeset associated with the child file revision.
This logic is performed in the '_adjustlinkrev' function. This function is
given the starting changeset and all the information regarding the parent file
revision. If the linkrev for the file revision is an ancestor of the starting
changeset, the linkrev is valid and will be used. If it is not, we detected a
topological jump caused by linkrev shadowing, we are going to walk the
ancestors of the starting changeset until we find one setting the file to the
revision we are trying to create.
The performance impact appears acceptable:
- We are walking the changelog once for each filelog traversal (as there should
be no overlap between searches),
- changelog traversal itself is fairly cheap, compared to what is likely going
to be perform on the result on the filelog traversal,
- We only touch the manifest for ancestors touching the file, And such
changesets are likely to be the one introducing the file. (except in
pathological cases involving merge),
- We use manifest diff instead of full manifest unpacking to check manifest
content, so it does not involve applying multiple diffs in most case.
- linkrev shadowing is not the common case.
Tests for fixed issues in log, annotate and rename detection have been
added.
But this changeset does not solve all problems. It fixes -ancestry-
computation, but if the linkrev-shadowed changesets is the starting one, we'll
still get things wrong. We'll have to fix the bootstrapping of such operations
in a later changeset. Also, the usage of `hg log FILE` without --follow still
has issues with linkrev pointing to hidden changesets, because it relies on the
`filelog` revset which implement its own traversal logic that is still to be
fixed.
Thanks goes to:
- Matt Mackall: for nudging me in the right direction
- Julien Cristau and RĂ©mi Cardona: for keep telling me linkrev bug were an
evolution show stopper for 3 years.
- Durham Goode: for finding a new linkrev issue every few weeks
- Mads Kiilerich: for that last rename bug who raise this topic over my
anoyance limit.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 31 Dec 2014 17:55:43 +0900] rev 23701
context: remove unreliable accessor methods from committablectx
There are two caching routes for (propertycache-ed) "_status" below in
committablectx:
- invoking "status()":
"dirstate.status()" is invoked, and the result of it is cached
into "_status". In this case, any of "listignored", "listclean"
and "listunknown" may be True.
- accessing "_status" directly before "status()":
Own "status()" is invoked, but all of "listignored", "listclean"
and "listunknown" arguments are False, in this case.
"ignored"/"clean"/"unknown" accessor methods of "committablectx" use
corresponded fields of "_status", but these fields aren't reliable,
because these fields are empty when:
- "_status" method is executed before accessors, or
- "status()" is executed with "list*=False" before accessors
In addition to it, these accessors aren't used in the recent Mercurial
implementation. At least, removing them doesn't cause any test
failures.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 31 Dec 2014 17:55:43 +0900] rev 23700
context: cache self._status correctly at workingctx.status
Before this patch, "workingctx.status" always replaces "self._status"
by the recent result, even though:
- status isn't calculated against the parent of the working directory, or
- specified "match" isn't "always" one
(status is only visible partially)
If "workingctx" object is shared between some procedures indirectly
referring "ctx._status", this incorrect caching may cause unexpected
result: for example, "ctx._status" is used via "manifest()", "files()"
and so on.
To cache "self._status" correctly at "workingctx.status", this patch
overwrites "self._status" in "workingctx._buildstatus" only when:
- status is calculated against the parent of the working directory, and
- specified "match" is "always" one
This patch can be applied (and effective) only on default branch,
because procedure around "basectx.status" is much different between
stable and default: for example, overwriting "self._status" itself is
executed not in "workingctx._buildstatus" but in
"workingctx._poststatus", on stable branch.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 23 Dec 2014 18:30:46 -0800] rev 23699
filectx.parents: also fetch the filelog of rename source too
we are going to need this filelog for the linkrev adjustment, so we better
normalise the list and have the filelog in all case.
This is done in a previous changeset to help readability.
Matt Mackall <mpm@selenic.com> [Thu, 01 Jan 2015 16:30:11 -0600] rev 23698
Added signature for changeset 1265a3a71d75
Matt Mackall <mpm@selenic.com> [Thu, 01 Jan 2015 16:29:51 -0600] rev 23697
Added tag 3.2.4 for changeset 1265a3a71d75
Mads Kiilerich <madski@unity3d.com> [Wed, 31 Dec 2014 14:46:03 +0100] rev 23696
largefiles: backout f72d73937853 - linear updates handle m -> a differently
f72d73937853 introduced a significant performance regression: All largefiles
were marked 'normallookup' by linear (or noop) updates and had to be rehashed
by the next command.
The previous change introduced a different solution to the problem f72d73937853
solved and we can thus back it out again.
Mads Kiilerich <madski@unity3d.com> [Wed, 31 Dec 2014 14:46:02 +0100] rev 23695
largefiles: mark lfile as added in lfdirstate when the standin is added
This is an alternative solution to the problem addressed by f72d73937853. This
implementation has the advantage that it doesn't mark clean largefiles as
normallookup. We can thus avoid repeated rehashing of all largefiles when
f72d73937853 is backed out.
This implementation use the existing 'lfmr' actions that 23fe278bde43
introduced for handling another part of the same cases.
Mads Kiilerich <madski@unity3d.com> [Wed, 31 Dec 2014 14:45:02 +0100] rev 23694
tests: add test coverage for lfdirstate invalidation of linear update
f72d73937853 introduced a significant performance regression: All largefiles
are marked 'normallookup' in lfdirstate by linear (or noop) updates and has to
be rehashed by the next command.
To avoid such regressions, keep an eye on the dirstate content after a plain
'hg up'.
Matt Mackall <mpm@selenic.com> [Tue, 30 Dec 2014 15:51:14 -0600] rev 23693
test-subrepo-git: ignore global git config
This was causing a test failure for people with company-wide settings.
Still need a way to ignore local config.
Siddharth Agarwal <sid0@fb.com> [Fri, 21 Nov 2014 16:02:26 -0800] rev 23692
tests/autodiff.py: explicitly only honor feature diffopts
This test extension manages the opts it cares about on its own anyway.
Siddharth Agarwal <sid0@fb.com> [Fri, 21 Nov 2014 16:01:55 -0800] rev 23691
cmdutil.changeset_printer: explicitly honor all diffopts
This is used in hg log -p so the output is expected to be the same as that of
hg diff.
Siddharth Agarwal <sid0@fb.com> [Tue, 18 Nov 2014 22:21:03 -0800] rev 23690
export: explicitly honor all diffopts
This is slightly more controversial than diff, but we hope that HGPLAIN=1
covers all the format-breaking ones.
A possible alternative here that breaks BC is to honor all opts except the
whitespace ones.
Siddharth Agarwal <sid0@fb.com> [Fri, 21 Nov 2014 16:16:03 -0800] rev 23689
webcommands.annotate: explicitly only honor whitespace diffopts
The whitespace ones are the only ones the annotate logic cares about anyway, so
there's no visible impact.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 23 Dec 2014 18:29:03 -0800] rev 23688
filectx.parents: filter nullrev parent sooner
We are going to introduce a linkrev-correction phases when computing parents.
It will be more convenient to have the nullid parent filtered out earlier. I
had to make a minimal adjustment to the rename handling logic to keep it
functional. That logic have been documented in the process since it took me
some time to check all the cases out.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 23 Dec 2014 17:13:51 -0800] rev 23687
context: catch FilteredRepoLookupError instead of RepoLookupError
Now that we have a more specialised exception, lets use it when we meant to
catch the more specialised case.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 27 Nov 2014 10:16:56 -0500] rev 23686
narrowmatcher: propagate the rel() method
The full path is propagated to the original match object since this is often
used directly for printing a file name to the user. This is cleaner than
requiring each caller to join the prefix with the file name prior to calling it,
and will lead to not having to pass the prefix around separately. It is also
consistent with the bad() and abs() methods in terms of the required input. The
uipath() method now inherits this path building property.
There is no visible change in path style for rel() because it ultimately calls
util.pathto(), which returns an os.sep based path. (The previous os.path.join()
was violating the documented usage of util.pathto(), that its third parameter be
'/' separated.) The doctest needed to be normalized to '/' separators to avoid
test differences on Windows, now that a full path is returned for a short
filename.
The test changes are to drop globs that are no longer necessary when printing an
absolute file in a subrepo, as returned from match.uipath(). Previously when
os.path.join() was used to add the prefix, the absolute path to a file in a
subrepo was printed with a mix of '/' and '\'. The absolute path for a file not
in a subrepo, as returned from match.uipath(), is still purely '/' based.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 28 Nov 2014 20:15:46 -0500] rev 23685
match: add the abs() method
This is a utility to make it easier for subrepos to convert a file name to the
full path rooted at the top repository. It can replace the various path joining
lambdas, and doesn't require the prefix to be passed into the method that wishes
to build such a path.
The name is derived from the following pattern in annotate() and other places:
name = ((pats and rel) or abs)
The pathname separator is not os.sep in part to avoid confusion with variables
named 'abs' or similar that _are_ '/' separated, and in part because some
methods like cmdutils.forget() and maybe cmdutils.add() need to build a '/'
separated path to the root repo. This can replace the manual path building
there.
Matt Mackall <mpm@selenic.com> [Mon, 29 Dec 2014 16:39:20 -0600] rev 23684
merge with stable
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 25 Dec 2014 23:33:26 +0900] rev 23683
posix: quote the specified string only when it may have to be quoted
This patch makes "posix.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety, like as the
previous patch for "windows.shellquote".
In fact, on POSIX environment, quoting itself doesn't cause issues
like issue4463. But (almost) equivalent quoting policy can avoid
examining test result differently on POSIX and Windows (even though
showing command line with "%r" causes such examination in
"test-extdiff.t").
The last hunk for "test-extdiff.t" in this patch isn't needed for the
previous patch for "windows.shellquote", because the code path of it
is executed only "#if execbit" (= avoided on Windows).
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 25 Dec 2014 23:33:26 +0900] rev 23682
windows: quote the specified string only when it has to be quoted
Before this patch, "windows.shellquote" (as used as "util.shellquote")
always quotes specified strings with double quotation marks, for
external process invocation.
But some problematic applications can't work correctly, when command
line arguments are quoted: see issue4463 for detail.
On the other hand, quoting itself is needed to specify arguments
containing whitespaces and/or some special characters exactly.
This patch makes "windows.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety.