Mercurial > hg
view contrib/hgfixes/fix_bytes.py @ 23702:c48924787eaa
filectx.parents: enforce changeid of parent to be in own changectx ancestors
Because of the way filenodes are computed, you can have multiple changesets
"introducing" the same file revision. For example, in the changeset graph
below, changeset 2 and 3 both change a file -to- and -from- the same content.
o 3: content = new
|
| o 2: content = new
|/
o 1: content = old
In such cases, the file revision is create once, when 2 is added, and just reused
for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing
to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for
optimization purposes when walking a file history, the linkrev-shadowing
results in an unexpected jump to another branch during such a walk.. This leads to
multiple bugs with log, annotate and rename detection.
One element to fix such bugs is to ensure that walking the file history sticks on
the same topology as the changeset's history. For this purpose, we extend the
logic in 'basefilectx.parents' so that it always defines the proper changeset
to associate the parent file revision with. This "proper" changeset has to be an
ancestor of the changeset associated with the child file revision.
This logic is performed in the '_adjustlinkrev' function. This function is
given the starting changeset and all the information regarding the parent file
revision. If the linkrev for the file revision is an ancestor of the starting
changeset, the linkrev is valid and will be used. If it is not, we detected a
topological jump caused by linkrev shadowing, we are going to walk the
ancestors of the starting changeset until we find one setting the file to the
revision we are trying to create.
The performance impact appears acceptable:
- We are walking the changelog once for each filelog traversal (as there should
be no overlap between searches),
- changelog traversal itself is fairly cheap, compared to what is likely going
to be perform on the result on the filelog traversal,
- We only touch the manifest for ancestors touching the file, And such
changesets are likely to be the one introducing the file. (except in
pathological cases involving merge),
- We use manifest diff instead of full manifest unpacking to check manifest
content, so it does not involve applying multiple diffs in most case.
- linkrev shadowing is not the common case.
Tests for fixed issues in log, annotate and rename detection have been
added.
But this changeset does not solve all problems. It fixes -ancestry-
computation, but if the linkrev-shadowed changesets is the starting one, we'll
still get things wrong. We'll have to fix the bootstrapping of such operations
in a later changeset. Also, the usage of `hg log FILE` without --follow still
has issues with linkrev pointing to hidden changesets, because it relies on the
`filelog` revset which implement its own traversal logic that is still to be
fixed.
Thanks goes to:
- Matt Mackall: for nudging me in the right direction
- Julien Cristau and RĂ©mi Cardona: for keep telling me linkrev bug were an
evolution show stopper for 3 years.
- Durham Goode: for finding a new linkrev issue every few weeks
- Mads Kiilerich: for that last rename bug who raise this topic over my
anoyance limit.
author | Pierre-Yves David <pierre-yves.david@fb.com> |
---|---|
date | Tue, 23 Dec 2014 15:30:38 -0800 |
parents | 48ef68004ec9 |
children |
line wrap: on
line source
"""Fixer that changes plain strings to bytes strings.""" import re from lib2to3 import fixer_base from lib2to3.pgen2 import token from lib2to3.fixer_util import Name from lib2to3.pygram import python_symbols as syms _re = re.compile(r'[rR]?[\'\"]') # XXX: Implementing a blacklist in 2to3 turned out to be more troublesome than # blacklisting some modules inside the fixers. So, this is what I came with. blacklist = ('mercurial/demandimport.py', 'mercurial/py3kcompat.py', # valid python 3 already 'mercurial/i18n.py', ) def isdocstring(node): def isclassorfunction(ancestor): symbols = (syms.funcdef, syms.classdef) # if the current node is a child of a function definition, a class # definition or a file, then it is a docstring if ancestor.type == syms.simple_stmt: try: while True: if ancestor.type in symbols: return True ancestor = ancestor.parent except AttributeError: return False return False def ismodule(ancestor): # Our child is a docstring if we are a simple statement, and our # ancestor is file_input. In other words, our child is a lone string in # the source file. try: if (ancestor.type == syms.simple_stmt and ancestor.parent.type == syms.file_input): return True except AttributeError: return False def isdocassignment(ancestor): # Assigning to __doc__, definitely a string try: while True: if (ancestor.type == syms.expr_stmt and Name('__doc__') in ancestor.children): return True ancestor = ancestor.parent except AttributeError: return False if ismodule(node.parent) or \ isdocassignment(node.parent) or \ isclassorfunction(node.parent): return True return False def shouldtransform(node): specialnames = ['__main__'] if node.value in specialnames: return False ggparent = node.parent.parent.parent sggparent = str(ggparent) if 'getattr' in sggparent or \ 'hasattr' in sggparent or \ 'setattr' in sggparent or \ 'encode' in sggparent or \ 'decode' in sggparent: return False return True class FixBytes(fixer_base.BaseFix): PATTERN = 'STRING' def transform(self, node, results): # The filename may be prefixed with a build directory. if self.filename.endswith(blacklist): return if node.type == token.STRING: if _re.match(node.value): if isdocstring(node): return if not shouldtransform(node): return new = node.clone() new.value = 'b' + new.value return new