hgext/automv.py
author Gregory Szorc <gregory.szorc@gmail.com>
Mon, 21 Feb 2022 11:10:47 -0700
changeset 48925 1414b65303d5
parent 48875 6000f5b25c9b
child 50043 5cfc48354d0f
permissions -rw-r--r--
localrepo: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12330
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     1
# automv.py
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     2
#
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     3
# Copyright 2013-2016 Facebook, Inc.
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     4
#
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
31599
e4aefdb58ebe automv: use lowercase for docstring title
Jun Wu <quark@fb.com>
parents: 29205
diff changeset
     7
"""check for unrecorded moves at commit time (EXPERIMENTAL)
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     8
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     9
This extension checks at commit/amend time if any of the committed files
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    10
comes from an unrecorded mv.
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    11
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    12
The threshold at which a file is considered a move can be set with the
28152
5ec1ce8fdf0a automv: switch to specifying the similarity as an integer (0-100)
Martijn Pieters <mjpieters@fb.com>
parents: 28151
diff changeset
    13
``automv.similarity`` config option. This option takes a percentage between 0
28183
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    14
(disabled) and 100 (files must be identical), the default is 95.
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    15
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    16
"""
28183
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    17
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    18
# Using 95 as a default similarity is based on an analysis of the mercurial
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    19
# repositories of the cpython, mozilla-central & mercurial repositories, as
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    20
# well as 2 very large facebook repositories. At 95 50% of all potential
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    21
# missed moves would be caught, as well as correspond with 87% of all
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    22
# explicitly marked moves.  Together, 80% of moved files are 95% similar or
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    23
# more.
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    24
#
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    25
# See http://markmail.org/thread/5pxnljesvufvom57 for context.
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    26
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    27
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28183
diff changeset
    28
from mercurial.i18n import _
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    29
from mercurial import (
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    30
    commands,
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    31
    copies,
28183
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    32
    error,
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    33
    extensions,
34971
38637dd39cfd py3: handle keyword arguments in hgext/automv.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33188
diff changeset
    34
    pycompat,
33188
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    35
    registrar,
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    36
    scmutil,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    37
    similar,
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    38
)
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    39
33188
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    40
configtable = {}
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    41
configitem = registrar.configitem(configtable)
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    42
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    43
configitem(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43077
diff changeset
    44
    b'automv',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43077
diff changeset
    45
    b'similarity',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43077
diff changeset
    46
    default=95,
33188
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    47
)
54bc88c56ec8 configitems: register the 'automv.similarity' config
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 31599
diff changeset
    48
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    49
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    50
def extsetup(ui):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    51
    entry = extensions.wrapcommand(commands.table, b'commit', mvcheck)
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    52
    entry[1].append(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    53
        (b'', b'no-automv', None, _(b'disable automatic file move detection'))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    54
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    55
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    56
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    57
def mvcheck(orig, ui, repo, *pats, **opts):
28149
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    58
    """Hook to check for moves at commit time"""
34971
38637dd39cfd py3: handle keyword arguments in hgext/automv.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33188
diff changeset
    59
    opts = pycompat.byteskwargs(opts)
28151
74e3d634a30e automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents: 28150
diff changeset
    60
    renames = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    61
    disabled = opts.pop(b'no_automv', False)
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    62
    if not disabled:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    63
        threshold = ui.configint(b'automv', b'similarity')
28183
e07daee83029 automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents: 28152
diff changeset
    64
        if not 0 <= threshold <= 100:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    65
            raise error.Abort(_(b'automv.similarity must be between 0 and 100'))
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    66
        if threshold > 0:
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    67
            match = scmutil.match(repo[None], pats, opts)
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    68
            added, removed = _interestingfiles(repo, match)
41660
f89aad980025 automv: respect ui.relative-paths
Martin von Zweigbergk <martinvonz@google.com>
parents: 34971
diff changeset
    69
            uipathfn = scmutil.getuipathfn(repo, legacyrelativevalue=True)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    70
            renames = _findrenames(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    71
                repo, uipathfn, added, removed, threshold / 100.0
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    72
            )
28151
74e3d634a30e automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents: 28150
diff changeset
    73
74e3d634a30e automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents: 28150
diff changeset
    74
    with repo.wlock():
74e3d634a30e automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents: 28150
diff changeset
    75
        if renames is not None:
28150
7a984cece04a automv: reuse existing scutil._markchanges() function
Martijn Pieters <mjpieters@fb.com>
parents: 28149
diff changeset
    76
            scmutil._markchanges(repo, (), (), renames)
34971
38637dd39cfd py3: handle keyword arguments in hgext/automv.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33188
diff changeset
    77
        return orig(ui, repo, *pats, **pycompat.strkwargs(opts))
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    78
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    79
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    80
def _interestingfiles(repo, matcher):
28149
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    81
    """Find what files were added or removed in this commit.
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    82
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    83
    Returns a tuple of two lists: (added, removed). Only files not *already*
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    84
    marked as moved are included in the added list.
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    85
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    86
    """
28146
28024d0d42dc automv: simplify retrieving the status
Martijn Pieters <mjpieters@fb.com>
parents: 28129
diff changeset
    87
    stat = repo.status(match=matcher)
42544
2702dfc7e029 automv: access status fields by name, not index
Martin von Zweigbergk <martinvonz@google.com>
parents: 42543
diff changeset
    88
    added = stat.added
2702dfc7e029 automv: access status fields by name, not index
Martin von Zweigbergk <martinvonz@google.com>
parents: 42543
diff changeset
    89
    removed = stat.removed
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    90
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    91
    copy = copies.pathcopies(repo[b'.'], repo[None], matcher)
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    92
    # remove the copy files for which we already have copy info
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    93
    added = [f for f in added if f not in copy]
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    94
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    95
    return added, removed
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    96
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
    97
41660
f89aad980025 automv: respect ui.relative-paths
Martin von Zweigbergk <martinvonz@google.com>
parents: 34971
diff changeset
    98
def _findrenames(repo, uipathfn, added, removed, similarity):
28149
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
    99
    """Find what files in added are really moved files.
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
   100
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
   101
    Any file named in removed that is at least similarity% similar to a file
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
   102
    in added is seen as a rename.
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
   103
d356d5250ab2 automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents: 28148
diff changeset
   104
    """
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   105
    renames = {}
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   106
    if similarity > 0:
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   107
        for src, dst, score in similar.findrenames(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
   108
            repo, added, removed, similarity
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
   109
        ):
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   110
            if repo.ui.verbose:
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   111
                repo.ui.status(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   112
                    _(b'detected move of %s as %s (%d%% similar)\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
   113
                    % (uipathfn(src), uipathfn(dst), score * 100)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42544
diff changeset
   114
                )
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   115
            renames[dst] = src
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   116
    if renames:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   117
        repo.ui.status(_(b'detected move of %d files\n') % len(renames))
28129
7c40b4b7f8f1 automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   118
    return renames