Mercurial > hg
annotate hgext/automv.py @ 30224:ad56071b37d4 stable
dirstate: fix debug.dirstate.delaywrite to use the new "now" after sleeping
It seems like the a regression has sneaked into debug.dirstate.delaywrite in
6c6b48aca328. It would sleep until no files were modified "now" any more, but
when writing the dirstate it would use the old "now" and still mark files as
'unset' instead of recording the timestamp that would make the file show up as
clean instead of unknown.
Instead of getting a new "now" from the file system, we trust the computed end
time as the new "now" and thus cause the actual modification time to be
writiten to the dirstate.
debug.dirstate.delaywrite is undocumented and only used in
test-largefiles-update.t . All tests seems to work fine for me without
debug.dirstate.delaywrite . Perhaps because it not really worked as intended
without the fix in this patch, and code and tests thus have evolved to do fine
without it? It could thus perhaps make sense to drop usage of this setting in
the tests. That could speed the test up a bit.
This functionality (or something very similar) can however apparently be very
convenient in setups where checking dirty-ness is expensive - such as when
using large files and have slow file filesystems or are CPU constrained. Now it
works and we can try it. (But ideally, for the largefile use case, it should
probably only delay lfdirstate writes - not ordinary dirstate.)
author | Mads Kiilerich <madski@unity3d.com> |
---|---|
date | Tue, 18 Oct 2016 16:52:35 +0200 |
parents | a0939666b836 |
children | e4aefdb58ebe |
rev | line source |
---|---|
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
1 # automv.py |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
2 # |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
3 # Copyright 2013-2016 Facebook, Inc. |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
4 # |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
7 """Check for unrecorded moves at commit time (EXPERIMENTAL) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
8 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
9 This extension checks at commit/amend time if any of the committed files |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
10 comes from an unrecorded mv. |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
11 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
12 The threshold at which a file is considered a move can be set with the |
28152
5ec1ce8fdf0a
automv: switch to specifying the similarity as an integer (0-100)
Martijn Pieters <mjpieters@fb.com>
parents:
28151
diff
changeset
|
13 ``automv.similarity`` config option. This option takes a percentage between 0 |
28183
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
14 (disabled) and 100 (files must be identical), the default is 95. |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
15 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
16 """ |
28183
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
17 |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
18 # Using 95 as a default similarity is based on an analysis of the mercurial |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
19 # repositories of the cpython, mozilla-central & mercurial repositories, as |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
20 # well as 2 very large facebook repositories. At 95 50% of all potential |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
21 # missed moves would be caught, as well as correspond with 87% of all |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
22 # explicitly marked moves. Together, 80% of moved files are 95% similar or |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
23 # more. |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
24 # |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
25 # See http://markmail.org/thread/5pxnljesvufvom57 for context. |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
26 |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
27 from __future__ import absolute_import |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
28 |
29205
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28183
diff
changeset
|
29 from mercurial.i18n import _ |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
30 from mercurial import ( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
31 commands, |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
32 copies, |
28183
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
33 error, |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
34 extensions, |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
35 scmutil, |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
36 similar |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
37 ) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
38 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
39 def extsetup(ui): |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
40 entry = extensions.wrapcommand( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
41 commands.table, 'commit', mvcheck) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
42 entry[1].append( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
43 ('', 'no-automv', None, |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
44 _('disable automatic file move detection'))) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
45 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
46 def mvcheck(orig, ui, repo, *pats, **opts): |
28149
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
47 """Hook to check for moves at commit time""" |
28151
74e3d634a30e
automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents:
28150
diff
changeset
|
48 renames = None |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
49 disabled = opts.pop('no_automv', False) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
50 if not disabled: |
28183
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
51 threshold = ui.configint('automv', 'similarity', 95) |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
52 if not 0 <= threshold <= 100: |
e07daee83029
automv: use 95 as the default similarity threshold
Martijn Pieters <mjpieters@fb.com>
parents:
28152
diff
changeset
|
53 raise error.Abort(_('automv.similarity must be between 0 and 100')) |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
54 if threshold > 0: |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
55 match = scmutil.match(repo[None], pats, opts) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
56 added, removed = _interestingfiles(repo, match) |
28152
5ec1ce8fdf0a
automv: switch to specifying the similarity as an integer (0-100)
Martijn Pieters <mjpieters@fb.com>
parents:
28151
diff
changeset
|
57 renames = _findrenames(repo, match, added, removed, |
5ec1ce8fdf0a
automv: switch to specifying the similarity as an integer (0-100)
Martijn Pieters <mjpieters@fb.com>
parents:
28151
diff
changeset
|
58 threshold / 100.0) |
28151
74e3d634a30e
automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents:
28150
diff
changeset
|
59 |
74e3d634a30e
automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents:
28150
diff
changeset
|
60 with repo.wlock(): |
74e3d634a30e
automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents:
28150
diff
changeset
|
61 if renames is not None: |
28150
7a984cece04a
automv: reuse existing scutil._markchanges() function
Martijn Pieters <mjpieters@fb.com>
parents:
28149
diff
changeset
|
62 scmutil._markchanges(repo, (), (), renames) |
28151
74e3d634a30e
automv: do not release lock between marking files and the actual commit
Martijn Pieters <mjpieters@fb.com>
parents:
28150
diff
changeset
|
63 return orig(ui, repo, *pats, **opts) |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
64 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
65 def _interestingfiles(repo, matcher): |
28149
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
66 """Find what files were added or removed in this commit. |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
67 |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
68 Returns a tuple of two lists: (added, removed). Only files not *already* |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
69 marked as moved are included in the added list. |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
70 |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
71 """ |
28146
28024d0d42dc
automv: simplify retrieving the status
Martijn Pieters <mjpieters@fb.com>
parents:
28129
diff
changeset
|
72 stat = repo.status(match=matcher) |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
73 added = stat[1] |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
74 removed = stat[2] |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
75 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
76 copy = copies._forwardcopies(repo['.'], repo[None], matcher) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
77 # remove the copy files for which we already have copy info |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
78 added = [f for f in added if f not in copy] |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
79 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
80 return added, removed |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
81 |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
82 def _findrenames(repo, matcher, added, removed, similarity): |
28149
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
83 """Find what files in added are really moved files. |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
84 |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
85 Any file named in removed that is at least similarity% similar to a file |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
86 in added is seen as a rename. |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
87 |
d356d5250ab2
automv: improve function docstrings
Martijn Pieters <mjpieters@fb.com>
parents:
28148
diff
changeset
|
88 """ |
28129
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
89 renames = {} |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
90 if similarity > 0: |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
91 for src, dst, score in similar.findrenames( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
92 repo, added, removed, similarity): |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
93 if repo.ui.verbose: |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
94 repo.ui.status( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
95 _('detected move of %s as %s (%d%% similar)\n') % ( |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
96 matcher.rel(src), matcher.rel(dst), score * 100)) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
97 renames[dst] = src |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
98 if renames: |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
99 repo.ui.status(_('detected move of %d files\n') % len(renames)) |
7c40b4b7f8f1
automv: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
100 return renames |