annotate hgext/relink.py @ 33649:377e8ddaebef stable

pathauditor: disable cache of audited paths by default (issue5628) The initial attempt was to discard cache when appropriate, but it appears to be error prone. We had to carefully inspect all places where audit() is called e.g. without actually updating filesystem, before removing files and directories, etc. So, this patch disables the cache of audited paths by default, and enables it only for the following cases: - short-lived auditor objects - repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories and considered sort of append-only (a file/directory would never be replaced with a symlink) There would be more cacheable vfs objects (e.g. mq.queue.opener), but I decided not to inspect all of them in this patch. We can make them cached later. Benchmark result: - using old clone of http://selenic.com/repo/linux-2.6/ (38319 files) - on tmpfs - run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null - try 4 times and take the last three results original: real 7.480 secs (user 1.140+22.760 sys 0.150+1.690) real 8.010 secs (user 1.070+22.280 sys 0.170+2.120) real 7.470 secs (user 1.120+22.390 sys 0.120+1.910) clearcache (the other series): real 7.680 secs (user 1.120+23.420 sys 0.140+1.970) real 7.670 secs (user 1.110+23.620 sys 0.130+1.810) real 7.740 secs (user 1.090+23.510 sys 0.160+1.940) enable cache only for vfs and svfs (this series): real 8.730 secs (user 1.500+25.190 sys 0.260+2.260) real 8.750 secs (user 1.490+25.170 sys 0.250+2.340) real 9.010 secs (user 1.680+25.340 sys 0.280+2.540) remove cache function at all (for reference): real 9.620 secs (user 1.440+27.120 sys 0.250+2.980) real 9.420 secs (user 1.400+26.940 sys 0.320+3.130) real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)
author Yuya Nishihara <yuya@tcha.org>
date Wed, 26 Jul 2017 22:10:15 +0900
parents 46ba2cdda476
children 4bc983568016
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
1 # Mercurial extension to provide 'hg relink' command
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
2 #
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
3 # Copyright (C) 2007 Brendan Cully <brendan@kublai.com>
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
4 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 4270
diff changeset
5 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10225
diff changeset
6 # GNU General Public License version 2 or any later version.
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
7
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
8 """recreates hardlinks between repository clones"""
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
9 from __future__ import absolute_import
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
10
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
11 import os
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
12 import stat
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
13
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28380
diff changeset
14 from mercurial.i18n import _
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
15 from mercurial import (
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
16 error,
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
17 hg,
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 29841
diff changeset
18 registrar,
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
19 util,
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
20 )
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
21
21252
da0eb4970913 relink: declare command using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20083
diff changeset
22 cmdtable = {}
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 29841
diff changeset
23 command = registrar.command(cmdtable)
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29205
diff changeset
24 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
25 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
26 # be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
27 # leave the attribute unspecified.
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29205
diff changeset
28 testedwith = 'ships-with-hg-core'
16743
38caf405d010 hgext: mark all first-party extensions as such
Augie Fackler <raf@durin42.com>
parents: 16686
diff changeset
29
21252
da0eb4970913 relink: declare command using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20083
diff changeset
30 @command('relink', [], _('[ORIGIN]'))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
31 def relink(ui, repo, origin=None, **opts):
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
32 """recreate hardlinks between two repositories
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
33
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
34 When repositories are cloned locally, their data files will be
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
35 hardlinked so that they only use the space of a single repository.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
36
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
37 Unfortunately, subsequent pulls into either repository will break
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
38 hardlinks for any files touched by the new changesets, even if
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
39 both repositories end up pulling the same changes.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
40
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
41 Similarly, passing --rev to "hg clone" will fail to use any
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
42 hardlinks, falling back to a complete copy of the source
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
43 repository.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
44
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
45 This command lets you recreate those hardlinks and reclaim that
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
46 wasted space.
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
47
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
48 This repository will be relinked to share space with ORIGIN, which
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
49 must be on the same local disk. If ORIGIN is omitted, looks for
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
50 "default-relink", then "default", in [paths].
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
51
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
52 Do not attempt any read operations on this repository while the
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
53 command is running. (Both repositories will be locked against
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
54 writes.)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
55 """
14945
11aad09a6370 hgext: replace uses of hasattr with util.safehasattr
Augie Fackler <durin42@gmail.com>
parents: 14709
diff changeset
56 if (not util.safehasattr(util, 'samefile') or
11aad09a6370 hgext: replace uses of hasattr with util.safehasattr
Augie Fackler <durin42@gmail.com>
parents: 14709
diff changeset
57 not util.safehasattr(util, 'samedevice')):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
58 raise error.Abort(_('hardlinks are not supported on this system'))
18825
f0564402d059 repo: repo isolation, do not pass on repo.ui for creating new repos
Simon Heimberg <simohe@besonet.ch>
parents: 17191
diff changeset
59 src = hg.repository(repo.baseui, ui.expandpath(origin or 'default-relink',
14556
517e1d88bf7e hg: change various repository() users to use peer() where appropriate
Matt Mackall <mpm@selenic.com>
parents: 14235
diff changeset
60 origin or 'default'))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
61 ui.status(_('relinking %s to %s\n') % (src.store.path, repo.store.path))
13657
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
62 if repo.root == src.root:
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
63 ui.status(_('there is nothing to relink\n'))
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
64 return
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
65
20083
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
66 if not util.samedevice(src.store.path, repo.store.path):
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
67 # No point in continuing
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
68 raise error.Abort(_('source and destination are on different devices'))
20083
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
69
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
70 locallock = repo.lock()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
71 try:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
72 remotelock = src.lock()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
73 try:
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
74 candidates = sorted(collect(src, ui))
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
75 targets = prune(candidates, src.store.path, repo.store.path, ui)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
76 do_relink(src.store.path, repo.store.path, targets, ui)
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
77 finally:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
78 remotelock.release()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
79 finally:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
80 locallock.release()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
81
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
82 def collect(src, ui):
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
83 seplen = len(os.path.sep)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
84 candidates = []
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
85 live = len(src['tip'].manifest())
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
86 # Your average repository has some files which were deleted before
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
87 # the tip revision. We account for that by assuming that there are
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
88 # 3 tracked files for every 2 live files as of the tip version of
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
89 # the repository.
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
90 #
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
91 # mozilla-central as of 2010-06-10 had a ratio of just over 7:5.
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
92 total = live * 3 // 2
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
93 src = src.store.path
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
94 pos = 0
26778
a95c975f42e3 l10n: use %d instead of %s for numbers
timeless@mozdev.org
parents: 26587
diff changeset
95 ui.status(_("tip has %d files, estimated total number of files: %d\n")
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
96 % (live, total))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
97 for dirpath, dirnames, filenames in os.walk(src):
11357
7914628b4751 relink: ensure deterministic directory walk in collect
Martin Geisler <mg@aragost.com>
parents: 11355
diff changeset
98 dirnames.sort()
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
99 relpath = dirpath[len(src) + seplen:]
11357
7914628b4751 relink: ensure deterministic directory walk in collect
Martin Geisler <mg@aragost.com>
parents: 11355
diff changeset
100 for filename in sorted(filenames):
16686
67964cda8701 cleanup: "not x in y" -> "x not in y"
Brodie Rao <brodie@sf.io>
parents: 14945
diff changeset
101 if filename[-2:] not in ('.d', '.i'):
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
102 continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
103 st = os.stat(os.path.join(dirpath, filename))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
104 if not stat.S_ISREG(st.st_mode):
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
105 continue
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
106 pos += 1
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
107 candidates.append((os.path.join(relpath, filename), st))
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
108 ui.progress(_('collecting'), pos, filename, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
109
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
110 ui.progress(_('collecting'), None)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
111 ui.status(_('collected %d candidate storage files\n') % len(candidates))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
112 return candidates
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
113
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
114 def prune(candidates, src, dst, ui):
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
115 def linkfilter(src, dst, st):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
116 try:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
117 ts = os.stat(dst)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
118 except OSError:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
119 # Destination doesn't have this file?
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
120 return False
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
121 if util.samefile(src, dst):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
122 return False
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
123 if not util.samedevice(src, dst):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
124 # No point in continuing
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
125 raise error.Abort(
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
126 _('source and destination are on different devices'))
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
127 if st.st_size != ts.st_size:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
128 return False
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
129 return st
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
130
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
131 targets = []
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
132 total = len(candidates)
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
133 pos = 0
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
134 for fn, st in candidates:
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
135 pos += 1
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
136 srcpath = os.path.join(src, fn)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
137 tgt = os.path.join(dst, fn)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
138 ts = linkfilter(srcpath, tgt, st)
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
139 if not ts:
14709
6c7283faa967 check-code: don't mark debug messages for translation
Matt Mackall <mpm@selenic.com>
parents: 14556
diff changeset
140 ui.debug('not linkable: %s\n' % fn)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
141 continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
142 targets.append((fn, ts.st_size))
12744
0793d763e413 progress: dropping superfluous space from units
timeless <timeless@gmail.com>
parents: 12062
diff changeset
143 ui.progress(_('pruning'), pos, fn, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
144
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
145 ui.progress(_('pruning'), None)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
146 ui.status(_('pruned down to %d probably relinkable files\n') % len(targets))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
147 return targets
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
148
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
149 def do_relink(src, dst, files, ui):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
150 def relinkfile(src, dst):
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
151 bak = dst + '.bak'
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
152 os.rename(dst, bak)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
153 try:
14235
b9e1b041744f rename util.os_link to oslink
Adrian Buehlmann <adrian@cadifra.com>
parents: 13898
diff changeset
154 util.oslink(src, dst)
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
155 except OSError:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
156 os.rename(bak, dst)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
157 raise
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
158 os.remove(bak)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
159
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
160 CHUNKLEN = 65536
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
161 relinked = 0
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
162 savedbytes = 0
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
163
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
164 pos = 0
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
165 total = len(files)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
166 for f, sz in files:
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
167 pos += 1
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
168 source = os.path.join(src, f)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
169 tgt = os.path.join(dst, f)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
170 # Binary mode, so that read() works correctly, especially on Windows
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
171 sfp = file(source, 'rb')
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
172 dfp = file(tgt, 'rb')
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
173 sin = sfp.read(CHUNKLEN)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
174 while sin:
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
175 din = dfp.read(CHUNKLEN)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
176 if sin != din:
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
177 break
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
178 sin = sfp.read(CHUNKLEN)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
179 sfp.close()
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
180 dfp.close()
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
181 if sin:
14709
6c7283faa967 check-code: don't mark debug messages for translation
Matt Mackall <mpm@selenic.com>
parents: 14556
diff changeset
182 ui.debug('not linkable: %s\n' % f)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
183 continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
184 try:
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
185 relinkfile(source, tgt)
12744
0793d763e413 progress: dropping superfluous space from units
timeless <timeless@gmail.com>
parents: 12062
diff changeset
186 ui.progress(_('relinking'), pos, f, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
187 relinked += 1
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
188 savedbytes += sz
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25186
diff changeset
189 except OSError as inst:
9790
819e6c7085fc relink: do not translate format string with no text
Martin Geisler <mg@lazybytes.net>
parents: 9729
diff changeset
190 ui.warn('%s: %s\n' % (tgt, str(inst)))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
191
10724
cb0a7faa29ea progress: drop extra args for pos=None calls (issue2087)
Matt Mackall <mpm@selenic.com>
parents: 10698
diff changeset
192 ui.progress(_('relinking'), None)
10424
93b5abcf5101 relink: properly use the progress API
Augie Fackler <durin42@gmail.com>
parents: 10264
diff changeset
193
13656
0e200e1801f4 relink: format reclaimed byte count nicely
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
194 ui.status(_('relinked %d files (%s reclaimed)\n') %
0e200e1801f4 relink: format reclaimed byte count nicely
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
195 (relinked, util.bytecount(savedbytes)))