hgext/relink.py
author Yuya Nishihara <yuya@tcha.org>
Sat, 03 Mar 2018 23:49:39 -0500
changeset 36701 d77c3b023393
parent 36632 9e2866065982
child 37084 f0b6fbea00cf
permissions -rw-r--r--
lock: block signal interrupt while making a lock file On Windows where symlink isn't supported, util.makelock() could leave an empty file if interrupted immediately after os.open(). This empty lock never dies as it has no process id recorded. ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) # an interrupt may occur here os.write(ld, info) os.close(ld) This was a long-standing bug of TortoiseHg which runs a command-server and kills it by CTRL_C_EVENT, reported by random Windows users. https://bitbucket.org/tortoisehg/thg/issues/4873/#comment-43591129 At first, I tried to fix makelock() to clean up a stale lock file, which turned out to be hard because any instructions may be interrupted by a signal. ld = None try: # CALL_FUNCTION # os.open(...) # an interrupt may occur here # STORE_FAST # ld = ... ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) os.write(ld, info) ... return True except: if ld: ... os.unlink(pathname) return False So I decided to block signals by temporarily replacing the signal handlers so makelcok() and held = 1 will never be interrupted. Many thanks to Fernando Najera for investigating the issue.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
     1
# Mercurial extension to provide 'hg relink' command
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     2
#
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     3
# Copyright (C) 2007 Brendan Cully <brendan@kublai.com>
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     4
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 4270
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10225
diff changeset
     6
# GNU General Public License version 2 or any later version.
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     7
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
     8
"""recreates hardlinks between repository clones"""
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
     9
from __future__ import absolute_import
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    10
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    11
import os
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    12
import stat
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    13
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28380
diff changeset
    14
from mercurial.i18n import _
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    15
from mercurial import (
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    16
    error,
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    17
    hg,
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 29841
diff changeset
    18
    registrar,
28380
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    19
    util,
ad266834251b relink: use absolute_import
timeless <timeless@mozdev.org>
parents: 26778
diff changeset
    20
)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    21
21252
da0eb4970913 relink: declare command using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20083
diff changeset
    22
cmdtable = {}
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 29841
diff changeset
    23
command = registrar.command(cmdtable)
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29205
diff changeset
    24
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
    25
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
    26
# be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 21252
diff changeset
    27
# leave the attribute unspecified.
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29205
diff changeset
    28
testedwith = 'ships-with-hg-core'
16743
38caf405d010 hgext: mark all first-party extensions as such
Augie Fackler <raf@durin42.com>
parents: 16686
diff changeset
    29
21252
da0eb4970913 relink: declare command using decorator
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20083
diff changeset
    30
@command('relink', [], _('[ORIGIN]'))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    31
def relink(ui, repo, origin=None, **opts):
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    32
    """recreate hardlinks between two repositories
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    33
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    34
    When repositories are cloned locally, their data files will be
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    35
    hardlinked so that they only use the space of a single repository.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    36
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    37
    Unfortunately, subsequent pulls into either repository will break
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    38
    hardlinks for any files touched by the new changesets, even if
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    39
    both repositories end up pulling the same changes.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    40
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    41
    Similarly, passing --rev to "hg clone" will fail to use any
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    42
    hardlinks, falling back to a complete copy of the source
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    43
    repository.
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    44
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    45
    This command lets you recreate those hardlinks and reclaim that
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    46
    wasted space.
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    47
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    48
    This repository will be relinked to share space with ORIGIN, which
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    49
    must be on the same local disk. If ORIGIN is omitted, looks for
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    50
    "default-relink", then "default", in [paths].
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    51
9886
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    52
    Do not attempt any read operations on this repository while the
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    53
    command is running. (Both repositories will be locked against
56af3f240a22 relink: wrap long lines in docstring
Martin Geisler <mg@lazybytes.net>
parents: 9790
diff changeset
    54
    writes.)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    55
    """
14945
11aad09a6370 hgext: replace uses of hasattr with util.safehasattr
Augie Fackler <durin42@gmail.com>
parents: 14709
diff changeset
    56
    if (not util.safehasattr(util, 'samefile') or
11aad09a6370 hgext: replace uses of hasattr with util.safehasattr
Augie Fackler <durin42@gmail.com>
parents: 14709
diff changeset
    57
        not util.safehasattr(util, 'samedevice')):
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
    58
        raise error.Abort(_('hardlinks are not supported on this system'))
18825
f0564402d059 repo: repo isolation, do not pass on repo.ui for creating new repos
Simon Heimberg <simohe@besonet.ch>
parents: 17191
diff changeset
    59
    src = hg.repository(repo.baseui, ui.expandpath(origin or 'default-relink',
14556
517e1d88bf7e hg: change various repository() users to use peer() where appropriate
Matt Mackall <mpm@selenic.com>
parents: 14235
diff changeset
    60
                                          origin or 'default'))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    61
    ui.status(_('relinking %s to %s\n') % (src.store.path, repo.store.path))
13657
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
    62
    if repo.root == src.root:
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
    63
        ui.status(_('there is nothing to relink\n'))
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
    64
        return
b69102740e57 relink: avoid trying to lock the same repo twice
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
    65
20083
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
    66
    if not util.samedevice(src.store.path, repo.store.path):
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
    67
        # No point in continuing
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
    68
        raise error.Abort(_('source and destination are on different devices'))
20083
c69e5911888d relink: abort earlier when on different devices (issue3916)
Simon Heimberg <simohe@besonet.ch>
parents: 18825
diff changeset
    69
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    70
    locallock = repo.lock()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    71
    try:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    72
        remotelock = src.lock()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    73
        try:
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    74
            candidates = sorted(collect(src, ui))
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
    75
            targets = prune(candidates, src.store.path, repo.store.path, ui)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    76
            do_relink(src.store.path, repo.store.path, targets, ui)
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    77
        finally:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    78
            remotelock.release()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    79
    finally:
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    80
        locallock.release()
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    81
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
    82
def collect(src, ui):
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    83
    seplen = len(os.path.sep)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    84
    candidates = []
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    85
    live = len(src['tip'].manifest())
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    86
    # Your average repository has some files which were deleted before
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    87
    # the tip revision. We account for that by assuming that there are
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    88
    # 3 tracked files for every 2 live files as of the tip version of
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    89
    # the repository.
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    90
    #
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    91
    # mozilla-central as of 2010-06-10 had a ratio of just over 7:5.
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    92
    total = live * 3 // 2
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    93
    src = src.store.path
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    94
    pos = 0
26778
a95c975f42e3 l10n: use %d instead of %s for numbers
timeless@mozdev.org
parents: 26587
diff changeset
    95
    ui.status(_("tip has %d files, estimated total number of files: %d\n")
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
    96
              % (live, total))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    97
    for dirpath, dirnames, filenames in os.walk(src):
11357
7914628b4751 relink: ensure deterministic directory walk in collect
Martin Geisler <mg@aragost.com>
parents: 11355
diff changeset
    98
        dirnames.sort()
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    99
        relpath = dirpath[len(src) + seplen:]
11357
7914628b4751 relink: ensure deterministic directory walk in collect
Martin Geisler <mg@aragost.com>
parents: 11355
diff changeset
   100
        for filename in sorted(filenames):
16686
67964cda8701 cleanup: "not x in y" -> "x not in y"
Brodie Rao <brodie@sf.io>
parents: 14945
diff changeset
   101
            if filename[-2:] not in ('.d', '.i'):
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   102
                continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   103
            st = os.stat(os.path.join(dirpath, filename))
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   104
            if not stat.S_ISREG(st.st_mode):
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   105
                continue
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
   106
            pos += 1
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   107
            candidates.append((os.path.join(relpath, filename), st))
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
   108
            ui.progress(_('collecting'), pos, filename, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   109
11355
9011036ba79d relink/progress: Adding progress for collecting stage
timeless <timeless@gmail.com>
parents: 11354
diff changeset
   110
    ui.progress(_('collecting'), None)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   111
    ui.status(_('collected %d candidate storage files\n') % len(candidates))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   112
    return candidates
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   113
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   114
def prune(candidates, src, dst, ui):
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   115
    def linkfilter(src, dst, st):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   116
        try:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   117
            ts = os.stat(dst)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   118
        except OSError:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   119
            # Destination doesn't have this file?
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   120
            return False
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   121
        if util.samefile(src, dst):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   122
            return False
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   123
        if not util.samedevice(src, dst):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   124
            # No point in continuing
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25660
diff changeset
   125
            raise error.Abort(
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   126
                _('source and destination are on different devices'))
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   127
        if st.st_size != ts.st_size:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   128
            return False
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   129
        return st
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   130
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   131
    targets = []
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
   132
    total = len(candidates)
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
   133
    pos = 0
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   134
    for fn, st in candidates:
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
   135
        pos += 1
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   136
        srcpath = os.path.join(src, fn)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   137
        tgt = os.path.join(dst, fn)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   138
        ts = linkfilter(srcpath, tgt, st)
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   139
        if not ts:
14709
6c7283faa967 check-code: don't mark debug messages for translation
Matt Mackall <mpm@selenic.com>
parents: 14556
diff changeset
   140
            ui.debug('not linkable: %s\n' % fn)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   141
            continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   142
        targets.append((fn, ts.st_size))
12744
0793d763e413 progress: dropping superfluous space from units
timeless <timeless@gmail.com>
parents: 12062
diff changeset
   143
        ui.progress(_('pruning'), pos, fn, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   144
11354
412a6e749f8d relink/progress: Adding progress for pruning stage
timeless <timeless@gmail.com>
parents: 11273
diff changeset
   145
    ui.progress(_('pruning'), None)
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   146
    ui.status(_('pruned down to %d probably relinkable files\n') % len(targets))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   147
    return targets
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   148
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   149
def do_relink(src, dst, files, ui):
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   150
    def relinkfile(src, dst):
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   151
        bak = dst + '.bak'
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   152
        os.rename(dst, bak)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   153
        try:
14235
b9e1b041744f rename util.os_link to oslink
Adrian Buehlmann <adrian@cadifra.com>
parents: 13898
diff changeset
   154
            util.oslink(src, dst)
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   155
        except OSError:
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   156
            os.rename(bak, dst)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   157
            raise
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   158
        os.remove(bak)
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   159
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   160
    CHUNKLEN = 65536
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   161
    relinked = 0
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   162
    savedbytes = 0
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   163
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   164
    pos = 0
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   165
    total = len(files)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   166
    for f, sz in files:
9729
aa9ccab5af37 Issue919: add a standard extension to recreate hardlinks between repositories.
Jesse Glick <jesse.glick@sun.com>
parents: 8225
diff changeset
   167
        pos += 1
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   168
        source = os.path.join(src, f)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   169
        tgt = os.path.join(dst, f)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   170
        # Binary mode, so that read() works correctly, especially on Windows
36394
4bc983568016 py3: replace file() with open()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32337
diff changeset
   171
        sfp = open(source, 'rb')
4bc983568016 py3: replace file() with open()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32337
diff changeset
   172
        dfp = open(tgt, 'rb')
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   173
        sin = sfp.read(CHUNKLEN)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   174
        while sin:
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   175
            din = dfp.read(CHUNKLEN)
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   176
            if sin != din:
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   177
                break
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   178
            sin = sfp.read(CHUNKLEN)
10218
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   179
        sfp.close()
750b7a4f01f6 Add support for relinking on Windows.
Siddharth Agarwal <sid.bugzilla@gmail.com>
parents: 9790
diff changeset
   180
        dfp.close()
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   181
        if sin:
14709
6c7283faa967 check-code: don't mark debug messages for translation
Matt Mackall <mpm@selenic.com>
parents: 14556
diff changeset
   182
            ui.debug('not linkable: %s\n' % f)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   183
            continue
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   184
        try:
4270
29eb88bd5c8d hg-relink: do not compare .d files
Brendan Cully <brendan@kublai.com>
parents: 4249
diff changeset
   185
            relinkfile(source, tgt)
12744
0793d763e413 progress: dropping superfluous space from units
timeless <timeless@gmail.com>
parents: 12062
diff changeset
   186
            ui.progress(_('relinking'), pos, f, _('files'), total)
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   187
            relinked += 1
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   188
            savedbytes += sz
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25186
diff changeset
   189
        except OSError as inst:
36632
9e2866065982 py3: use util.forcevytestr to convert error to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36394
diff changeset
   190
            ui.warn('%s: %s\n' % (tgt, util.forcebytestr(inst)))
4249
7663780b55a7 Add hg-relink script to contrib
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
   191
10724
cb0a7faa29ea progress: drop extra args for pos=None calls (issue2087)
Matt Mackall <mpm@selenic.com>
parents: 10698
diff changeset
   192
    ui.progress(_('relinking'), None)
10424
93b5abcf5101 relink: properly use the progress API
Augie Fackler <durin42@gmail.com>
parents: 10264
diff changeset
   193
13656
0e200e1801f4 relink: format reclaimed byte count nicely
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
   194
    ui.status(_('relinked %d files (%s reclaimed)\n') %
0e200e1801f4 relink: format reclaimed byte count nicely
Martin Geisler <mg@lazybytes.net>
parents: 12744
diff changeset
   195
              (relinked, util.bytecount(savedbytes)))