tests/md5sum.py
author Jun Wu <quark@fb.com>
Wed, 29 Mar 2017 12:37:03 -0700
changeset 31720 dea2a17cbfd0
parent 29731 8d1cdee372e6
child 32852 3a64ac39b893
permissions -rwxr-xr-x
hardlink: check directory's st_dev when copying files Previously, when copying a file, copyfiles will compare src's st_dev with dirname(dst)'s st_dev, to decide whether to enable hardlink or not. That could have issues on Linux's overlayfs, where stating directories could result in different st_dev from st_dev of stating files, even if both the directories and the files exist in the overlay's upperdir. This patch fixes it by checking dirname(src) instead. It's more consistent because we are checking directories for both src and dest. That fixes test-hardlinks.t running on common Docker setups.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4122
306055f5b65c Unified #! paths for python scripts and removed them for test modules.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 3223
diff changeset
     1
#!/usr/bin/env python
1928
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     2
#
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     3
# Based on python's Tools/scripts/md5sum.py
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     4
#
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     5
# This software may be used and distributed according to the terms
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     6
# of the PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2, which is
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     7
# GPL-compatible.
50e1c90b0fcf clarify license on md5sum.py
Peter van Dijk <peter@dataloss.nl>
parents: 1924
diff changeset
     8
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
     9
from __future__ import absolute_import
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    10
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    11
import os
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    12
import sys
6470
ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6212
diff changeset
    13
ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6212
diff changeset
    14
try:
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    15
    import hashlib
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    16
    md5 = hashlib.md5
6470
ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6212
diff changeset
    17
except ImportError:
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    18
    import md5
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 25660
diff changeset
    19
    md5 = md5.md5
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    20
7080
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    21
try:
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    22
    import msvcrt
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    23
    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    24
    msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    25
except ImportError:
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    26
    pass
a6477aa893b8 tests: Windows compatibility fixes
Patrick Mezard <pmezard@gmail.com>
parents: 6470
diff changeset
    27
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    28
for filename in sys.argv[1:]:
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    29
    try:
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    30
        fp = open(filename, 'rb')
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 14494
diff changeset
    31
    except IOError as msg:
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    32
        sys.stderr.write('%s: Can\'t open: %s\n' % (filename, msg))
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    33
        sys.exit(1)
3223
53e843840349 Whitespace/Tab cleanup
Thomas Arendsen Hein <thomas@intevation.de>
parents: 1928
diff changeset
    34
6470
ac0bcd951c2c python 2.6 compatibility: compatibility wrappers for hash functions
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6212
diff changeset
    35
    m = md5()
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    36
    try:
29731
8d1cdee372e6 md5sum: use `iter(callable, sentinel)` instead of while True
Augie Fackler <augie@google.com>
parents: 29485
diff changeset
    37
        for data in iter(lambda: fp.read(8192), ''):
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    38
            m.update(data)
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 14494
diff changeset
    39
    except IOError as msg:
1924
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    40
        sys.stderr.write('%s: I/O error: %s\n' % (filename, msg))
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    41
        sys.exit(1)
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    42
    sys.stdout.write('%s  %s\n' % (m.hexdigest(), filename))
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    43
46fb38ef9a91 add md5sum.py required by fix in previous changeset
Peter van Dijk <peter@dataloss.nl>
parents:
diff changeset
    44
sys.exit(0)