tests/f
author Manuel Jacob <me@manueljacob.de>
Mon, 29 Jun 2020 15:03:36 +0200
branchstable
changeset 45022 e3b19004087a
parent 43691 47ef023d0165
child 45830 c102b704edb5
permissions -rwxr-xr-x
convert: correctly convert paths to UTF-8 for Subversion The previous code using encoding.tolocal() only worked by chance in these situations: * The string is ASCII: The fast path was triggered and the string was returned unmodified. * The local encoding is UTF-8: The source and target encoding is the same. * The string is not valid UTF-8 and the native encoding is ISO-8859-1: If the string doesn’t decode using UTF-8, ISO-8859-1 is tried as a fallback. During `hg convert`, the local encoding is always UTF-8. The irony is that in this case, encoding.tolocal() behaves like what someone would expect the reverse function, encoding.fromlocal(), to do. When the locale encoding is ISO-8859-15, trying to convert a SVN repo `/tmp/a€` failed before like this: file:///tmp/a%C2%A4 does not look like a Subversion repository to libsvn version 1.14.0 The correct URL is `file:///tmp/a%E2%82%AC`. Unlike previously (with the ISO-8859-1 fallback), decoding the path using the locale encoding can fail. In this case, we have to bail out, as Subversion won’t be able to do anything useful with the path.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     1
#!/usr/bin/env python
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     2
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     3
"""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     4
Utility for inspecting files in various ways.
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     5
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     6
This tool is like the collection of tools found in a unix environment but are
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     7
cross platform and stable and suitable for our needs in the test suite.
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     8
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
     9
This can be used instead of tools like:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    10
  [
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    11
  dd
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    12
  find
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    13
  head
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    14
  hexdump
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    15
  ls
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    16
  md5sum
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    17
  readlink
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    18
  sha1sum
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    19
  stat
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    20
  tail
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    21
  test
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    22
  readlink.py
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    23
  md5sum.py
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    24
"""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    25
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    26
from __future__ import absolute_import
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    27
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    28
import binascii
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    29
import glob
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
    30
import hashlib
29160
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    31
import optparse
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    32
import os
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    33
import re
0362605b82cf py3: make tests/f use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 28046
diff changeset
    34
import sys
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    35
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    36
# Python 3 adapters
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    37
ispy3 = sys.version_info[0] >= 3
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    38
if ispy3:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    39
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    40
    def iterbytes(s):
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    41
        for i in range(len(s)):
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    42
            yield s[i : i + 1]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    43
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    44
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    45
else:
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    46
    iterbytes = iter
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    47
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    48
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    49
def visit(opts, filenames, outfile):
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    50
    """Process filenames in the way specified in opts, writing output to
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    51
    outfile."""
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    52
    for f in sorted(filenames):
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    53
        isstdin = f == '-'
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    54
        if not isstdin and not os.path.lexists(f):
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    55
            outfile.write(b'%s: file not found\n' % f.encode('utf-8'))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    56
            continue
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    57
        quiet = opts.quiet and not opts.recurse or isstdin
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    58
        isdir = os.path.isdir(f)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    59
        islink = os.path.islink(f)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    60
        isfile = os.path.isfile(f) and not islink
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    61
        dirfiles = None
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    62
        content = None
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    63
        facts = []
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    64
        if isfile:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    65
            if opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    66
                facts.append(b'file')
35472
c1f7037c2ded tests: teach `f` to handle sha256 checksums
Matt Harbison <matt_harbison@yahoo.com>
parents: 35471
diff changeset
    67
            if any((opts.hexdump, opts.dump, opts.md5, opts.sha1, opts.sha256)):
36948
0585337ea787 cleanup: fix some latent open(path).read() et al calls we previously missed
Augie Fackler <augie@google.com>
parents: 36261
diff changeset
    68
                with open(f, 'rb') as fobj:
0585337ea787 cleanup: fix some latent open(path).read() et al calls we previously missed
Augie Fackler <augie@google.com>
parents: 36261
diff changeset
    69
                    content = fobj.read()
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    70
        elif islink:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    71
            if opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    72
                facts.append(b'link')
40246
be0a5d2d5c78 tests: fix last failure in test-tools.t
Augie Fackler <augie@google.com>
parents: 38364
diff changeset
    73
            content = os.readlink(f).encode('utf8')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    74
        elif isstdin:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
    75
            content = getattr(sys.stdin, 'buffer', sys.stdin).read()
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    76
            if opts.size:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    77
                facts.append(b'size=%d' % len(content))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    78
        elif isdir:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    79
            if opts.recurse or opts.type:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    80
                dirfiles = glob.glob(f + '/*')
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    81
                facts.append(b'directory with %d files' % len(dirfiles))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    82
        elif opts.type:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    83
            facts.append(b'type unknown')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    84
        if not isstdin:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    85
            stat = os.lstat(f)
23911
593a5cd709a2 tests: teach f not to report directory size
Matt Mackall <mpm@selenic.com>
parents: 23860
diff changeset
    86
            if opts.size and not isdir:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    87
                facts.append(b'size=%d' % stat.st_size)
23912
7d0aa6269ece tests: teach f not to report symlink mode bits
Matt Mackall <mpm@selenic.com>
parents: 23911
diff changeset
    88
            if opts.mode and not islink:
36261
c69e78ef2b54 py3: port f to Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35472
diff changeset
    89
                facts.append(b'mode=%o' % (stat.st_mode & 0o777))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    90
            if opts.links:
38364
06c85cbd6824 py3: use '%d' for os.stat_result.st_nlink instead of '%s'
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36948
diff changeset
    91
                facts.append(b'links=%d' % stat.st_nlink)
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    92
            if opts.newer:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    93
                # mtime might be in whole seconds so newer file might be same
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    94
                if stat.st_mtime >= os.stat(opts.newer).st_mtime:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    95
                    facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    96
                        b'newer than %s' % opts.newer.encode('utf8', 'replace')
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    97
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
    98
                else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
    99
                    facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   100
                        b'older than %s' % opts.newer.encode('utf8', 'replace')
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   101
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   102
        if opts.md5 and content is not None:
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
   103
            h = hashlib.md5(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   104
            facts.append(b'md5=%s' % binascii.hexlify(h.digest())[: opts.bytes])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   105
        if opts.sha1 and content is not None:
29233
318534bb5dfd tests: make 'f' utility import hashlib unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 29230
diff changeset
   106
            h = hashlib.sha1(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   107
            facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   108
                b'sha1=%s' % binascii.hexlify(h.digest())[: opts.bytes]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   109
            )
35472
c1f7037c2ded tests: teach `f` to handle sha256 checksums
Matt Harbison <matt_harbison@yahoo.com>
parents: 35471
diff changeset
   110
        if opts.sha256 and content is not None:
c1f7037c2ded tests: teach `f` to handle sha256 checksums
Matt Harbison <matt_harbison@yahoo.com>
parents: 35471
diff changeset
   111
            h = hashlib.sha256(content)
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   112
            facts.append(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   113
                b'sha256=%s' % binascii.hexlify(h.digest())[: opts.bytes]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   114
            )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   115
        if isstdin:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   116
            outfile.write(b', '.join(facts) + b'\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   117
        elif facts:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   118
            outfile.write(b'%s: %s\n' % (f.encode('utf-8'), b', '.join(facts)))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   119
        elif not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   120
            outfile.write(b'%s:\n' % f.encode('utf-8'))
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   121
        if content is not None:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   122
            chunk = content
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   123
            if not islink:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   124
                if opts.lines:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   125
                    if opts.lines >= 0:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   126
                        chunk = b''.join(chunk.splitlines(True)[: opts.lines])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   127
                    else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   128
                        chunk = b''.join(chunk.splitlines(True)[opts.lines :])
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   129
                if opts.bytes:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   130
                    if opts.bytes >= 0:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   131
                        chunk = chunk[: opts.bytes]
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   132
                    else:
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   133
                        chunk = chunk[opts.bytes :]
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   134
            if opts.hexdump:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   135
                for i in range(0, len(chunk), 16):
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   136
                    s = chunk[i : i + 16]
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   137
                    outfile.write(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   138
                        b'%04x: %-47s |%s|\n'
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   139
                        % (
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   140
                            i,
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   141
                            b' '.join(b'%02x' % ord(c) for c in iterbytes(s)),
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   142
                            re.sub(b'[^ -~]', b'.', s),
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   143
                        )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   144
                    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   145
            if opts.dump:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   146
                if not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   147
                    outfile.write(b'>>>\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   148
                outfile.write(chunk)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   149
                if not quiet:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   150
                    if chunk.endswith(b'\n'):
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   151
                        outfile.write(b'<<<\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   152
                    else:
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   153
                        outfile.write(b'\n<<< no trailing newline\n')
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   154
        if opts.recurse and dirfiles:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   155
            assert not isstdin
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   156
            visit(opts, dirfiles, outfile)
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   157
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   158
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   159
if __name__ == "__main__":
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   160
    parser = optparse.OptionParser("%prog [options] [filenames]")
43691
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   161
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   162
        "-t",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   163
        "--type",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   164
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   165
        help="show file type (file or directory)",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   166
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   167
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   168
        "-m", "--mode", action="store_true", help="show file mode"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   169
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   170
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   171
        "-l", "--links", action="store_true", help="show number of links"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   172
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   173
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   174
        "-s", "--size", action="store_true", help="show size of file"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   175
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   176
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   177
        "-n", "--newer", action="store", help="check if file is newer (or same)"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   178
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   179
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   180
        "-r", "--recurse", action="store_true", help="recurse into directories"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   181
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   182
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   183
        "-S",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   184
        "--sha1",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   185
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   186
        help="show sha1 hash of the content",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   187
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   188
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   189
        "",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   190
        "--sha256",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   191
        action="store_true",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   192
        help="show sha256 hash of the content",
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   193
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   194
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   195
        "-M", "--md5", action="store_true", help="show md5 hash of the content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   196
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   197
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   198
        "-D", "--dump", action="store_true", help="dump file content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   199
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   200
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   201
        "-H", "--hexdump", action="store_true", help="hexdump file content"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   202
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   203
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   204
        "-B", "--bytes", type="int", help="number of characters to dump"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   205
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   206
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   207
        "-L", "--lines", type="int", help="number of lines to dump"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   208
    )
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   209
    parser.add_option(
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   210
        "-q", "--quiet", action="store_true", help="no default output"
47ef023d0165 black: blacken scripts
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40306
diff changeset
   211
    )
23860
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   212
    (opts, filenames) = parser.parse_args(sys.argv[1:])
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   213
    if not filenames:
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   214
        filenames = ['-']
bead0c7b4f68 tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff changeset
   215
34270
3db2365d43e4 tests: update `f` helper script to work on Python 3
Augie Fackler <augie@google.com>
parents: 32898
diff changeset
   216
    visit(opts, filenames, getattr(sys.stdout, 'buffer', sys.stdout))