mercurial/utils/resourceutil.py
author Martin von Zweigbergk <martinvonz@google.com>
Thu, 04 Mar 2021 16:06:55 -0800
changeset 46634 ad30b29bc23d
parent 45452 dd9e28612468
child 46783 7fd369644c68
permissions -rw-r--r--
copies: choose target directory based on longest match If one side of a merge renames `dir1/` to `dir2/` and the subdirectory `dir1/subdir1/` to `dir2/subdir2/`, and the other side of the merge adds a file in `dir1/subdir1/`, we should clearly move that into `dir2/subdir2/`. We already detect the directories correctly before this patch, but we iterate over them in arbitrary order. That results in the new file sometimes ending up in `dir2/subdir1/` instead. This patch fixes it by iterating over the source directories by visiting subdirectories first. That's achieved by simply iterating over them in reverse lexicographical order. Without the fix, the test case still passes on Python 2 but fails on Python 3. It depends on the iteration order of the dict. I did not look into how it's built up and why it behaved differently before the fix. I could probably have gotten it to fail on Python 2 as well by choosing different directory names. Differential Revision: https://phab.mercurial-scm.org/D10115
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
43671
664e24207728 procutil: move mainfrozen() to new resourceutil.py
Martin von Zweigbergk <martinvonz@google.com>
parents: 43657
diff changeset
     1
# resourceutil.py - utility for looking up resources
8226
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     2
#
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     3
#  Copyright 2005 K. Thananchayan <thananck@yahoo.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     4
#  Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     5
#  Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     6
#
8b2cd04a6e97 put license and copyright info into comment blocks
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
     7
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9996
diff changeset
     8
# GNU General Public License version 2 or any later version.
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
     9
37118
5be286db5fb5 procutil: move process/executable management functions to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37117
diff changeset
    10
from __future__ import absolute_import
27358
ac839ee45b6a util: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27357
diff changeset
    11
ac839ee45b6a util: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27357
diff changeset
    12
import imp
43672
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    13
import os
27358
ac839ee45b6a util: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27357
diff changeset
    14
import sys
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43019
diff changeset
    15
43671
664e24207728 procutil: move mainfrozen() to new resourceutil.py
Martin von Zweigbergk <martinvonz@google.com>
parents: 43657
diff changeset
    16
from .. import pycompat
1293
a6ffcebd3315 Enhance the file filtering capabilities.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1292
diff changeset
    17
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43019
diff changeset
    18
14228
116de1da2154 rename util.main_is_frozen to mainfrozen
Adrian Buehlmann <adrian@cadifra.com>
parents: 14167
diff changeset
    19
def mainfrozen():
6499
479847ccabe0 Added hgexecutable support for py2exe/frozen scripts
"Paul Moore <p.f.moore@gmail.com>"
parents: 5659
diff changeset
    20
    """return True if we are a frozen executable.
479847ccabe0 Added hgexecutable support for py2exe/frozen scripts
"Paul Moore <p.f.moore@gmail.com>"
parents: 5659
diff changeset
    21
479847ccabe0 Added hgexecutable support for py2exe/frozen scripts
"Paul Moore <p.f.moore@gmail.com>"
parents: 5659
diff changeset
    22
    The code supports py2exe (most common, Windows only) and tools/freeze
479847ccabe0 Added hgexecutable support for py2exe/frozen scripts
"Paul Moore <p.f.moore@gmail.com>"
parents: 5659
diff changeset
    23
    (portable, not much used).
479847ccabe0 Added hgexecutable support for py2exe/frozen scripts
"Paul Moore <p.f.moore@gmail.com>"
parents: 5659
diff changeset
    24
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 43019
diff changeset
    25
    return (
44924
6c8384afbf77 resourceutil: fix location of line comments
Manuel Jacob <me@manueljacob.de>
parents: 44241
diff changeset
    26
        pycompat.safehasattr(sys, "frozen")  # new py2exe
6c8384afbf77 resourceutil: fix location of line comments
Manuel Jacob <me@manueljacob.de>
parents: 44241
diff changeset
    27
        or pycompat.safehasattr(sys, "importers")  # old py2exe
6c8384afbf77 resourceutil: fix location of line comments
Manuel Jacob <me@manueljacob.de>
parents: 44241
diff changeset
    28
        or imp.is_frozen("__main__")  # tools/freeze
6c8384afbf77 resourceutil: fix location of line comments
Manuel Jacob <me@manueljacob.de>
parents: 44241
diff changeset
    29
    )
43672
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    30
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    31
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    32
# the location of data files matching the source code
44241
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    33
if mainfrozen() and getattr(sys, "frozen", None) != "macosx_app":
43672
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    34
    # executable version (py2exe) doesn't support __file__
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    35
    datapath = os.path.dirname(pycompat.sysexecutable)
44208
9e367157a990 resourceutil: correct the root path for file based lookup under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44152
diff changeset
    36
    _rootpath = datapath
44219
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    37
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    38
    # The installers store the files outside of library.zip, like
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    39
    # C:\Program Files\Mercurial\defaultrc\*.rc.  This strips the
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    40
    # leading "mercurial." off of the package name, so that these
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    41
    # pseudo resources are found in their directory next to the
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    42
    # executable.
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    43
    def _package_path(package):
44241
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    44
        dirs = package.split(b".")
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    45
        assert dirs[0] == b"mercurial"
44219
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    46
        return os.path.join(_rootpath, *dirs[1:])
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    47
44241
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    48
43672
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    49
else:
9fb85668ee15 util: move definition of datapath to resourceutil
Martin von Zweigbergk <martinvonz@google.com>
parents: 43671
diff changeset
    50
    datapath = os.path.dirname(os.path.dirname(pycompat.fsencode(__file__)))
44027
52f0140c2604 resourceutil: don't limit resources to the `mercurial` package
Matt Harbison <matt_harbison@yahoo.com>
parents: 43955
diff changeset
    51
    _rootpath = os.path.dirname(datapath)
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    52
44219
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    53
    def _package_path(package):
44241
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    54
        return os.path.join(_rootpath, *package.split(b"."))
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    55
44219
aab70b540d3d resourceutil: account for the non-resource-like file hierarchy under py2exe
Matt Harbison <matt_harbison@yahoo.com>
parents: 44208
diff changeset
    56
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    57
try:
45452
dd9e28612468 resourceutil: document when we expect to take the importlib.resouces code path
Martin von Zweigbergk <martinvonz@google.com>
parents: 44924
diff changeset
    58
    # importlib.resources exists from Python 3.7; see fallback in except clause
dd9e28612468 resourceutil: document when we expect to take the importlib.resouces code path
Martin von Zweigbergk <martinvonz@google.com>
parents: 44924
diff changeset
    59
    # further down
43955
75ec46c63657 resourceutil: use `from importlib import resources`
Martin von Zweigbergk <martinvonz@google.com>
parents: 43905
diff changeset
    60
    from importlib import resources
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    61
44028
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    62
    from .. import encoding
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    63
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    64
    # Force loading of the resources module
43955
75ec46c63657 resourceutil: use `from importlib import resources`
Martin von Zweigbergk <martinvonz@google.com>
parents: 43905
diff changeset
    65
    resources.open_binary  # pytype: disable=module-attr
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    66
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    67
    def open_resource(package, name):
43955
75ec46c63657 resourceutil: use `from importlib import resources`
Martin von Zweigbergk <martinvonz@google.com>
parents: 43905
diff changeset
    68
        return resources.open_binary(  # pytype: disable=module-attr
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    69
            pycompat.sysstr(package), pycompat.sysstr(name)
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    70
        )
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    71
44028
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    72
    def is_resource(package, name):
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    73
        return resources.is_resource(
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    74
            pycompat.sysstr(package), encoding.strfromlocal(name)
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    75
        )
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    76
44029
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    77
    def contents(package):
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    78
        for r in resources.contents(pycompat.sysstr(package)):
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    79
            yield encoding.strtolocal(r)
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    80
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    81
43955
75ec46c63657 resourceutil: use `from importlib import resources`
Martin von Zweigbergk <martinvonz@google.com>
parents: 43905
diff changeset
    82
except (ImportError, AttributeError):
45452
dd9e28612468 resourceutil: document when we expect to take the importlib.resouces code path
Martin von Zweigbergk <martinvonz@google.com>
parents: 44924
diff changeset
    83
    # importlib.resources was not found (almost definitely because we're on a
dd9e28612468 resourceutil: document when we expect to take the importlib.resouces code path
Martin von Zweigbergk <martinvonz@google.com>
parents: 44924
diff changeset
    84
    # Python version before 3.7)
43871
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    85
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    86
    def open_resource(package, name):
1390bb81163e help: get helptext/ data from `resources` module if available
Martin von Zweigbergk <martinvonz@google.com>
parents: 43862
diff changeset
    87
        path = os.path.join(_package_path(package), name)
44241
aea79f41ee55 resourceutil: blacken
Augie Fackler <augie@google.com>
parents: 44219
diff changeset
    88
        return open(path, "rb")
44028
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    89
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    90
    def is_resource(package, name):
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    91
        path = os.path.join(_package_path(package), name)
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    92
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    93
        try:
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    94
            return os.path.isfile(pycompat.fsdecode(path))
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    95
        except (IOError, OSError):
42a897bf678c resourceutil: implement `is_resource()`
Matt Harbison <matt_harbison@yahoo.com>
parents: 44027
diff changeset
    96
            return False
44029
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    97
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    98
    def contents(package):
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
    99
        path = pycompat.fsdecode(_package_path(package))
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
   100
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
   101
        for p in os.listdir(path):
bba9149adc14 resourceutil: implement `contents()` to iterate over resources in a package
Matt Harbison <matt_harbison@yahoo.com>
parents: 44028
diff changeset
   102
            yield pycompat.fsencode(p)