annotate mercurial/scmwindows.py @ 47120:7109a38830c9

dirstate-tree: Fold "tracked descendants" counter update in main walk For the purpose of implementing `has_tracked_dir` (which means "has tracked descendants) without an expensive sub-tree traversal, we maintaing a counter of tracked descendants on each "directory" node of the tree-shaped dirstate. Before this changeset, mutating or inserting a node at a given path would involve: * Walking the tree from root through ancestors to find the node or the spot where to insert it * Looking at the previous node if any to decide what counter update is needed * Performing any node mutation * Walking the tree *again* to update counters in ancestor nodes When profiling `hg status` on a large repo, this second walk takes times while loading a the dirstate from disk. It turns out we have enough information to decide before he first tree walk what counter update is needed. This changeset merges the two walks, gaining ~10% of the total time for `hg update` (in the same hyperfine benchmark as the previous changeset). --- Profiling was done by compiling with this `.cargo/config`: [profile.release] debug = true then running with: py-spy record -r 500 -n -o /tmp/hg.json --format speedscope -- \ ./hg status -R $REPO --config experimental.dirstate-tree.in-memory=1 then visualizing the recorded JSON file in https://www.speedscope.app/ Differential Revision: https://phab.mercurial-scm.org/D10554
author Simon Sapin <simon.sapin@octobus.net>
date Fri, 30 Apr 2021 14:22:14 +0200
parents 224af78021de
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
27481
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
1 from __future__ import absolute_import
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
2
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
3 import os
27481
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
4
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
5 from . import (
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30612
diff changeset
6 encoding,
30612
d623cc6b3742 py3: replace os.pathsep with pycompat.ospathsep
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30314
diff changeset
7 pycompat,
27481
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
8 util,
30309
4b1af1c867fa scmutil: move util.termwidth()
Yuya Nishihara <yuya@tcha.org>
parents: 29760
diff changeset
9 win32,
27481
029f02757c20 scmwindows: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26625
diff changeset
10 )
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
11
29760
3df9f780c90e py3: conditionalize _winreg import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 27481
diff changeset
12 try:
43768
fe73ec69350e windows: suppress pytype warnings for Windows imports and functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 43077
diff changeset
13 import _winreg as winreg # pytype: disable=import-error
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
14
29760
3df9f780c90e py3: conditionalize _winreg import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 27481
diff changeset
15 winreg.CloseKey
3df9f780c90e py3: conditionalize _winreg import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 27481
diff changeset
16 except ImportError:
43768
fe73ec69350e windows: suppress pytype warnings for Windows imports and functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 43077
diff changeset
17 # py2 only
fe73ec69350e windows: suppress pytype warnings for Windows imports and functions
Matt Harbison <matt_harbison@yahoo.com>
parents: 43077
diff changeset
18 import winreg # pytype: disable=import-error
29760
3df9f780c90e py3: conditionalize _winreg import
Pulkit Goyal <7895pulkit@gmail.com>
parents: 27481
diff changeset
19
32078
bf5e13e38390 pager: use less as a fallback on Unix
Yuya Nishihara <yuya@tcha.org>
parents: 30637
diff changeset
20 # MS-DOS 'more' is the only pager available by default on Windows.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
21 fallbackpager = b'more'
32078
bf5e13e38390 pager: use less as a fallback on Unix
Yuya Nishihara <yuya@tcha.org>
parents: 30637
diff changeset
22
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
23
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
24 def systemrcpath():
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
25 '''return default os-specific hgrc search path'''
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
26 rcpath = []
37095
e24802ea8dbd rcutil: directly call win32.executablepath()
Yuya Nishihara <yuya@tcha.org>
parents: 32208
diff changeset
27 filename = win32.executablepath()
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
28 # Use mercurial.ini found in directory with hg.exe
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
29 progrc = os.path.join(os.path.dirname(filename), b'mercurial.ini')
26625
adae8928fe09 windows: read all global config files, not just the first (issue4491) (BC)
Mads Kiilerich <madski@unity3d.com>
parents: 22583
diff changeset
30 rcpath.append(progrc)
43925
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
31
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
32 def _processdir(progrcd):
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
33 if os.path.isdir(progrcd):
45824
9ac96b9fa76e config: read system hgrc in lexicographical order
Martin von Zweigbergk <martinvonz@google.com>
parents: 43951
diff changeset
34 for f, kind in sorted(util.listdir(progrcd)):
43925
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
35 if f.endswith(b'.rc'):
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
36 rcpath.append(os.path.join(progrcd, f))
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
37
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
38 # Use hgrc.d found in directory with hg.exe
43925
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
39 _processdir(os.path.join(os.path.dirname(filename), b'hgrc.d'))
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
40
43951
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
41 # treat a PROGRAMDATA directory as equivalent to /etc/mercurial
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
42 programdata = encoding.environ.get(b'PROGRAMDATA')
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
43 if programdata:
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
44 programdata = os.path.join(programdata, b'Mercurial')
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
45 _processdir(os.path.join(programdata, b'hgrc.d'))
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
46
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
47 ini = os.path.join(programdata, b'mercurial.ini')
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
48 if os.path.isfile(ini):
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
49 rcpath.append(ini)
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
50
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
51 ini = os.path.join(programdata, b'hgrc')
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
52 if os.path.isfile(ini):
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
53 rcpath.append(ini)
1ccf340acf14 windows: add a global equivalent to /etc/mercurial for *.rc processing
Matt Harbison <matt_harbison@yahoo.com>
parents: 43925
diff changeset
54
43923
9a3ac902d597 windows: clarify a comment about the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43793
diff changeset
55 # next look for a system rcpath in the registry
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
56 value = util.lookupreg(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
57 b'SOFTWARE\\Mercurial', None, winreg.HKEY_LOCAL_MACHINE
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
58 )
43924
fa3835a15a17 windows: don't return early from building the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43923
diff changeset
59 if value and isinstance(value, bytes):
fa3835a15a17 windows: don't return early from building the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43923
diff changeset
60 value = util.localpath(value)
fa3835a15a17 windows: don't return early from building the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43923
diff changeset
61 for p in value.split(pycompat.ospathsep):
fa3835a15a17 windows: don't return early from building the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43923
diff changeset
62 if p.lower().endswith(b'mercurial.ini'):
fa3835a15a17 windows: don't return early from building the hgrc search path
Matt Harbison <matt_harbison@yahoo.com>
parents: 43923
diff changeset
63 rcpath.append(p)
43925
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
64 else:
7929bb58146f windows: factor the hgrc directory scan into a function
Matt Harbison <matt_harbison@yahoo.com>
parents: 43924
diff changeset
65 _processdir(p)
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
66 return rcpath
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
67
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
68
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
69 def userrcpath():
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
70 '''return os-specific hgrc search path to the user dir'''
46093
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
71 home = _legacy_expanduser(b'~')
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
72 path = [os.path.join(home, b'mercurial.ini'), os.path.join(home, b'.hgrc')]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
73 userprofile = encoding.environ.get(b'USERPROFILE')
22583
23c995ed466b config: don't read the same config file twice
Mads Kiilerich <madski@unity3d.com>
parents: 18712
diff changeset
74 if userprofile and userprofile != home:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
75 path.append(os.path.join(userprofile, b'mercurial.ini'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
76 path.append(os.path.join(userprofile, b'.hgrc'))
18690
4c6f7f0dadab scmutil: split platform-specific bits into their own modules
Kevin Bullock <kbullock@ringworld.org>
parents:
diff changeset
77 return path
30309
4b1af1c867fa scmutil: move util.termwidth()
Yuya Nishihara <yuya@tcha.org>
parents: 29760
diff changeset
78
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 37095
diff changeset
79
46093
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
80 def _legacy_expanduser(path):
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
81 """Expand ~ and ~user constructs in the pre 3.8 style"""
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
82
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
83 # Python 3.8+ changed the expansion of '~' from HOME to USERPROFILE. See
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
84 # https://bugs.python.org/issue36264. It also seems to capitalize the drive
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
85 # letter, as though it was processed through os.path.realpath().
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
86 if not path.startswith(b'~'):
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
87 return path
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
88
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
89 i, n = 1, len(path)
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
90 while i < n and path[i] not in b'\\/':
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
91 i += 1
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
92
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
93 if b'HOME' in encoding.environ:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
94 userhome = encoding.environ[b'HOME']
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
95 elif b'USERPROFILE' in encoding.environ:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
96 userhome = encoding.environ[b'USERPROFILE']
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
97 elif b'HOMEPATH' not in encoding.environ:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
98 return path
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
99 else:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
100 try:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
101 drive = encoding.environ[b'HOMEDRIVE']
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
102 except KeyError:
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
103 drive = b''
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
104 userhome = os.path.join(drive, encoding.environ[b'HOMEPATH'])
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
105
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
106 if i != 1: # ~user
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
107 userhome = os.path.join(os.path.dirname(userhome), path[1:i])
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
108
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
109 return userhome + path[i:]
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
110
224af78021de windows: continue looking at `%HOME%` for user config files with py3.8+
Matt Harbison <matt_harbison@yahoo.com>
parents: 45824
diff changeset
111
30314
365812902904 scmutil: extend termwidth() to return terminal height, renamed to termsize()
Yuya Nishihara <yuya@tcha.org>
parents: 30310
diff changeset
112 def termsize(ui):
365812902904 scmutil: extend termwidth() to return terminal height, renamed to termsize()
Yuya Nishihara <yuya@tcha.org>
parents: 30310
diff changeset
113 return win32.termsize()