annotate tests/generate-working-copy-states.py @ 43786:421ea5772039

copies: split the combination of the copies mapping in its own function In some case, this part take up to 95% of the copy tracing that take about a hundred second. This poor performance comes from the fact we keep duplciating and merging dictionary that are mostly similar. I want to experiment with smarter native code to do this, so I need to isolate the function first.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Wed, 13 Nov 2019 20:42:13 +0100
parents 2372284d9457
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
1 # Helper script used for generating history and working copy files and content.
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
2 # The file's name corresponds to its history. The number of changesets can
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
3 # be specified on the command line. With 2 changesets, files with names like
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
4 # content1_content2_content1-untracked are generated. The first two filename
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
5 # segments describe the contents in the two changesets. The third segment
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
6 # ("content1-untracked") describes the state in the working copy, i.e.
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
7 # the file has content "content1" and is untracked (since it was previously
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
8 # tracked, it has been forgotten).
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
9 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
10 # This script generates the filenames and their content, but it's up to the
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
11 # caller to tell hg about the state.
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
12 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
13 # There are two subcommands:
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
14 # filelist <numchangesets>
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
15 # state <numchangesets> (<changeset>|wc)
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
16 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
17 # Typical usage:
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
18 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
19 # $ python $TESTDIR/generate-working-copy-states.py state 2 1
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
20 # $ hg addremove --similarity 0
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
21 # $ hg commit -m 'first'
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
22 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
23 # $ python $TESTDIR/generate-working-copy-states.py state 2 1
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
24 # $ hg addremove --similarity 0
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
25 # $ hg commit -m 'second'
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
26 #
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
27 # $ python $TESTDIR/generate-working-copy-states.py state 2 wc
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
28 # $ hg addremove --similarity 0
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
29 # $ hg forget *_*_*-untracked
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
30 # $ rm *_*_missing-*
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
31
28725
3cf1995dbdd5 py3: use print_function in generate-working-copy-states.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 27295
diff changeset
32 from __future__ import absolute_import, print_function
27295
a327a24acfea tests: use absolute_import in generate-working-copy-states.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23494
diff changeset
33
a327a24acfea tests: use absolute_import in generate-working-copy-states.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23494
diff changeset
34 import os
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
35 import sys
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
36
23446
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
37 # Generates pairs of (filename, contents), where 'contents' is a list
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
38 # describing the file's content at each revision (or in the working copy).
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
39 # At each revision, it is either None or the file's actual content. When not
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
40 # None, it may be either new content or the same content as an earlier
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
41 # revisions, so all of (modified,clean,added,removed) can be tested.
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
42 def generatestates(maxchangesets, parentcontents):
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
43 depth = len(parentcontents)
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
44 if depth == maxchangesets + 1:
36378
27ab9264dd61 py3: make sure we use bytes in generate-working-copy-states.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32291
diff changeset
45 for tracked in (b'untracked', b'tracked'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
46 filename = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
47 b"_".join(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
48 [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
49 (content is None and b'missing' or content)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
50 for content in parentcontents
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
51 ]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
52 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
53 + b"-"
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
54 + tracked
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
55 )
23446
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
56 yield (filename, parentcontents)
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
57 else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
58 for content in {None, b'content' + (b"%d" % (depth + 1))} | set(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
59 parentcontents
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
60 ):
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
61 for combination in generatestates(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
62 maxchangesets, parentcontents + [content]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
63 ):
23446
e51027c85dcd generate-working-copy-states: generalize for depth
Martin von Zweigbergk <martinvonz@google.com>
parents: 23445
diff changeset
64 yield combination
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
65
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 36786
diff changeset
66
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
67 # retrieve the command line arguments
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
68 target = sys.argv[1]
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
69 maxchangesets = int(sys.argv[2])
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
70 if target == 'state':
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
71 depth = sys.argv[3]
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
72
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
73 # sort to make sure we have stable output
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
74 combinations = sorted(generatestates(maxchangesets, []))
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
75
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
76 # compute file content
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
77 content = []
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
78 for filename, states in combinations:
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
79 if target == 'filelist':
36786
ed46d48453e8 py3: drop b'' from generate-working-copy-states.py output
Yuya Nishihara <yuya@tcha.org>
parents: 36378
diff changeset
80 print(filename.decode('ascii'))
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
81 elif target == 'state':
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
82 if depth == 'wc':
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
83 # Make sure there is content so the file gets written and can be
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
84 # tracked. It will be deleted outside of this script.
36378
27ab9264dd61 py3: make sure we use bytes in generate-working-copy-states.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32291
diff changeset
85 content.append((filename, states[maxchangesets] or b'TOBEDELETED'))
23447
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
86 else:
815e76a45b24 generate-working-copy-states: accept depth arguments on command line
Martin von Zweigbergk <martinvonz@google.com>
parents: 23446
diff changeset
87 content.append((filename, states[int(depth) - 1]))
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
88 else:
28725
3cf1995dbdd5 py3: use print_function in generate-working-copy-states.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 27295
diff changeset
89 print("unknown target:", target, file=sys.stderr)
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
90 sys.exit(1)
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
91
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
92 # write actual content
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
93 for filename, data in content:
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
94 if data is not None:
23494
3849b89459b0 generate-working-copy-states: open() in binary mode when writing content
Matt Harbison <matt_harbison@yahoo.com>
parents: 23447
diff changeset
95 f = open(filename, 'wb')
36378
27ab9264dd61 py3: make sure we use bytes in generate-working-copy-states.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32291
diff changeset
96 f.write(data + b'\n')
23195
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
97 f.close()
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
98 elif os.path.exists(filename):
29977b315be1 test-revert: move embedded script to its own file
Martin von Zweigbergk <martinvonz@google.com>
parents:
diff changeset
99 os.remove(filename)