Mercurial > hg
annotate contrib/synthrepo.py @ 43104:74802979dd9d
py3: define and use pycompat.itervalues()
.itervalues() only exists on Python 2. Python 3's equivalent is
.values(). But we don't want to blindly use .values() everywhere
because on Python 2, it will create a list, which will have performance
implications.
This commit introduces pycompat.itervalues() which will call the appropriate
method on the passed object. We update all callers of obj.itervalues()
to pycompat.itervalues(obj) instead.
With this commit, the only source tranforming remaining is for
iteritems(). Victory is near...
Differential Revision: https://phab.mercurial-scm.org/D7013
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sun, 06 Oct 2019 17:59:15 -0400 |
parents | 2372284d9457 |
children | 89a2afe31e82 |
rev | line source |
---|---|
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
1 # synthrepo.py - repo synthesis |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
2 # |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
3 # Copyright 2012 Facebook |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
4 # |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
7 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
8 '''synthesize structurally interesting change history |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
9 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
10 This extension is useful for creating a repository with properties |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
11 that are statistically similar to an existing repository. During |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
12 analysis, a simple probability table is constructed from the history |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
13 of an existing repository. During synthesis, these properties are |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
14 reconstructed. |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
15 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
16 Properties that are analyzed and synthesized include the following: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
17 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
18 - Lines added or removed when an existing file is modified |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
19 - Number and sizes of files added |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
20 - Number of files removed |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
21 - Line lengths |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
22 - Topological distance to parent changeset(s) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
23 - Probability of a commit being a merge |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
24 - Probability of a newly added file being added to a new directory |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
25 - Interarrival time, and time zone, of commits |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
26 - Number of files in each directory |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
27 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
28 A few obvious properties that are not currently handled realistically: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
29 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
30 - Merges are treated as regular commits with two parents, which is not |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
31 realistic |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
32 - Modifications are not treated as operations on hunks of lines, but |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
33 as insertions and deletions of randomly chosen single lines |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
34 - Committer ID (always random) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
35 - Executability of files |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
36 - Symlinks and binary files are ignored |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
37 ''' |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
38 |
28563
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
39 from __future__ import absolute_import |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
40 import bisect |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
41 import collections |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
42 import itertools |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
43 import json |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
44 import os |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
45 import random |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
46 import sys |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
47 import time |
29205
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
48 |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
49 from mercurial.i18n import _ |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
50 from mercurial.node import ( |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
51 nullid, |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
52 nullrev, |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
53 short, |
a0939666b836
py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents:
28563
diff
changeset
|
54 ) |
28563
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
55 from mercurial import ( |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
56 context, |
38588
1c93e0237a24
diffutil: move the module out of utils package
Yuya Nishihara <yuya@tcha.org>
parents:
38587
diff
changeset
|
57 diffutil, |
28563
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
58 error, |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
59 hg, |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
60 patch, |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
61 pycompat, |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
32291
diff
changeset
|
62 registrar, |
28563
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
63 scmutil, |
62250a48dc7f
contrib: synthrepo use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
26587
diff
changeset
|
64 ) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
65 from mercurial.utils import dateutil |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
66 |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29216
diff
changeset
|
67 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
25186
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24306
diff
changeset
|
68 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24306
diff
changeset
|
69 # be specifying the version(s) of Mercurial they are tested with, or |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24306
diff
changeset
|
70 # leave the attribute unspecified. |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29216
diff
changeset
|
71 testedwith = 'ships-with-hg-core' |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
72 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
73 cmdtable = {} |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
32291
diff
changeset
|
74 command = registrar.command(cmdtable) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
75 |
32291
bd872f64a8ba
cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents:
29841
diff
changeset
|
76 newfile = {'new fi', 'rename', 'copy f', 'copy t'} |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
77 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
78 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
79 def zerodict(): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
80 return collections.defaultdict(lambda: 0) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
81 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
82 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
83 def roundto(x, k): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
84 if x > k * 2: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
85 return int(round(x / float(k)) * k) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
86 return int(round(x)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
87 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
88 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
89 def parsegitdiff(lines): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
90 filename, mar, lineadd, lineremove = None, None, zerodict(), 0 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
91 binary = False |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
92 for line in lines: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
93 start = line[:6] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
94 if start == 'diff -': |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
95 if filename: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
96 yield filename, mar, lineadd, lineremove, binary |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
97 mar, lineadd, lineremove, binary = 'm', zerodict(), 0, False |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
98 filename = patch.gitre.match(line).group(1) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
99 elif start in newfile: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
100 mar = 'a' |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
101 elif start == 'GIT bi': |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
102 binary = True |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
103 elif start == 'delete': |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
104 mar = 'r' |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
105 elif start: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
106 s = start[0] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
107 if s == '-' and not line.startswith('--- '): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
108 lineremove += 1 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
109 elif s == '+' and not line.startswith('+++ '): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
110 lineadd[roundto(len(line) - 1, 5)] += 1 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
111 if filename: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
112 yield filename, mar, lineadd, lineremove, binary |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
113 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
114 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
115 @command( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
116 'analyze', |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
117 [ |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
118 ('o', 'output', '', _('write output to given file'), _('FILE')), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
119 ('r', 'rev', [], _('analyze specified revisions'), _('REV')), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
120 ], |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
121 _('hg analyze'), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
122 optionalrepo=True, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
123 ) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
124 def analyze(ui, repo, *revs, **opts): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
125 '''create a simple model of a repository to use for later synthesis |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
126 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
127 This command examines every changeset in the given range (or all |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
128 of history if none are specified) and creates a simple statistical |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
129 model of the history of the repository. It also measures the directory |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
130 structure of the repository as checked out. |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
131 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
132 The model is written out to a JSON file, and can be used by |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
133 :hg:`synthesize` to create or augment a repository with synthetic |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
134 commits that have a structure that is statistically similar to the |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
135 analyzed repository. |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
136 ''' |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
137 root = repo.root |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
138 if not root.endswith(os.path.sep): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
139 root += os.path.sep |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
140 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
141 revs = list(revs) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
142 revs.extend(opts['rev']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
143 if not revs: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
144 revs = [':'] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
145 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
146 output = opts['output'] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
147 if not output: |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
148 output = os.path.basename(root) + '.json' |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
149 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
150 if output == '-': |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
151 fp = sys.stdout |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
152 else: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
153 fp = open(output, 'w') |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
154 |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
155 # Always obtain file counts of each directory in the given root directory. |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
156 def onerror(e): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
157 ui.warn(_('error walking directory structure: %s\n') % e) |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
158 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
159 dirs = {} |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
160 rootprefixlen = len(root) |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
161 for dirpath, dirnames, filenames in os.walk(root, onerror=onerror): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
162 dirpathfromroot = dirpath[rootprefixlen:] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
163 dirs[dirpathfromroot] = len(filenames) |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
164 if '.hg' in dirnames: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
165 dirnames.remove('.hg') |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
166 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
167 lineschanged = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
168 children = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
169 p1distance = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
170 p2distance = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
171 linesinfilesadded = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
172 fileschanged = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
173 filesadded = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
174 filesremoved = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
175 linelengths = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
176 interarrival = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
177 parents = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
178 dirsadded = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
179 tzoffset = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
180 |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
181 # If a mercurial repo is available, also model the commit history. |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
182 if repo: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
183 revs = scmutil.revrange(repo, revs) |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
184 revs.sort() |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
185 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
186 progress = ui.makeprogress( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
187 _('analyzing'), unit=_('changesets'), total=len(revs) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
188 ) |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
189 for i, rev in enumerate(revs): |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
190 progress.update(i) |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
191 ctx = repo[rev] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
192 pl = ctx.parents() |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
193 pctx = pl[0] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
194 prev = pctx.rev() |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
195 children[prev] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
196 p1distance[rev - prev] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
197 parents[len(pl)] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
198 tzoffset[ctx.date()[1]] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
199 if len(pl) > 1: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
200 p2distance[rev - pl[1].rev()] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
201 if prev == rev - 1: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
202 lastctx = pctx |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
203 else: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
204 lastctx = repo[rev - 1] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
205 if lastctx.rev() != nullrev: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
206 timedelta = ctx.date()[0] - lastctx.date()[0] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
207 interarrival[roundto(timedelta, 300)] += 1 |
38587
b62000a28812
diffutil: remove diffopts() in favor of diffallopts()
Yuya Nishihara <yuya@tcha.org>
parents:
38584
diff
changeset
|
208 diffopts = diffutil.diffallopts(ui, {'git': True}) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
209 diff = sum( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
210 (d.splitlines() for d in ctx.diff(pctx, opts=diffopts)), [] |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
211 ) |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
212 fileadds, diradds, fileremoves, filechanges = 0, 0, 0, 0 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
213 for filename, mar, lineadd, lineremove, isbin in parsegitdiff(diff): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
214 if isbin: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
215 continue |
43104
74802979dd9d
py3: define and use pycompat.itervalues()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
43076
diff
changeset
|
216 added = sum(pycompat.itervalues(lineadd), 0) |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
217 if mar == 'm': |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
218 if added and lineremove: |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
219 lineschanged[ |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
220 roundto(added, 5), roundto(lineremove, 5) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
221 ] += 1 |
22709
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
222 filechanges += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
223 elif mar == 'a': |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
224 fileadds += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
225 if '/' in filename: |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
226 filedir = filename.rsplit('/', 1)[0] |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
227 if filedir not in pctx.dirs(): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
228 diradds += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
229 linesinfilesadded[roundto(added, 5)] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
230 elif mar == 'r': |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
231 fileremoves += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
232 for length, count in lineadd.iteritems(): |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
233 linelengths[length] += count |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
234 fileschanged[filechanges] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
235 filesadded[fileadds] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
236 dirsadded[diradds] += 1 |
889789a2ca9f
contrib/synthrepo: walk a repo's directory structure during analysis
Mike Edgar <adgar@google.com>
parents:
22708
diff
changeset
|
237 filesremoved[fileremoves] += 1 |
38409
ce65c25dc161
synthrepo: close progress topics
Martin von Zweigbergk <martinvonz@google.com>
parents:
38408
diff
changeset
|
238 progress.complete() |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
239 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
240 invchildren = zerodict() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
241 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
242 for rev, count in children.iteritems(): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
243 invchildren[count] += 1 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
244 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
245 if output != '-': |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
246 ui.status(_('writing output to %s\n') % output) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
247 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
248 def pronk(d): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
249 return sorted(d.iteritems(), key=lambda x: x[1], reverse=True) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
250 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
251 json.dump( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
252 { |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
253 'revs': len(revs), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
254 'initdirs': pronk(dirs), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
255 'lineschanged': pronk(lineschanged), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
256 'children': pronk(invchildren), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
257 'fileschanged': pronk(fileschanged), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
258 'filesadded': pronk(filesadded), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
259 'linesinfilesadded': pronk(linesinfilesadded), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
260 'dirsadded': pronk(dirsadded), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
261 'filesremoved': pronk(filesremoved), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
262 'linelengths': pronk(linelengths), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
263 'parents': pronk(parents), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
264 'p1distance': pronk(p1distance), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
265 'p2distance': pronk(p2distance), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
266 'interarrival': pronk(interarrival), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
267 'tzoffset': pronk(tzoffset), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
268 }, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
269 fp, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
270 ) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
271 fp.close() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
272 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
273 |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
274 @command( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
275 'synthesize', |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
276 [ |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
277 ('c', 'count', 0, _('create given number of commits'), _('COUNT')), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
278 ('', 'dict', '', _('path to a dictionary of words'), _('FILE')), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
279 ('', 'initfiles', 0, _('initial file count to create'), _('COUNT')), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
280 ], |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
281 _('hg synthesize [OPTION].. DESCFILE'), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
282 ) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
283 def synthesize(ui, repo, descpath, **opts): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
284 '''synthesize commits based on a model of an existing repository |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
285 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
286 The model must have been generated by :hg:`analyze`. Commits will |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
287 be generated randomly according to the probabilities described in |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
288 the model. If --initfiles is set, the repository will be seeded with |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
289 the given number files following the modeled repository's directory |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
290 structure. |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
291 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
292 When synthesizing new content, commit descriptions, and user |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
293 names, words will be chosen randomly from a dictionary that is |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
294 presumed to contain one word per line. Use --dict to specify the |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
295 path to an alternate dictionary to use. |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
296 ''' |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
297 try: |
17887
0e2846b2482c
url: use open and not url.open for local files (issue3624)
Siddharth Agarwal <sid0@fb.com>
parents:
17734
diff
changeset
|
298 fp = hg.openpath(ui, descpath) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25186
diff
changeset
|
299 except Exception as err: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25660
diff
changeset
|
300 raise error.Abort('%s: %s' % (descpath, err[0].strerror)) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
301 desc = json.load(fp) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
302 fp.close() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
303 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
304 def cdf(l): |
18047
9196638b08ce
synthrepo: do not crash if a list is empty
Bryan O'Sullivan <bryano@fb.com>
parents:
17887
diff
changeset
|
305 if not l: |
9196638b08ce
synthrepo: do not crash if a list is empty
Bryan O'Sullivan <bryano@fb.com>
parents:
17887
diff
changeset
|
306 return [], [] |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
307 vals, probs = zip(*sorted(l, key=lambda x: x[1], reverse=True)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
308 t = float(sum(probs, 0)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
309 s, cdfs = 0, [] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
310 for v in probs: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
311 s += v |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
312 cdfs.append(s / t) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
313 return vals, cdfs |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
314 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
315 lineschanged = cdf(desc['lineschanged']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
316 fileschanged = cdf(desc['fileschanged']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
317 filesadded = cdf(desc['filesadded']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
318 dirsadded = cdf(desc['dirsadded']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
319 filesremoved = cdf(desc['filesremoved']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
320 linelengths = cdf(desc['linelengths']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
321 parents = cdf(desc['parents']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
322 p1distance = cdf(desc['p1distance']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
323 p2distance = cdf(desc['p2distance']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
324 interarrival = cdf(desc['interarrival']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
325 linesinfilesadded = cdf(desc['linesinfilesadded']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
326 tzoffset = cdf(desc['tzoffset']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
327 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
328 dictfile = opts.get('dict') or '/usr/share/dict/words' |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
329 try: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
330 fp = open(dictfile, 'rU') |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25186
diff
changeset
|
331 except IOError as err: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25660
diff
changeset
|
332 raise error.Abort('%s: %s' % (dictfile, err.strerror)) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
333 words = fp.read().splitlines() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
334 fp.close() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
335 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
336 initdirs = {} |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
337 if desc['initdirs']: |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
338 for k, v in desc['initdirs']: |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
339 initdirs[k.encode('utf-8').replace('.hg', '_hg')] = v |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
340 initdirs = renamedirs(initdirs, words) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
341 initdirscdf = cdf(initdirs) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
342 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
343 def pick(cdf): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
344 return cdf[0][bisect.bisect_left(cdf[1], random.random())] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
345 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
346 def pickpath(): |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
347 return os.path.join(pick(initdirscdf), random.choice(words)) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
348 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
349 def makeline(minimum=0): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
350 total = max(minimum, pick(linelengths)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
351 c, l = 0, [] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
352 while c < total: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
353 w = random.choice(words) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
354 c += len(w) + 1 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
355 l.append(w) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
356 return ' '.join(l) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
357 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
358 wlock = repo.wlock() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
359 lock = repo.lock() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
360 |
32291
bd872f64a8ba
cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents:
29841
diff
changeset
|
361 nevertouch = {'.hgsub', '.hgignore', '.hgtags'} |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
362 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
363 _synthesizing = _('synthesizing') |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
364 _files = _('initial files') |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
365 _changesets = _('changesets') |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
366 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
367 # Synthesize a single initial revision adding files to the repo according |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
368 # to the modeled directory structure. |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
369 initcount = int(opts['initfiles']) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
370 if initcount and initdirs: |
41398
2ff8994ac71d
cleanup: use repo['.'] instead of repo[None].p1()
Martin von Zweigbergk <martinvonz@google.com>
parents:
41397
diff
changeset
|
371 pctx = repo['.'] |
23778
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
372 dirs = set(pctx.dirs()) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
373 files = {} |
23778
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
374 |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
375 def validpath(path): |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
376 # Don't pick filenames which are already directory names. |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
377 if path in dirs: |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
378 return False |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
379 # Don't pick directories which were used as file names. |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
380 while path: |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
381 if path in files: |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
382 return False |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
383 path = os.path.dirname(path) |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
384 return True |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
385 |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
386 progress = ui.makeprogress(_synthesizing, unit=_files, total=initcount) |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
387 for i in pycompat.xrange(0, initcount): |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
388 progress.update(i) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
389 |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
390 path = pickpath() |
23778
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
391 while not validpath(path): |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
392 path = pickpath() |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
393 data = '%s contents\n' % path |
35398
2123e7629ec0
synthrepo: create filectx instance in 'filectxfn' callback
Martin von Zweigbergk <martinvonz@google.com>
parents:
34023
diff
changeset
|
394 files[path] = data |
23778
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
395 dir = os.path.dirname(path) |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
396 while dir and dir not in dirs: |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
397 dirs.add(dir) |
a5dbec255f14
synthrepo: new filenames must not also be new directories, and vice-versa
Mike Edgar <adgar@google.com>
parents:
23235
diff
changeset
|
398 dir = os.path.dirname(dir) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
399 |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
400 def filectxfn(repo, memctx, path): |
35400
8a0cac20a1ad
memfilectx: make changectx argument mandatory in constructor (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
35398
diff
changeset
|
401 return context.memfilectx(repo, memctx, path, files[path]) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
402 |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
403 progress.complete() |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
404 message = 'synthesized wide repo with %d files' % (len(files),) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
405 mc = context.memctx( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
406 repo, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
407 [pctx.node(), nullid], |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
408 message, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
409 files, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
410 filectxfn, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
411 ui.username(), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
412 '%d %d' % dateutil.makedate(), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
413 ) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
414 initnode = mc.commit() |
24306
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
415 if ui.debugflag: |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
416 hexfn = hex |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
417 else: |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
418 hexfn = short |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
419 ui.status( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
420 _('added commit %s with %d files\n') % (hexfn(initnode), len(files)) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
421 ) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
422 |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
423 # Synthesize incremental revisions to the repository, adding repo depth. |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
424 count = int(opts['count']) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
425 heads = set(map(repo.changelog.rev, repo.heads())) |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
426 progress = ui.makeprogress(_synthesizing, unit=_changesets, total=count) |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
427 for i in pycompat.xrange(count): |
38408
6540333acb95
synthrepo: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents:
36607
diff
changeset
|
428 progress.update(i) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
429 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
430 node = repo.changelog.node |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
431 revs = len(repo) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
432 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
433 def pickhead(heads, distance): |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
434 if heads: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
435 lheads = sorted(heads) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
436 rev = revs - min(pick(distance), revs) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
437 if rev < lheads[-1]: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
438 rev = lheads[bisect.bisect_left(lheads, rev)] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
439 else: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
440 rev = lheads[-1] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
441 return rev, node(rev) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
442 return nullrev, nullid |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
443 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
444 r1 = revs - min(pick(p1distance), revs) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
445 p1 = node(r1) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
446 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
447 # the number of heads will grow without bound if we use a pure |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
448 # model, so artificially constrain their proliferation |
22472
2e2577b0ccbe
contrib/synthrepo: only generate 2 parents if model contains merges
Mike Edgar <adgar@google.com>
parents:
22446
diff
changeset
|
449 toomanyheads = len(heads) > random.randint(1, 20) |
2e2577b0ccbe
contrib/synthrepo: only generate 2 parents if model contains merges
Mike Edgar <adgar@google.com>
parents:
22446
diff
changeset
|
450 if p2distance[0] and (pick(parents) == 2 or toomanyheads): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
451 r2, p2 = pickhead(heads.difference([r1]), p2distance) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
452 else: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
453 r2, p2 = nullrev, nullid |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
454 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
455 pl = [p1, p2] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
456 pctx = repo[r1] |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
457 mf = pctx.manifest() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
458 mfk = mf.keys() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
459 changes = {} |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
460 if mfk: |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
461 for __ in pycompat.xrange(pick(fileschanged)): |
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
462 for __ in pycompat.xrange(10): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
463 fctx = pctx.filectx(random.choice(mfk)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
464 path = fctx.path() |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
465 if not ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
466 path in nevertouch |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
467 or fctx.isbinary() |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
468 or 'l' in fctx.flags() |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
469 ): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
470 break |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
471 lines = fctx.data().splitlines() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
472 add, remove = pick(lineschanged) |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
473 for __ in pycompat.xrange(remove): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
474 if not lines: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
475 break |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
476 del lines[random.randrange(0, len(lines))] |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
477 for __ in pycompat.xrange(add): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
478 lines.insert(random.randint(0, len(lines)), makeline()) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
479 path = fctx.path() |
35398
2123e7629ec0
synthrepo: create filectx instance in 'filectxfn' callback
Martin von Zweigbergk <martinvonz@google.com>
parents:
34023
diff
changeset
|
480 changes[path] = '\n'.join(lines) + '\n' |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
481 for __ in pycompat.xrange(pick(filesremoved)): |
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
482 for __ in pycompat.xrange(10): |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
483 path = random.choice(mfk) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
484 if path not in changes: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
485 break |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
486 if filesadded: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
487 dirs = list(pctx.dirs()) |
23235
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
488 dirs.insert(0, '') |
43004
c07812bdd568
synthrepo: use pycompat.xrange
Gregory Szorc <gregory.szorc@gmail.com>
parents:
41398
diff
changeset
|
489 for __ in pycompat.xrange(pick(filesadded)): |
23235
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
490 pathstr = '' |
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
491 while pathstr in dirs: |
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
492 path = [random.choice(dirs)] |
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
493 if pick(dirsadded): |
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
494 path.append(random.choice(words)) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
495 path.append(random.choice(words)) |
23235
4cdc3e2810b9
synthrepo: when adding files, ensure new path is not a directory
Mike Edgar <adgar@google.com>
parents:
23234
diff
changeset
|
496 pathstr = '/'.join(filter(None, path)) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
497 data = ( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
498 '\n'.join( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
499 makeline() |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
500 for __ in pycompat.xrange(pick(linesinfilesadded)) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
501 ) |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
502 + '\n' |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
503 ) |
35398
2123e7629ec0
synthrepo: create filectx instance in 'filectxfn' callback
Martin von Zweigbergk <martinvonz@google.com>
parents:
34023
diff
changeset
|
504 changes[pathstr] = data |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
505 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
506 def filectxfn(repo, memctx, path): |
35398
2123e7629ec0
synthrepo: create filectx instance in 'filectxfn' callback
Martin von Zweigbergk <martinvonz@google.com>
parents:
34023
diff
changeset
|
507 if path not in changes: |
2123e7629ec0
synthrepo: create filectx instance in 'filectxfn' callback
Martin von Zweigbergk <martinvonz@google.com>
parents:
34023
diff
changeset
|
508 return None |
35400
8a0cac20a1ad
memfilectx: make changectx argument mandatory in constructor (API)
Martin von Zweigbergk <martinvonz@google.com>
parents:
35398
diff
changeset
|
509 return context.memfilectx(repo, memctx, path, changes[path]) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
510 |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
511 if not changes: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
512 continue |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
513 if revs: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
514 date = repo['tip'].date()[0] + pick(interarrival) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
515 else: |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
516 date = time.time() - (86400 * count) |
23234
944d6cfbe166
synthrepo: synthesized dates must be positive, fit in 32-bit signed ints
Mike Edgar <adgar@google.com>
parents:
22709
diff
changeset
|
517 # dates in mercurial must be positive, fit in 32-bit signed integers. |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
518 date = min(0x7FFFFFFF, max(0, date)) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
519 user = random.choice(words) + '@' + random.choice(words) |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
520 mc = context.memctx( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
521 repo, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
522 pl, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
523 makeline(minimum=2), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
524 sorted(changes), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
525 filectxfn, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
526 user, |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
527 '%d %d' % (date, pick(tzoffset)), |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
528 ) |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
529 newnode = mc.commit() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
530 heads.add(repo.changelog.rev(newnode)) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
531 heads.discard(r1) |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
532 heads.discard(r2) |
38409
ce65c25dc161
synthrepo: close progress topics
Martin von Zweigbergk <martinvonz@google.com>
parents:
38408
diff
changeset
|
533 progress.complete() |
17734
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
534 |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
535 lock.release() |
619068c280fd
contrib: add a commit synthesizer for reproducing scaling problems
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
536 wlock.release() |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
537 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
538 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
539 def renamedirs(dirs, words): |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
540 '''Randomly rename the directory names in the per-dir file count dict.''' |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
541 wordgen = itertools.cycle(words) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
542 replacements = {'': ''} |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
543 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
544 def rename(dirpath): |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
545 '''Recursively rename the directory and all path prefixes. |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
546 |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
547 The mapping from path to renamed path is stored for all path prefixes |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
548 as in dynamic programming, ensuring linear runtime and consistent |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
549 renaming regardless of iteration order through the model. |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
550 ''' |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
551 if dirpath in replacements: |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
552 return replacements[dirpath] |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
553 head, _ = os.path.split(dirpath) |
24306
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
554 if head: |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
555 head = rename(head) |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
556 else: |
6ddc86eedc3b
style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents:
23778
diff
changeset
|
557 head = '' |
29216
ead25aa27a43
py3: convert to next() function
timeless <timeless@mozdev.org>
parents:
29205
diff
changeset
|
558 renamed = os.path.join(head, next(wordgen)) |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
559 replacements[dirpath] = renamed |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
560 return renamed |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
43004
diff
changeset
|
561 |
22708
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
562 result = [] |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
563 for dirpath, count in dirs.iteritems(): |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
564 result.append([rename(dirpath.lstrip(os.sep)), count]) |
4c66e70c3488
contrib/synthrepo: generate initial repo contents using directory shape model
Mike Edgar <adgar@google.com>
parents:
22473
diff
changeset
|
565 return result |