annotate tests/test-storage.py @ 40417:49c7b701fdc2 stable

phase: add an archived phase This phase allows for hidden changesets in the "user space". It differs from the "internal" phase which is intended for internal by-product only. There have been discussions at the 4.8 sprint to use such phase to speedup cleanup after history rewriting operation. Shipping it in the same release as the 'internal-phase' groups the associated `requires` entry. The important bit is to have support for this phase in the earliest version of mercurial possible. Adding the UI to manipulate this new phase later seems fine. The current plan for archived usage and user interface are as follow. On a repository with internal-phase on and evolution off: * history rewriting command set rewritten changeset in the archived phase. (This mean updating the cleanupnodes method). * keep `hg unbundle .hg/strip-backup/X.hg` as a way to restore changeset for now (backup bundle need to contains phase data) * [maybe] add a `hg strip --soft` advance flag (a light way to expose the feature without getting in the way of a better UI) Mercurial 4.8 freeze is too close to get the above in by then. We don't introduce a new repository `requirement` as we reuse the one introduced with the 'archived' phase during the 4.8 cycle.
author Boris Feld <boris.feld@octobus.net>
date Wed, 17 Oct 2018 14:47:01 +0200
parents c3ad9ef0876c
children 1bf3e6041e2c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 # This test verifies the conformance of various classes to various
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 # storage interfaces.
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 from __future__ import absolute_import
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 import silenttestrunner
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7 from mercurial import (
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
8 error,
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 filelog,
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
10 revlog,
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 transaction,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 ui as uimod,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 vfs as vfsmod,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 )
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 from mercurial.testing import (
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 storage as storagetesting,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 )
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19
40363
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
20 try:
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
21 from hgext import (
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
22 sqlitestore,
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
23 )
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
24 except ImportError:
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
25 sqlitestore = None
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
26
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
27 try:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
28 from mercurial import zstd
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
29 zstd.__version__
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
30 except ImportError:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
31 zstd = None
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
32
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 STATE = {
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34 'lastindex': 0,
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 'ui': uimod.ui(),
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 'vfs': vfsmod.vfs(b'.', realpath=True),
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37 }
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 def makefilefn(self):
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40 """Factory for filelog instances."""
39953
a3a9b93bff80 py3: byteify test-storage.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39772
diff changeset
41 fl = filelog.filelog(STATE['vfs'], b'filelog-%d' % STATE['lastindex'])
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 STATE['lastindex'] += 1
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 return fl
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 def maketransaction(self):
40320
9b2e1b00ee94 tests: use byte literals in test-storage.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40056
diff changeset
46 vfsmap = {b'plain': STATE['vfs'], b'store': STATE['vfs']}
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 return transaction.transaction(STATE['ui'].warn, STATE['vfs'], vfsmap,
39953
a3a9b93bff80 py3: byteify test-storage.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39772
diff changeset
49 b'journal', b'undo')
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
51 def addrawrevision(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
52 delta=None, censored=False, ellipsis=False, extstored=False):
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
53 flags = 0
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
54
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
55 if censored:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
56 flags |= revlog.REVIDX_ISCENSORED
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
57 if ellipsis:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
58 flags |= revlog.REVIDX_ELLIPSIS
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
59 if extstored:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
60 flags |= revlog.REVIDX_EXTSTORED
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
61
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
62 if rawtext is not None:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
63 fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags)
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
64 elif delta is not None:
40323
2c0aa02ecd5a testing: switch to inserting deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40320
diff changeset
65 fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags,
2c0aa02ecd5a testing: switch to inserting deltas
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40320
diff changeset
66 cachedelta=delta)
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
67 else:
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
68 raise error.Abort('must supply rawtext or delta arguments')
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
69
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
70 # We may insert bad data. Clear caches to prevent e.g. cache hits to
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
71 # bypass hash verification.
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
72 fl._revlog.clearcaches()
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
73
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74 # Assigning module-level attributes that inherit from unittest.TestCase
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 # is all that is needed to register tests.
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 filelogindextests = storagetesting.makeifileindextests(makefilefn,
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
77 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
78 addrawrevision)
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79 filelogdatatests = storagetesting.makeifiledatatests(makefilefn,
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
80 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
81 addrawrevision)
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82 filelogmutationtests = storagetesting.makeifilemutationtests(makefilefn,
40051
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
83 maketransaction,
cdf61ab1f54c testing: add file storage integration for bad hashes and censoring
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39953
diff changeset
84 addrawrevision)
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
86 def makesqlitefile(self):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
87 path = STATE['vfs'].join(b'db-%d.db' % STATE['lastindex'])
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
88 STATE['lastindex'] += 1
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
89
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
90 db = sqlitestore.makedb(path)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
91
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
92 compression = b'zstd' if zstd else b'zlib'
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
93
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
94 return sqlitestore.sqlitefilestore(db, b'dummy-path', compression)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
95
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
96 def addrawrevisionsqlite(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
97 delta=None, censored=False, ellipsis=False,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
98 extstored=False):
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
99 flags = 0
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
100
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
101 if censored:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
102 flags |= sqlitestore.FLAG_CENSORED
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
103
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
104 if ellipsis | extstored:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
105 raise error.Abort(b'support for ellipsis and extstored flags not '
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
106 b'supported')
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
107
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
108 if rawtext is not None:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
109 fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2, flags=flags)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
110 elif delta is not None:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
111 fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2,
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
112 storedelta=delta, flags=flags)
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
113 else:
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
114 raise error.Abort(b'must supply rawtext or delta arguments')
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
115
40363
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
116 if sqlitestore is not None:
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
117 sqlitefileindextests = storagetesting.makeifileindextests(
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
118 makesqlitefile, maketransaction, addrawrevisionsqlite)
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
119 sqlitefiledatatests = storagetesting.makeifiledatatests(
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
120 makesqlitefile, maketransaction, addrawrevisionsqlite)
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
121 sqlitefilemutationtests = storagetesting.makeifilemutationtests(
c3ad9ef0876c tests: don't emit false failures when sqlite3 is missing
Augie Fackler <augie@google.com>
parents: 40326
diff changeset
122 makesqlitefile, maketransaction, addrawrevisionsqlite)
40326
fed697fa1734 sqlitestore: file storage backend using SQLite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40323
diff changeset
123
39772
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
124 if __name__ == '__main__':
ae531f5e583c testing: add interface unit tests for file storage
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
125 silenttestrunner.main(__name__)