Mercurial > hg
annotate hgext/censor.py @ 39772:ae531f5e583c
testing: add interface unit tests for file storage
Our strategy for supporting alternate storage backends is to define
interfaces for everything then "code to the interface."
We already have interfaces for various primitives, including file
and manifest storage.
What we don't have is generic unit tests for those interfaces. Up
to this point we've been relying on high-level integration tests
(mainly in the form of existing .t tests) to test alternate storage
backends. And my experience with developing the "simple store" test
extension is that such testing is very tedious: it takes several
minutes to run all tests and when you find a failure, it is often
non-trivial to debug.
This commit starts to change that.
This commit introduces the mercurial.testing.storage module. It
contains testing code for storage. Currently, it defines some
unittest.TestCase classes for testing the file storage interfaces.
It also defines some factory functions that allow a caller to easily
spawn a custom TestCase "bound" to a specific file storage backend
implementation.
A new .py test has been added. It simply defines a callable to produce
filelog and transaction instances on demand and then "registers" the
various test classes so the filelog class can be tested with the
storage interface unit tests.
As part of writing the tests, I identified a couple of apparent
bugs in revlog.py and filelog.py! These are tracked with inline
TODO comments.
Writing the tests makes it more obvious where the storage interface
is lacking. For example, we raise either IndexError or
error.LookupError for missing revisions depending on whether we
use an integer revision or a node. Also, we raise error.RevlogError
in various places when we should be raising a storage-agnostic
error type.
The storage interfaces are currently far from perfect and there is much
work to be done to improve them. But at least with this commit we
finally have the start of unit tests that can be used to "qualify"
the behavior of a storage backend. And when implementing and debugging
new storage backends, we now have an obvious place to define new
tests and have obvious places to insert breakpoints to facilitate
debugging. This should be invaluable when implementing new storage
backends.
I added the mercurial.testing package because these interface
conformance tests are generic and need to be usable by all storage
backends. Having the code live in tests/ would make it difficult for
storage backends implemented in extensions to test their interface
conformance. First, it would require obtaining a copy of Mercurial's
storage test code in order to test. Second, it would make testing
against multiple Mercurial versions difficult, as you would need to
import N copies of the storage testing code in order to achieve test
coverage. By making the test code part of the Mercurial distribution
itself, extensions can `import mercurial.testing.*` to access and run
the test code. The test will run against whatever Mercurial version
is active.
FWIW I've always wanted to move parts of run-tests.py into the
mercurial.* package to make the testing story simpler (e.g. imagine an
`hg debugruntests` command that could invoke the test harness). While I
have no plans to do that in the near future, establishing the
mercurial.testing package does provide a natural home for that code
should someone do this in the future.
Differential Revision: https://phab.mercurial-scm.org/D4650
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Tue, 18 Sep 2018 16:52:11 -0700 |
parents | 8bfbb25859f1 |
children | a6b3c4c1019f |
rev | line source |
---|---|
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
1 # Copyright (C) 2015 - Mike Edgar <adgar@google.com> |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
2 # |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
3 # This extension enables removal of file content at a given revision, |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
4 # rewriting the data/metadata of successive revisions to preserve revision log |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
5 # integrity. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
6 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
7 """erase file content at a given revision |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
8 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
9 The censor command instructs Mercurial to erase all content of a file at a given |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
10 revision *without updating the changeset hash.* This allows existing history to |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
11 remain valid while preventing future clones/pulls from receiving the erased |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
12 data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
13 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
14 Typical uses for censor are due to security or legal requirements, including:: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
15 |
26781
1aee2ab0f902
spelling: trivial spell checking
Mads Kiilerich <madski@unity3d.com>
parents:
26587
diff
changeset
|
16 * Passwords, private keys, cryptographic material |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
17 * Licensed data/code/libraries for which the license has expired |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
18 * Personally Identifiable Information or other private data |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
19 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
20 Censored nodes can interrupt mercurial's typical operation whenever the excised |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
21 data needs to be materialized. Some commands, like ``hg cat``/``hg revert``, |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
22 simply fail when asked to produce censored data. Others, like ``hg verify`` and |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
23 ``hg update``, must be capable of tolerating censored data to continue to |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
24 function in a meaningful way. Such commands only tolerate censored file |
24890
cba84b06b702
censor: fix incorrect configuration name for ignoring error at censored file
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24880
diff
changeset
|
25 revisions if they are allowed by the "censor.policy=ignore" config option. |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
26 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
27 |
28092
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
28 from __future__ import absolute_import |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
29 |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
30 from mercurial.i18n import _ |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
31 from mercurial.node import short |
28092
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
32 |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
33 from mercurial import ( |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
34 error, |
38783
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38441
diff
changeset
|
35 pycompat, |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
32315
diff
changeset
|
36 registrar, |
28092
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
37 revlog, |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
38 scmutil, |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
39 util, |
5166b7a84b72
censor: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents:
27290
diff
changeset
|
40 ) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
41 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
42 cmdtable = {} |
32337
46ba2cdda476
registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents:
32315
diff
changeset
|
43 command = registrar.command(cmdtable) |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
28092
diff
changeset
|
44 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
25186
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
45 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
46 # be specifying the version(s) of Mercurial they are tested with, or |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
47 # leave the attribute unspecified. |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
28092
diff
changeset
|
48 testedwith = 'ships-with-hg-core' |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
49 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
50 @command('censor', |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
51 [('r', 'rev', '', _('censor file from specified revision'), _('REV')), |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
52 ('t', 'tombstone', '', _('replacement tombstone data'), _('TEXT'))], |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
53 _('-r REV [-t TEXT] [FILE]')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
54 def censor(ui, repo, path, rev='', tombstone='', **opts): |
38441
e219e355e088
censor: use context manager for lock management
Matt Harbison <matt_harbison@yahoo.com>
parents:
37442
diff
changeset
|
55 with repo.wlock(), repo.lock(): |
27290
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
56 return _docensor(ui, repo, path, rev, tombstone, **opts) |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
57 |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
58 def _docensor(ui, repo, path, rev='', tombstone='', **opts): |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
59 if not path: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
60 raise error.Abort(_('must specify file path to censor')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
61 if not rev: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
62 raise error.Abort(_('must specify revision to censor')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
63 |
25806
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
64 wctx = repo[None] |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
65 |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
66 m = scmutil.match(wctx, (path,)) |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
67 if m.anypats() or len(m.files()) != 1: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
68 raise error.Abort(_('can only specify an explicit filename')) |
25806
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
69 path = m.files()[0] |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
70 flog = repo.file(path) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
71 if not len(flog): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
72 raise error.Abort(_('cannot censor file with no history')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
73 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
74 rev = scmutil.revsingle(repo, rev, rev).rev() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
75 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
76 ctx = repo[rev] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
77 except KeyError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
78 raise error.Abort(_('invalid revision identifier %s') % rev) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
79 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
80 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
81 fctx = ctx.filectx(path) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
82 except error.LookupError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
83 raise error.Abort(_('file does not exist at revision %s') % rev) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
84 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
85 fnode = fctx.filenode() |
39615
a658f97c1ce4
censor: use a reasonable amount of memory
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
38783
diff
changeset
|
86 heads = [] |
a658f97c1ce4
censor: use a reasonable amount of memory
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
38783
diff
changeset
|
87 for headnode in repo.heads(): |
39661
8bfbb25859f1
censor: rename loop variable to silence pyflakes warning
Yuya Nishihara <yuya@tcha.org>
parents:
39615
diff
changeset
|
88 hc = repo[headnode] |
8bfbb25859f1
censor: rename loop variable to silence pyflakes warning
Yuya Nishihara <yuya@tcha.org>
parents:
39615
diff
changeset
|
89 if path in hc and hc.filenode(path) == fnode: |
8bfbb25859f1
censor: rename loop variable to silence pyflakes warning
Yuya Nishihara <yuya@tcha.org>
parents:
39615
diff
changeset
|
90 heads.append(hc) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
91 if heads: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
92 headlist = ', '.join([short(c.node()) for c in heads]) |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
93 raise error.Abort(_('cannot censor file in heads (%s)') % headlist, |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
94 hint=_('clean/delete and commit first')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
95 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
96 wp = wctx.parents() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
97 if ctx.node() in [p.node() for p in wp]: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
98 raise error.Abort(_('cannot censor working directory'), |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
99 hint=_('clean/delete/update first')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
100 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
101 flogv = flog.version & 0xFFFF |
32315
67026d65a4fc
revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29841
diff
changeset
|
102 if flogv != revlog.REVLOGV1: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
103 raise error.Abort( |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
104 _('censor does not support revlog version %d') % (flogv,)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
105 |
37442
0596d27457c6
revlog: move parsemeta() and packmeta() from filelog (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
32337
diff
changeset
|
106 tombstone = revlog.packmeta({"censored": tombstone}, "") |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
107 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
108 crev = fctx.filerev() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
109 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
110 if len(tombstone) > flog.rawsize(crev): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
111 raise error.Abort(_( |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
112 'censor tombstone must be no longer than censored data')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
113 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
114 # Using two files instead of one makes it easy to rewrite entry-by-entry |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
115 idxread = repo.svfs(flog.indexfile, 'r') |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
116 idxwrite = repo.svfs(flog.indexfile, 'wb', atomictemp=True) |
32315
67026d65a4fc
revlog: rename constants (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29841
diff
changeset
|
117 if flog.version & revlog.FLAG_INLINE_DATA: |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
118 dataread, datawrite = idxread, idxwrite |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
119 else: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
120 dataread = repo.svfs(flog.datafile, 'r') |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
121 datawrite = repo.svfs(flog.datafile, 'wb', atomictemp=True) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
122 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
123 # Copy all revlog data up to the entry to be censored. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
124 rio = revlog.revlogio() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
125 offset = flog.start(crev) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
126 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
127 for chunk in util.filechunkiter(idxread, limit=crev * rio.size): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
128 idxwrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
129 for chunk in util.filechunkiter(dataread, limit=offset): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
130 datawrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
131 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
132 def rewriteindex(r, newoffs, newdata=None): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
133 """Rewrite the index entry with a new data offset and optional new data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
134 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
135 The newdata argument, if given, is a tuple of three positive integers: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
136 (new compressed, new uncompressed, added flag bits). |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
137 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
138 offlags, comp, uncomp, base, link, p1, p2, nodeid = flog.index[r] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
139 flags = revlog.gettype(offlags) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
140 if newdata: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
141 comp, uncomp, nflags = newdata |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
142 flags |= nflags |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
143 offlags = revlog.offset_type(newoffs, flags) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
144 e = (offlags, comp, uncomp, r, link, p1, p2, nodeid) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
145 idxwrite.write(rio.packentry(e, None, flog.version, r)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
146 idxread.seek(rio.size, 1) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
147 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
148 def rewrite(r, offs, data, nflags=revlog.REVIDX_DEFAULT_FLAGS): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
149 """Write the given full text to the filelog with the given data offset. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
150 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
151 Returns: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
152 The integer number of data bytes written, for tracking data offsets. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
153 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
154 flag, compdata = flog.compress(data) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
155 newcomp = len(flag) + len(compdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
156 rewriteindex(r, offs, (newcomp, len(data), nflags)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
157 datawrite.write(flag) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
158 datawrite.write(compdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
159 dataread.seek(flog.length(r), 1) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
160 return newcomp |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
161 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
162 # Rewrite censored revlog entry with (padded) tombstone data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
163 pad = ' ' * (flog.rawsize(crev) - len(tombstone)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
164 offset += rewrite(crev, offset, tombstone + pad, revlog.REVIDX_ISCENSORED) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
165 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
166 # Rewrite all following filelog revisions fixing up offsets and deltas. |
38783
e7aa113b14f7
global: use pycompat.xrange()
Gregory Szorc <gregory.szorc@gmail.com>
parents:
38441
diff
changeset
|
167 for srev in pycompat.xrange(crev + 1, len(flog)): |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
168 if crev in flog.parentrevs(srev): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
169 # Immediate children of censored node must be re-added as fulltext. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
170 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
171 revdata = flog.revision(srev) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25186
diff
changeset
|
172 except error.CensoredNodeError as e: |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
173 revdata = e.tombstone |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
174 dlen = rewrite(srev, offset, revdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
175 else: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
176 # Copy any other revision data verbatim after fixing up the offset. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
177 rewriteindex(srev, offset) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
178 dlen = flog.length(srev) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
179 for chunk in util.filechunkiter(dataread, limit=dlen): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
180 datawrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
181 offset += dlen |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
182 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
183 idxread.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
184 idxwrite.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
185 if dataread is not idxread: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
186 dataread.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
187 datawrite.close() |