Mercurial > hg
annotate hgext/censor.py @ 27290:525d9b3f0a31
censor: make censor acquire locks before processing
Before this patch, "hg censor" executes below:
- without acquisition of wlock, examine whether the working
directory refers the revision of the file to be censored or not
- without acquisition of store lock (slock), replace existing
filelog of file to be censored with censored one,
Replacement consists of steps below, and it is assumed that the
destination filelog at (1) isn't changed before renaming at (3).
1. read existing filelog in
2. write filelog entries (both censored and not) into temporary file
3. rename from temporary file to existing filelog to be censored
It may cause unintentional result, if another command runs parallelly
(see also issue4368).
This patch makes "hg censor" acquire wlock and slock before
processing.
author | FUJIWARA Katsunori <foozy@lares.dti.ne.jp> |
---|---|
date | Wed, 09 Dec 2015 08:28:53 +0900 |
parents | 1aee2ab0f902 |
children | 5166b7a84b72 |
rev | line source |
---|---|
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
1 # Copyright (C) 2015 - Mike Edgar <adgar@google.com> |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
2 # |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
3 # This extension enables removal of file content at a given revision, |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
4 # rewriting the data/metadata of successive revisions to preserve revision log |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
5 # integrity. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
6 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
7 """erase file content at a given revision |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
8 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
9 The censor command instructs Mercurial to erase all content of a file at a given |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
10 revision *without updating the changeset hash.* This allows existing history to |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
11 remain valid while preventing future clones/pulls from receiving the erased |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
12 data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
13 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
14 Typical uses for censor are due to security or legal requirements, including:: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
15 |
26781
1aee2ab0f902
spelling: trivial spell checking
Mads Kiilerich <madski@unity3d.com>
parents:
26587
diff
changeset
|
16 * Passwords, private keys, cryptographic material |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
17 * Licensed data/code/libraries for which the license has expired |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
18 * Personally Identifiable Information or other private data |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
19 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
20 Censored nodes can interrupt mercurial's typical operation whenever the excised |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
21 data needs to be materialized. Some commands, like ``hg cat``/``hg revert``, |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
22 simply fail when asked to produce censored data. Others, like ``hg verify`` and |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
23 ``hg update``, must be capable of tolerating censored data to continue to |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
24 function in a meaningful way. Such commands only tolerate censored file |
24890
cba84b06b702
censor: fix incorrect configuration name for ignoring error at censored file
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
24880
diff
changeset
|
25 revisions if they are allowed by the "censor.policy=ignore" config option. |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
26 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
27 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
28 from mercurial.node import short |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
29 from mercurial import cmdutil, error, filelog, revlog, scmutil, util |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
30 from mercurial.i18n import _ |
27290
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
31 from mercurial import lock as lockmod |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
32 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
33 cmdtable = {} |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
34 command = cmdutil.command(cmdtable) |
25186
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
35 # Note for extension authors: ONLY specify testedwith = 'internal' for |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
36 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
37 # be specifying the version(s) of Mercurial they are tested with, or |
80c5b2666a96
extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents:
24890
diff
changeset
|
38 # leave the attribute unspecified. |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
39 testedwith = 'internal' |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
40 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
41 @command('censor', |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
42 [('r', 'rev', '', _('censor file from specified revision'), _('REV')), |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
43 ('t', 'tombstone', '', _('replacement tombstone data'), _('TEXT'))], |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
44 _('-r REV [-t TEXT] [FILE]')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
45 def censor(ui, repo, path, rev='', tombstone='', **opts): |
27290
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
46 wlock = lock = None |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
47 try: |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
48 wlock = repo.wlock() |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
49 lock = repo.lock() |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
50 return _docensor(ui, repo, path, rev, tombstone, **opts) |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
51 finally: |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
52 lockmod.release(lock, wlock) |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
53 |
525d9b3f0a31
censor: make censor acquire locks before processing
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
26781
diff
changeset
|
54 def _docensor(ui, repo, path, rev='', tombstone='', **opts): |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
55 if not path: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
56 raise error.Abort(_('must specify file path to censor')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
57 if not rev: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
58 raise error.Abort(_('must specify revision to censor')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
59 |
25806
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
60 wctx = repo[None] |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
61 |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
62 m = scmutil.match(wctx, (path,)) |
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
63 if m.anypats() or len(m.files()) != 1: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
64 raise error.Abort(_('can only specify an explicit filename')) |
25806
5e18f6e39006
censor: make various path forms available like other Mercurial commands
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
25660
diff
changeset
|
65 path = m.files()[0] |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
66 flog = repo.file(path) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
67 if not len(flog): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
68 raise error.Abort(_('cannot censor file with no history')) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
69 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
70 rev = scmutil.revsingle(repo, rev, rev).rev() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
71 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
72 ctx = repo[rev] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
73 except KeyError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
74 raise error.Abort(_('invalid revision identifier %s') % rev) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
75 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
76 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
77 fctx = ctx.filectx(path) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
78 except error.LookupError: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
79 raise error.Abort(_('file does not exist at revision %s') % rev) |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
80 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
81 fnode = fctx.filenode() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
82 headctxs = [repo[c] for c in repo.heads()] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
83 heads = [c for c in headctxs if path in c and c.filenode(path) == fnode] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
84 if heads: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
85 headlist = ', '.join([short(c.node()) for c in heads]) |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
86 raise error.Abort(_('cannot censor file in heads (%s)') % headlist, |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
87 hint=_('clean/delete and commit first')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
88 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
89 wp = wctx.parents() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
90 if ctx.node() in [p.node() for p in wp]: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
91 raise error.Abort(_('cannot censor working directory'), |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
92 hint=_('clean/delete/update first')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
93 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
94 flogv = flog.version & 0xFFFF |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
95 if flogv != revlog.REVLOGNG: |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
96 raise error.Abort( |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
97 _('censor does not support revlog version %d') % (flogv,)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
98 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
99 tombstone = filelog.packmeta({"censored": tombstone}, "") |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
100 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
101 crev = fctx.filerev() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
102 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
103 if len(tombstone) > flog.rawsize(crev): |
26587
56b2bcea2529
error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
25806
diff
changeset
|
104 raise error.Abort(_( |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
105 'censor tombstone must be no longer than censored data')) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
106 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
107 # Using two files instead of one makes it easy to rewrite entry-by-entry |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
108 idxread = repo.svfs(flog.indexfile, 'r') |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
109 idxwrite = repo.svfs(flog.indexfile, 'wb', atomictemp=True) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
110 if flog.version & revlog.REVLOGNGINLINEDATA: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
111 dataread, datawrite = idxread, idxwrite |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
112 else: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
113 dataread = repo.svfs(flog.datafile, 'r') |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
114 datawrite = repo.svfs(flog.datafile, 'wb', atomictemp=True) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
115 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
116 # Copy all revlog data up to the entry to be censored. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
117 rio = revlog.revlogio() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
118 offset = flog.start(crev) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
119 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
120 for chunk in util.filechunkiter(idxread, limit=crev * rio.size): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
121 idxwrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
122 for chunk in util.filechunkiter(dataread, limit=offset): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
123 datawrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
124 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
125 def rewriteindex(r, newoffs, newdata=None): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
126 """Rewrite the index entry with a new data offset and optional new data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
127 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
128 The newdata argument, if given, is a tuple of three positive integers: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
129 (new compressed, new uncompressed, added flag bits). |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
130 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
131 offlags, comp, uncomp, base, link, p1, p2, nodeid = flog.index[r] |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
132 flags = revlog.gettype(offlags) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
133 if newdata: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
134 comp, uncomp, nflags = newdata |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
135 flags |= nflags |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
136 offlags = revlog.offset_type(newoffs, flags) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
137 e = (offlags, comp, uncomp, r, link, p1, p2, nodeid) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
138 idxwrite.write(rio.packentry(e, None, flog.version, r)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
139 idxread.seek(rio.size, 1) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
140 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
141 def rewrite(r, offs, data, nflags=revlog.REVIDX_DEFAULT_FLAGS): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
142 """Write the given full text to the filelog with the given data offset. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
143 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
144 Returns: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
145 The integer number of data bytes written, for tracking data offsets. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
146 """ |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
147 flag, compdata = flog.compress(data) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
148 newcomp = len(flag) + len(compdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
149 rewriteindex(r, offs, (newcomp, len(data), nflags)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
150 datawrite.write(flag) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
151 datawrite.write(compdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
152 dataread.seek(flog.length(r), 1) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
153 return newcomp |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
154 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
155 # Rewrite censored revlog entry with (padded) tombstone data. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
156 pad = ' ' * (flog.rawsize(crev) - len(tombstone)) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
157 offset += rewrite(crev, offset, tombstone + pad, revlog.REVIDX_ISCENSORED) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
158 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
159 # Rewrite all following filelog revisions fixing up offsets and deltas. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
160 for srev in xrange(crev + 1, len(flog)): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
161 if crev in flog.parentrevs(srev): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
162 # Immediate children of censored node must be re-added as fulltext. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
163 try: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
164 revdata = flog.revision(srev) |
25660
328739ea70c3
global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents:
25186
diff
changeset
|
165 except error.CensoredNodeError as e: |
24347
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
166 revdata = e.tombstone |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
167 dlen = rewrite(srev, offset, revdata) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
168 else: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
169 # Copy any other revision data verbatim after fixing up the offset. |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
170 rewriteindex(srev, offset) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
171 dlen = flog.length(srev) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
172 for chunk in util.filechunkiter(dataread, limit=dlen): |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
173 datawrite.write(chunk) |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
174 offset += dlen |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
175 |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
176 idxread.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
177 idxwrite.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
178 if dataread is not idxread: |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
179 dataread.close() |
1bcfecbbf569
censor: add censor command to hgext with basic client-side tests
Mike Edgar <adgar@google.com>
parents:
diff
changeset
|
180 datawrite.close() |