Mercurial > hg
view tests/test-censor.t @ 51448:89cd0dd41e4d stable
crecord: drop calls to `curses.endwin()`
We got a bug report where `curses.endwin()` failed with `_curses.error: endwin()
returned ERR`. Looking at e306d552dfb12, it seems like we should be able to just
remove these calls.
author | Martin von Zweigbergk <martinvonz@google.com> |
---|---|
date | Thu, 29 Feb 2024 14:13:21 -0800 |
parents | 508fd40dc86a |
children |
line wrap: on
line source
#require no-reposimplestore #testcases revlogv1 revlogv2 #if revlogv2 $ cat >> $HGRCPATH <<EOF > [experimental] > revlogv2=enable-unstable-format-and-corrupt-my-data > EOF #endif $ cp $HGRCPATH $HGRCPATH.orig Create repo with unimpeachable content $ hg init r $ cd r $ echo 'Initially untainted file' > target $ echo 'Normal file here' > bystander $ hg add target bystander $ hg ci -m init Clone repo so we can test pull later $ cd .. $ hg clone r rpull updating to branch default 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd r Introduce content which will ultimately require censorship. Name the first censored node C1, second C2, and so on $ echo 'Tainted file' > target $ echo 'Passwords: hunter2' >> target $ hg ci -m taint target $ C1=`hg id --debug -i` $ echo 'hunter3' >> target $ echo 'Normal file v2' > bystander $ hg ci -m moretaint target bystander $ C2=`hg id --debug -i` Add a new sanitized versions to correct our mistake. Name the first head H1, the second head H2, and so on $ echo 'Tainted file is now sanitized' > target $ hg ci -m sanitized target $ H1=`hg id --debug -i` $ hg update -r $C2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ echo 'Tainted file now super sanitized' > target $ hg ci -m 'super sanitized' target created new head $ H2=`hg id --debug -i` Verify target contents before censorship at each revision $ hg cat -r $H1 target | head -n 10 Tainted file is now sanitized $ hg cat -r $H2 target | head -n 10 Tainted file now super sanitized $ hg cat -r $C2 target | head -n 10 Tainted file Passwords: hunter2 hunter3 $ hg cat -r $C1 target | head -n 10 Tainted file Passwords: hunter2 $ hg cat -r 0 target | head -n 10 Initially untainted file Censor revision with 2 offenses (this also tests file pattern matching: path relative to cwd case) $ mkdir -p foo/bar/baz $ hg --config extensions.censor= --cwd foo/bar/baz censor -r $C2 -t "remove password" ../../../target checking for the censored content in 2 heads checking for the censored content in the working directory censoring 1 file revisions $ hg cat -r $H1 target | head -n 10 Tainted file is now sanitized $ hg cat -r $H2 target | head -n 10 Tainted file now super sanitized $ hg cat -r $C2 target | head -n 10 abort: censored node: 1e0247a9a4b7 (set censor.policy to ignore errors) $ hg cat -r $C1 target | head -n 10 Tainted file Passwords: hunter2 $ hg cat -r 0 target | head -n 10 Initially untainted file Censor revision with 1 offense (this also tests file pattern matching: with 'path:' scheme) $ hg --config extensions.censor= --cwd foo/bar/baz censor -r $C1 path:target checking for the censored content in 2 heads checking for the censored content in the working directory censoring 1 file revisions $ hg cat -r $H1 target | head -n 10 Tainted file is now sanitized $ hg cat -r $H2 target | head -n 10 Tainted file now super sanitized $ hg cat -r $C2 target | head -n 10 abort: censored node: 1e0247a9a4b7 (set censor.policy to ignore errors) $ hg cat -r $C1 target | head -n 10 abort: censored node: 613bc869fceb (set censor.policy to ignore errors) $ hg cat -r 0 target | head -n 10 Initially untainted file Can only checkout target at uncensored revisions, -X is workaround for --all $ hg revert -r $C2 target | head -n 10 abort: censored node: 1e0247a9a4b7 (set censor.policy to ignore errors) $ hg revert -r $C1 target | head -n 10 abort: censored node: 613bc869fceb (set censor.policy to ignore errors) $ hg revert -r $C1 --all reverting bystander reverting target abort: censored node: 613bc869fceb (set censor.policy to ignore errors) [255] $ hg revert -r $C1 --all -X target $ cat target | head -n 10 Tainted file now super sanitized $ hg revert -r 0 --all reverting target $ cat target | head -n 10 Initially untainted file $ hg revert -r $H2 --all reverting bystander reverting target $ cat target | head -n 10 Tainted file now super sanitized Uncensored file can be viewed at any revision $ hg cat -r $H1 bystander | head -n 10 Normal file v2 $ hg cat -r $C2 bystander | head -n 10 Normal file v2 $ hg cat -r $C1 bystander | head -n 10 Normal file here $ hg cat -r 0 bystander | head -n 10 Normal file here Can update to children of censored revision $ hg update -r $H1 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Tainted file is now sanitized $ hg update -r $H2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Tainted file now super sanitized Set censor policy to abort in trusted $HGRC so hg verify fails $ cp $HGRCPATH.orig $HGRCPATH $ cat >> $HGRCPATH <<EOF > [censor] > policy = abort > EOF Repo fails verification due to censorship $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files target@1: censored file data target@2: censored file data not checking dirstate because of previous errors checked 5 changesets with 7 changes to 2 files 2 integrity errors encountered! (first damaged changeset appears to be 1) [1] Cannot update to revision with censored data $ hg update -r $C2 abort: censored node: 1e0247a9a4b7 (set censor.policy to ignore errors) [255] $ hg update -r $C1 abort: censored node: 613bc869fceb (set censor.policy to ignore errors) [255] $ hg update -r 0 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg update -r $H2 2 files updated, 0 files merged, 0 files removed, 0 files unresolved Set censor policy to ignore in trusted $HGRC so hg verify passes $ cp $HGRCPATH.orig $HGRCPATH $ cat >> $HGRCPATH <<EOF > [censor] > policy = ignore > EOF Repo passes verification with warnings with explicit config $ hg verify -q May update to revision with censored data with explicit config $ hg update -r $C2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 $ hg update -r $C1 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 $ hg update -r 0 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Initially untainted file $ hg update -r $H2 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Tainted file now super sanitized Can merge in revision with censored data. Test requires one branch of history with the file censored, but we can't censor at a head, so advance H1. $ hg update -r $H1 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ C3=$H1 $ echo 'advanced head H1' > target $ hg ci -m 'advance head H1' target $ H1=`hg id --debug -i` $ hg --config extensions.censor= censor -r $C3 target checking for the censored content in 2 heads checking for the censored content in the working directory censoring 1 file revisions $ hg update -r $H2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg merge -r $C3 merging target 0 files updated, 1 files merged, 0 files removed, 0 files unresolved (branch merge, don't forget to commit) Revisions present in repository heads may not be censored $ hg update -C -r $H2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg --config extensions.censor= censor -r $H2 target checking for the censored content in 2 heads abort: cannot censor file in heads (78a8fc215e79) (clean/delete and commit first) [255] $ echo 'twiddling thumbs' > bystander $ hg ci -m 'bystander commit' $ H2=`hg id --debug -i` $ hg --config extensions.censor= censor -r "$H2^" target checking for the censored content in 2 heads abort: cannot censor file in heads (efbe78065929) (clean/delete and commit first) [255] Cannot censor working directory $ echo 'seriously no passwords' > target $ hg ci -m 'extend second head arbitrarily' target $ H2=`hg id --debug -i` $ hg update -r "$H2^" 1 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg --config extensions.censor= censor -r . target checking for the censored content in 2 heads checking for the censored content in the working directory abort: cannot censor working directory (clean/delete/update first) [255] $ hg update -r $H2 1 files updated, 0 files merged, 0 files removed, 0 files unresolved Can re-add file after being deleted + censored $ C4=$H2 $ hg rm target $ hg ci -m 'delete target so it may be censored' $ H2=`hg id --debug -i` $ hg --config extensions.censor= censor -r $C4 target checking for the censored content in 2 heads checking for the censored content in the working directory censoring 1 file revisions $ hg cat -r $C4 target | head -n 10 $ hg cat -r "$H2^^" target | head -n 10 Tainted file now super sanitized $ echo 'fresh start' > target $ hg add target $ hg ci -m reincarnated target $ H2=`hg id --debug -i` $ hg cat -r $H2 target | head -n 10 fresh start $ hg cat -r "$H2^" target | head -n 10 target: no such file in rev 452ec1762369 $ hg cat -r $C4 target | head -n 10 $ hg cat -r "$H2^^^" target | head -n 10 Tainted file now super sanitized Can censor enough revision to move back to inline storage $ hg debugrevlogstats | grep target rev-count data-size inl type target 8 ??? no file target (glob) (revlogv2 !) 8 ??? yes file target (glob) (revlogv1 !) $ $TESTDIR/seq.py 4000 | $TESTDIR/sha256line.py > target $ hg ci -m 'add 100k passwords' $ H2=`hg id --debug -i` $ C5=$H2 $ hg revert -r "$H2^" target $ hg ci -m 'cleaned 100k passwords' $ H2=`hg id --debug -i` $ hg debugrevlogstats | grep target rev-count data-size inl type target 10 ?????? no file target (glob) $ hg --config extensions.censor= censor -r $C5 target checking for the censored content in 2 heads checking for the censored content in the working directory censoring 1 file revisions The important part is for the censor operation to not crash and the repository to not be corrupted. Right now this involve keeping the revlog split. $ hg debugrevlogstats | grep target rev-count data-size inl type target 10 ??? no file target (glob) $ hg cat -r $C5 target | head -n 10 $ hg cat -r $H2 target | head -n 10 fresh start $ hg verify checking changesets checking manifests crosschecking files in changesets and manifests checking files checking dirstate checked 12 changesets with 13 changes to 2 files Repo with censored nodes can be cloned and cloned nodes are censored $ cd .. $ hg clone r rclone updating to branch default 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd rclone $ hg cat -r $H1 target | head -n 10 advanced head H1 $ hg cat -r $H2~5 target | head -n 10 Tainted file now super sanitized $ hg cat -r $C2 target | head -n 10 $ hg cat -r $C1 target | head -n 10 $ hg cat -r 0 target | head -n 10 Initially untainted file $ hg verify -q Repo cloned before tainted content introduced can pull censored nodes $ cd ../rpull $ hg cat -r tip target | head -n 10 Initially untainted file $ hg verify -q $ hg pull -r $H1 -r $H2 pulling from $TESTTMP/r searching for changes adding changesets adding manifests adding file changes added 11 changesets with 11 changes to 2 files (+1 heads) new changesets * (glob) (run 'hg heads' to see heads, 'hg merge' to merge) $ hg update 4 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Tainted file now super sanitized $ hg cat -r $H1 target | head -n 10 advanced head H1 $ hg cat -r $H2~5 target | head -n 10 Tainted file now super sanitized $ hg cat -r $C2 target | head -n 10 $ hg cat -r $C1 target | head -n 10 $ hg cat -r 0 target | head -n 10 Initially untainted file $ hg verify -q Censored nodes can be pushed if they censor previously unexchanged nodes $ echo 'Passwords: hunter2hunter2' > target $ hg ci -m 're-add password from clone' target created new head $ H3=`hg id --debug -i` $ REV=$H3 $ echo 'Re-sanitized; nothing to see here' > target $ hg ci -m 're-sanitized' target $ H2=`hg id --debug -i` $ CLEANREV=$H2 $ hg cat -r $REV target | head -n 10 Passwords: hunter2hunter2 $ hg --config extensions.censor= censor -r $REV target checking for the censored content in 3 heads checking for the censored content in the working directory censoring 1 file revisions $ hg cat -r $REV target | head -n 10 $ hg cat -r $CLEANREV target | head -n 10 Re-sanitized; nothing to see here $ hg push -f -r $H2 pushing to $TESTTMP/r searching for changes adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 1 files (+1 heads) $ cd ../r $ hg cat -r $REV target | head -n 10 $ hg cat -r $CLEANREV target | head -n 10 Re-sanitized; nothing to see here $ hg update $CLEANREV 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Re-sanitized; nothing to see here Censored nodes can be bundled up and unbundled in another repo $ hg bundle --base 0 ../pwbundle 13 changesets found $ cd ../rclone $ hg unbundle ../pwbundle adding changesets adding manifests adding file changes added 2 changesets with 2 changes to 2 files (+1 heads) new changesets * (glob) (run 'hg heads .' to see heads, 'hg merge' to merge) $ hg cat -r $REV target | head -n 10 $ hg cat -r $CLEANREV target | head -n 10 Re-sanitized; nothing to see here $ hg update $CLEANREV 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Re-sanitized; nothing to see here $ hg verify -q Grepping only warns, doesn't error out $ cd ../rpull $ hg grep 'Normal file' bystander:Normal file v2 $ hg grep nothing target:Re-sanitized; nothing to see here $ hg grep --diff 'Normal file' cannot search in censored file: target:7 cannot search in censored file: target:10 cannot search in censored file: target:12 bystander:6:-:Normal file v2 cannot search in censored file: target:1 cannot search in censored file: target:2 cannot search in censored file: target:3 bystander:2:-:Normal file here bystander:2:+:Normal file v2 bystander:0:+:Normal file here $ hg grep --diff nothing cannot search in censored file: target:7 cannot search in censored file: target:10 cannot search in censored file: target:12 target:13:+:Re-sanitized; nothing to see here cannot search in censored file: target:1 cannot search in censored file: target:2 cannot search in censored file: target:3 Censored nodes can be imported on top of censored nodes, consecutively $ hg init ../rimport $ hg bundle --base 1 ../rimport/splitbundle 12 changesets found $ cd ../rimport $ hg pull -r $H1 -r $H2 ../r pulling from ../r adding changesets adding manifests adding file changes added 8 changesets with 10 changes to 2 files (+1 heads) new changesets e97f55b2665a:dcbaf17bf3a1 (run 'hg heads' to see heads, 'hg merge' to merge) $ hg unbundle splitbundle adding changesets adding manifests adding file changes added 6 changesets with 5 changes to 2 files (+1 heads) new changesets * (glob) (run 'hg heads .' to see heads, 'hg merge' to merge) $ hg update $H2 2 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cat target | head -n 10 Re-sanitized; nothing to see here $ hg verify -q $ cd ../r Can import bundle where first revision of a file is censored $ hg init ../rinit $ hg --config extensions.censor= censor -r 0 target checking for the censored content in 3 heads checking for the censored content in the working directory censoring 1 file revisions $ hg bundle -r 0 --base null ../rinit/initbundle 1 changesets found $ cd ../rinit $ hg unbundle initbundle adding changesets adding manifests adding file changes added 1 changesets with 2 changes to 2 files new changesets e97f55b2665a (1 drafts) (run 'hg update' to get a working copy) $ hg cat -r 0 target | head -n 10 Can skip the head checking steps $ hg --config extensions.censor= censor -r 0 --no-check-heads target checking for the censored content in the working directory censoring 1 file revisions Can censor multiple revision in one go. $ cd ../r $ hg --config extensions.censor= censor -r 0+1 target checking for the censored content in 3 heads checking for the censored content in the working directory censoring 2 file revisions #if revlogv2 Testing feature that does not work in revlog v1 =============================================== Censoring a revision that is used as delta base ----------------------------------------------- $ cd .. $ hg init censor-with-delta $ cd censor-with-delta $ echo root > target $ hg add target $ hg commit -m root $ B0=`hg id --debug -i` $ for x in `"$PYTHON" $TESTDIR/seq.py 0 50000` > do > echo "Password: hunter$x" >> target > done $ hg ci -m 'write a long file' $ B1=`hg id --debug -i` $ echo 'small change (should create a delta)' >> target $ hg ci -m 'create a delta over the password' (should show that the last revision is a delta, not a snapshot) $ B2=`hg id --debug -i` Make sure the last revision is a delta against the revision we will censor $ hg debugdeltachain target -T '{rev} {chainid} {chainlen} {prevrev}\n' 0 1 1 -1 1 2 1 -1 2 2 2 1 Censor the file $ hg cat -r $B1 target | wc -l *50002 (re) $ hg --config extensions.censor= censor -r $B1 target checking for the censored content in 1 heads checking for the censored content in the working directory censoring 1 file revisions $ hg cat -r $B1 target | wc -l *0 (re) Check the children is fine $ hg cat -r $B2 target | wc -l *50003 (re) #endif Testing repository upgrade with censors revision ================================================ $ cd ../rclone With the "abort" policy ======================= $ hg verify --config censor.policy=ignore checking changesets checking manifests crosschecking files in changesets and manifests checking files checking dirstate checked 14 changesets with 15 changes to 2 files $ hg debugupgraderepo --run --quiet \ > --optimize re-delta-parent \ > --config censor.policy=abort upgrade will perform the following actions: requirements preserved: * (glob) optimisations: re-delta-parent processed revlogs: - all-filelogs - changelog - manifest $ hg verify --config censor.policy=ignore checking changesets checking manifests crosschecking files in changesets and manifests checking files checking dirstate checked 14 changesets with 15 changes to 2 files With the "ignore" policy ======================== $ hg verify --config censor.policy=ignore checking changesets checking manifests crosschecking files in changesets and manifests checking files checking dirstate checked 14 changesets with 15 changes to 2 files $ hg debugupgraderepo --run --quiet \ > --optimize re-delta-parent \ > --config censor.policy=ignore upgrade will perform the following actions: requirements preserved: * (glob) optimisations: re-delta-parent processed revlogs: - all-filelogs - changelog - manifest $ hg verify --config censor.policy=ignore checking changesets checking manifests crosschecking files in changesets and manifests checking files checking dirstate checked 14 changesets with 15 changes to 2 files