Mercurial > hg
view tests/test-cbor.py @ 46582:b0a3ca02d17a
copies-rust: implement PartialEqual manually
Now that we know that each (dest, rev) pair has at most a unique CopySource, we
can simplify comparison a lot.
This "simple" step buy a good share of the previous slowdown back in some case:
Repo Case Source-Rev Dest-Rev # of revisions old time new time Difference Factor time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mozilla-try x00000_revs_x00000_added_x000_copies 9b2a99adc05e 8e29777b48e6 : 382065 revs, 43.304637 s, 34.443661 s, -8.860976 s, × 0.7954, 90 µs/rev
Full benchmark:
Repo Case Source-Rev Dest-Rev # of revisions old time new time Difference Factor time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mercurial x_revs_x_added_0_copies ad6b123de1c7 39cfcef4f463 : 1 revs, 0.000043 s, 0.000043 s, +0.000000 s, × 1.0000, 43 µs/rev
mercurial x_revs_x_added_x_copies 2b1c78674230 0c1d10351869 : 6 revs, 0.000114 s, 0.000117 s, +0.000003 s, × 1.0263, 19 µs/rev
mercurial x000_revs_x000_added_x_copies 81f8ff2a9bf2 dd3267698d84 : 1032 revs, 0.004937 s, 0.004892 s, -0.000045 s, × 0.9909, 4 µs/rev
pypy x_revs_x_added_0_copies aed021ee8ae8 099ed31b181b : 9 revs, 0.000339 s, 0.000196 s, -0.000143 s, × 0.5782, 21 µs/rev
pypy x_revs_x000_added_0_copies 4aa4e1f8e19a 359343b9ac0e : 1 revs, 0.000049 s, 0.000050 s, +0.000001 s, × 1.0204, 50 µs/rev
pypy x_revs_x_added_x_copies ac52eb7bbbb0 72e022663155 : 7 revs, 0.000202 s, 0.000117 s, -0.000085 s, × 0.5792, 16 µs/rev
pypy x_revs_x00_added_x_copies c3b14617fbd7 ace7255d9a26 : 1 revs, 0.000409 s, 0.6f1f4a s, -0.000087 s, × 0.7873, 322 µs/rev
pypy x_revs_x000_added_x000_copies df6f7a526b60 a83dc6a2d56f : 6 revs, 0.011984 s, 0.011949 s, -0.000035 s, × 0.9971, 1991 µs/rev
pypy x000_revs_xx00_added_0_copies 89a76aede314 2f22446ff07e : 4785 revs, 0.050820 s, 0.050802 s, -0.000018 s, × 0.9996, 10 µs/rev
pypy x000_revs_x000_added_x_copies 8a3b5bfd266e 2c68e87c3efe : 6780 revs, 0.087953 s, 0.088090 s, +0.000137 s, × 1.0016, 12 µs/rev
pypy x000_revs_x000_added_x000_copies 89a76aede314 7b3dda341c84 : 5441 revs, 0.062902 s, 0.062079 s, -0.000823 s, × 0.9869, 11 µs/rev
pypy x0000_revs_x_added_0_copies d1defd0dc478 c9cb1334cc78 : 43645 revs, 0.679234 s, 0.635337 s, -0.043897 s, × 0.9354, 14 µs/rev
pypy x0000_revs_xx000_added_0_copies bf2c629d0071 4ffed77c095c : 2 revs, 0.013095 s, 0.013262 s, +0.000167 s, × 1.0128, 6631 µs/rev
pypy x0000_revs_xx000_added_x000_copies 08ea3258278e d9fa043f30c0 : 11316 revs, 0.120910 s, 0.120085 s, -0.000825 s, × 0.9932, 10 µs/rev
netbeans x_revs_x_added_0_copies fb0955ffcbcd a01e9239f9e7 : 2 revs, 0.000087 s, 0.000085 s, -0.000002 s, × 0.9770, 42 µs/rev
netbeans x_revs_x000_added_0_copies 6f360122949f 20eb231cc7d0 : 2 revs, 0.000107 s, 0.000110 s, +0.000003 s, × 1.0280, 55 µs/rev
netbeans x_revs_x_added_x_copies 1ada3faf6fb6 5a39d12eecf4 : 3 revs, 0.000186 s, 0.000177 s, -0.000009 s, × 0.9516, 59 µs/rev
netbeans x_revs_x00_added_x_copies 35be93ba1e2c 9eec5e90c05f : 9 revs, 0.000754 s, 0.000743 s, -0.000011 s, × 0.9854, 82 µs/rev
netbeans x000_revs_xx00_added_0_copies eac3045b4fdd 51d4ae7f1290 : 1421 revs, 0.010443 s, 0.010168 s, -0.000275 s, × 0.9737, 7 µs/rev
netbeans x000_revs_x000_added_x_copies e2063d266acd 6081d72689dc : 1533 revs, 0.015697 s, 0.015946 s, +0.000249 s, × 1.0159, 10 µs/rev
netbeans x000_revs_x000_added_x000_copies ff453e9fee32 411350406ec2 : 5750 revs, 0.063528 s, 0.062712 s, -0.000816 s, × 0.9872, 10 µs/rev
netbeans x0000_revs_xx000_added_x000_copies 588c2d1ced70 1aad62e59ddd : 66949 revs, 0.545515 s, 0.523832 s, -0.021683 s, × 0.9603, 7 µs/rev
mozilla-central x_revs_x_added_0_copies 3697f962bb7b 7015fcdd43a2 : 2 revs, 0.000089 s, 0.000090 s, +0.000001 s, × 1.0112, 45 µs/rev
mozilla-central x_revs_x000_added_0_copies dd390860c6c9 40d0c5bed75d : 8 revs, 0.000265 s, 0.000264 s, -0.000001 s, × 0.9962, 33 µs/rev
mozilla-central x_revs_x_added_x_copies 8d198483ae3b 14207ffc2b2f : 9 revs, 0.000381 s, 0.000187 s, -0.000194 s, × 0.4908, 20 µs/rev
mozilla-central x_revs_x00_added_x_copies 98cbc58cc6bc 446a150332c3 : 7 revs, 0.000672 s, 0.000665 s, -0.000007 s, × 0.9896, 95 µs/rev
mozilla-central x_revs_x000_added_x000_copies 3c684b4b8f68 0a5e72d1b479 : 3 revs, 0.003497 s, 0.003556 s, +0.000059 s, × 1.0169, 1185 µs/rev
mozilla-central x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 6 revs, 0.073204 s, 0.071345 s, -0.001859 s, × 0.9746, 11890 µs/rev
mozilla-central x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 1593 revs, 0.006482 s, 0.006551 s, +0.000069 s, × 1.0106, 4 µs/rev
mozilla-central x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 41 revs, 0.005066 s, 0.005078 s, +0.000012 s, × 1.0024, 123 µs/rev
mozilla-central x000_revs_x000_added_x000_copies 7c97034feb78 4407bd0c6330 : 7839 revs, 0.065707 s, 0.065823 s, +0.000116 s, × 1.0018, 8 µs/rev
mozilla-central x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 615 revs, 0.026800 s, 0.027050 s, +0.000250 s, × 1.0093, 43 µs/rev
mozilla-central x0000_revs_xx000_added_x000_copies f78c615a656c 96a38b690156 : 30263 revs, 0.203856 s, 0.202443 s, -0.001413 s, × 0.9931, 6 µs/rev
mozilla-central x00000_revs_x0000_added_x0000_copies 6832ae71433c 4c222a1d9a00 : 153721 revs, 1.293394 s, 1.261583 s, -0.031811 s, × 0.9754, 8 µs/rev
mozilla-central x00000_revs_x00000_added_x000_copies 76caed42cf7c 1daa622bbe42 : 204976 revs, 1.698239 s, 1.643869 s, -0.054370 s, × 0.9680, 8 µs/rev
mozilla-try x_revs_x_added_0_copies aaf6dde0deb8 9790f499805a : 2 revs, 0.000875 s, 0.000868 s, -0.000007 s, × 0.9920, 434 µs/rev
mozilla-try x_revs_x000_added_0_copies d8d0222927b4 5bb8ce8c7450 : 2 revs, 0.000891 s, 0.000887 s, -0.000004 s, × 0.9955, 443 µs/rev
mozilla-try x_revs_x_added_x_copies 092fcca11bdb 936255a0384a : 4 revs, 0.000292 s, 0.000168 s, -0.000124 s, × 0.5753, 42 µs/rev
mozilla-try x_revs_x00_added_x_copies b53d2fadbdb5 017afae788ec : 2 revs, 0.003939 s, 0.001160 s, -0.002779 s, × 0.2945, 580 µs/rev
mozilla-try x_revs_x000_added_x000_copies 20408ad61ce5 6f0ee96e21ad : 1 revs, 0.033027 s, 0.033016 s, -0.000011 s, × 0.9997, 33016 µs/rev
mozilla-try x_revs_x0000_added_x0000_copies effb563bb7e5 c07a39dc4e80 : 6 revs, 0.073703 s, 0.073312 s, -0.39ae31 s, × 0.9947, 12218 µs/rev
mozilla-try x000_revs_xx00_added_0_copies 6100d773079a 04a55431795e : 1593 revs, 0.006469 s, 0.006485 s, +0.000016 s, × 1.0025, 4 µs/rev
mozilla-try x000_revs_x000_added_x_copies 9f17a6fc04f9 2d37b966abed : 41 revs, 0.005278 s, 0.005494 s, +0.000216 s, × 1.0409, 134 µs/rev
mozilla-try x000_revs_x000_added_x000_copies 1346fd0130e4 4c65cbdabc1f : 6657 revs, 0.064995 s, 0.064879 s, -0.000116 s, × 0.9982, 9 µs/rev
mozilla-try x0000_revs_x_added_0_copies 63519bfd42ee a36a2a865d92 : 40314 revs, 0.301041 s, 0.301469 s, +0.000428 s, × 1.0014, 7 µs/rev
mozilla-try x0000_revs_x_added_x_copies 9fe69ff0762d bcabf2a78927 : 38690 revs, 0.285575 s, 0.297113 s, +0.011538 s, × 1.0404, 7 µs/rev
mozilla-try x0000_revs_xx000_added_x_copies 156f6e2674f2 4d0f2c178e66 : 8598 revs, 0.085597 s, 0.085890 s, +0.000293 s, × 1.0034, 9 µs/rev
mozilla-try x0000_revs_xx000_added_0_copies 9eec5917337d 67118cc6dcad : 615 revs, 0.027118 s, 0.027718 s, +0.000600 s, × 1.0221, 45 µs/rev
mozilla-try x0000_revs_xx000_added_x000_copies 89294cd501d9 7ccb2fc7ccb5 : 97052 revs, 2.119204 s, 2.048949 s, -0.070255 s, × 0.9668, 21 µs/rev
mozilla-try x0000_revs_x0000_added_x0000_copies e928c65095ed e951f4ad123a : 52031 revs, 0.701479 s, 0.685924 s, -0.015555 s, × 0.9778, 13 µs/rev
mozilla-try x00000_revs_x_added_0_copies 6a320851d377 1ebb79acd503 : 363753 revs, 4.482399 s, 4.482891 s, +0.000492 s, × 1.0001, 12 µs/rev
mozilla-try x00000_revs_x00000_added_0_copies dc8a3ca7010e d16fde900c9c : 34414 revs, 0.574082 s, 0.577633 s, +0.003551 s, × 1.0062, 16 µs/rev
mozilla-try x00000_revs_x_added_x_copies 5173c4b6f97c 95d83ee7242d : 362229 revs, 4.480366 s, 4.397816 s, -0.082550 s, × 0.9816, 12 µs/rev
mozilla-try x00000_revs_x000_added_x_copies 9126823d0e9c ca82787bb23c : 359344 revs, 4.369070 s, 4.370538 s, +0.001468 s, × 1.0003, 12 µs/rev
mozilla-try x00000_revs_x0000_added_x0000_copies 8d3fafa80d4b eb884023b810 : 192665 revs, 1.592506 s, 1.570439 s, -0.022067 s, × 0.9861, 8 µs/rev
mozilla-try x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d : 228985 revs, 87.824489 s, 88.388512 s, +0.564023 s, × 1.0064, 386 µs/rev
mozilla-try x00000_revs_x00000_added_x000_copies 9b2a99adc05e 8e29777b48e6 : 382065 revs, 43.304637 s, 34.443661 s, -8.860976 s, × 0.7954, 90 µs/rev
private : 459513 revs, 33.853687 s, 27.370148 s, -6.483539 s, × 0.8085, 59 µs/rev
Differential Revision: https://phab.mercurial-scm.org/D9653
author | Pierre-Yves David <pierre-yves.david@octobus.net> |
---|---|
date | Wed, 16 Dec 2020 11:11:05 +0100 |
parents | 89a2afe31e82 |
children | 6000f5b25c9b |
line wrap: on
line source
from __future__ import absolute_import import os import sys import unittest # TODO migrate to canned cbor test strings and stop using thirdparty.cbor tpp = os.path.normpath( os.path.join(os.path.dirname(__file__), '..', 'mercurial', 'thirdparty') ) if not os.path.exists(tpp): # skip, not in a repo sys.exit(80) sys.path[0:0] = [tpp] import cbor del sys.path[0] from mercurial.utils import cborutil class TestCase(unittest.TestCase): if not getattr(unittest.TestCase, 'assertRaisesRegex', False): # Python 3.7 deprecates the regex*p* version, but 2.7 lacks # the regex version. assertRaisesRegex = ( # camelcase-required unittest.TestCase.assertRaisesRegexp ) def loadit(it): return cbor.loads(b''.join(it)) class BytestringTests(TestCase): def testsimple(self): self.assertEqual( list(cborutil.streamencode(b'foobar')), [b'\x46', b'foobar'] ) self.assertEqual(loadit(cborutil.streamencode(b'foobar')), b'foobar') self.assertEqual(cborutil.decodeall(b'\x46foobar'), [b'foobar']) self.assertEqual( cborutil.decodeall(b'\x46foobar\x45fizbi'), [b'foobar', b'fizbi'] ) def testlong(self): source = b'x' * 1048576 self.assertEqual(loadit(cborutil.streamencode(source)), source) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) def testfromiter(self): # This is the example from RFC 7049 Section 2.2.2. source = [b'\xaa\xbb\xcc\xdd', b'\xee\xff\x99'] self.assertEqual( list(cborutil.streamencodebytestringfromiter(source)), [ b'\x5f', b'\x44', b'\xaa\xbb\xcc\xdd', b'\x43', b'\xee\xff\x99', b'\xff', ], ) self.assertEqual( loadit(cborutil.streamencodebytestringfromiter(source)), b''.join(source), ) self.assertEqual( cborutil.decodeall( b'\x5f\x44\xaa\xbb\xcc\xdd' b'\x43\xee\xff\x99\xff' ), [b'\xaa\xbb\xcc\xdd', b'\xee\xff\x99', b''], ) for i, chunk in enumerate( cborutil.decodeall( b'\x5f\x44\xaa\xbb\xcc\xdd' b'\x43\xee\xff\x99\xff' ) ): self.assertIsInstance(chunk, cborutil.bytestringchunk) if i == 0: self.assertTrue(chunk.isfirst) else: self.assertFalse(chunk.isfirst) if i == 2: self.assertTrue(chunk.islast) else: self.assertFalse(chunk.islast) def testfromiterlarge(self): source = [b'a' * 16, b'b' * 128, b'c' * 1024, b'd' * 1048576] self.assertEqual( loadit(cborutil.streamencodebytestringfromiter(source)), b''.join(source), ) def testindefinite(self): source = b'\x00\x01\x02\x03' + b'\xff' * 16384 it = cborutil.streamencodeindefinitebytestring(source, chunksize=2) self.assertEqual(next(it), b'\x5f') self.assertEqual(next(it), b'\x42') self.assertEqual(next(it), b'\x00\x01') self.assertEqual(next(it), b'\x42') self.assertEqual(next(it), b'\x02\x03') self.assertEqual(next(it), b'\x42') self.assertEqual(next(it), b'\xff\xff') dest = b''.join( cborutil.streamencodeindefinitebytestring(source, chunksize=42) ) self.assertEqual(cbor.loads(dest), source) self.assertEqual(b''.join(cborutil.decodeall(dest)), source) for chunk in cborutil.decodeall(dest): self.assertIsInstance(chunk, cborutil.bytestringchunk) self.assertIn(len(chunk), (0, 8, 42)) encoded = b'\x5f\xff' b = cborutil.decodeall(encoded) self.assertEqual(b, [b'']) self.assertTrue(b[0].isfirst) self.assertTrue(b[0].islast) def testdecodevariouslengths(self): for i in (0, 1, 22, 23, 24, 25, 254, 255, 256, 65534, 65535, 65536): source = b'x' * i encoded = b''.join(cborutil.streamencode(source)) if len(source) < 24: hlen = 1 elif len(source) < 256: hlen = 2 elif len(source) < 65536: hlen = 3 elif len(source) < 1048576: hlen = 5 self.assertEqual( cborutil.decodeitem(encoded), (True, source, hlen + len(source), cborutil.SPECIAL_NONE), ) def testpartialdecode(self): encoded = b''.join(cborutil.streamencode(b'foobar')) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -6, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -5, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:7]), (True, b'foobar', 7, cborutil.SPECIAL_NONE), ) def testpartialdecodevariouslengths(self): lens = [ 2, 3, 10, 23, 24, 25, 31, 100, 254, 255, 256, 257, 16384, 65534, 65535, 65536, 65537, 131071, 131072, 131073, 1048575, 1048576, 1048577, ] for size in lens: if size < 24: hlen = 1 elif size < 2 ** 8: hlen = 2 elif size < 2 ** 16: hlen = 3 elif size < 2 ** 32: hlen = 5 else: assert False source = b'x' * size encoded = b''.join(cborutil.streamencode(source)) res = cborutil.decodeitem(encoded[0:1]) if hlen > 1: self.assertEqual( res, (False, None, -(hlen - 1), cborutil.SPECIAL_NONE) ) else: self.assertEqual( res, (False, None, -(size + hlen - 1), cborutil.SPECIAL_NONE), ) # Decoding partial header reports remaining header size. for i in range(hlen - 1): self.assertEqual( cborutil.decodeitem(encoded[0 : i + 1]), (False, None, -(hlen - i - 1), cborutil.SPECIAL_NONE), ) # Decoding complete header reports item size. self.assertEqual( cborutil.decodeitem(encoded[0:hlen]), (False, None, -size, cborutil.SPECIAL_NONE), ) # Decoding single byte after header reports item size - 1 self.assertEqual( cborutil.decodeitem(encoded[0 : hlen + 1]), (False, None, -(size - 1), cborutil.SPECIAL_NONE), ) # Decoding all but the last byte reports -1 needed. self.assertEqual( cborutil.decodeitem(encoded[0 : hlen + size - 1]), (False, None, -1, cborutil.SPECIAL_NONE), ) # Decoding last byte retrieves value. self.assertEqual( cborutil.decodeitem(encoded[0 : hlen + size]), (True, source, hlen + size, cborutil.SPECIAL_NONE), ) def testindefinitepartialdecode(self): encoded = b''.join( cborutil.streamencodebytestringfromiter([b'foobar', b'biz']) ) # First item should be begin of bytestring special. self.assertEqual( cborutil.decodeitem(encoded[0:1]), (True, None, 1, cborutil.SPECIAL_START_INDEFINITE_BYTESTRING), ) # Second item should be the first chunk. But only available when # we give it 7 bytes (1 byte header + 6 byte chunk). self.assertEqual( cborutil.decodeitem(encoded[1:2]), (False, None, -6, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:3]), (False, None, -5, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:4]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:5]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:6]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:7]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[1:8]), (True, b'foobar', 7, cborutil.SPECIAL_NONE), ) # Third item should be second chunk. But only available when # we give it 4 bytes (1 byte header + 3 byte chunk). self.assertEqual( cborutil.decodeitem(encoded[8:9]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[8:10]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[8:11]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[8:12]), (True, b'biz', 4, cborutil.SPECIAL_NONE), ) # Fourth item should be end of indefinite stream marker. self.assertEqual( cborutil.decodeitem(encoded[12:13]), (True, None, 1, cborutil.SPECIAL_INDEFINITE_BREAK), ) # Now test the behavior when going through the decoder. self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:1]), (False, 1, 0) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:2]), (False, 1, 6) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:3]), (False, 1, 5) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:4]), (False, 1, 4) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:5]), (False, 1, 3) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:6]), (False, 1, 2) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:7]), (False, 1, 1) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:8]), (True, 8, 0) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:9]), (True, 8, 3) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:10]), (True, 8, 2) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:11]), (True, 8, 1) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:12]), (True, 12, 0) ) self.assertEqual( cborutil.sansiodecoder().decode(encoded[0:13]), (True, 13, 0) ) decoder = cborutil.sansiodecoder() decoder.decode(encoded[0:8]) values = decoder.getavailable() self.assertEqual(values, [b'foobar']) self.assertTrue(values[0].isfirst) self.assertFalse(values[0].islast) self.assertEqual(decoder.decode(encoded[8:12]), (True, 4, 0)) values = decoder.getavailable() self.assertEqual(values, [b'biz']) self.assertFalse(values[0].isfirst) self.assertFalse(values[0].islast) self.assertEqual(decoder.decode(encoded[12:]), (True, 1, 0)) values = decoder.getavailable() self.assertEqual(values, [b'']) self.assertFalse(values[0].isfirst) self.assertTrue(values[0].islast) class StringTests(TestCase): def testdecodeforbidden(self): encoded = b'\x63foo' with self.assertRaisesRegex( cborutil.CBORDecodeError, 'string major type not supported' ): cborutil.decodeall(encoded) class IntTests(TestCase): def testsmall(self): self.assertEqual(list(cborutil.streamencode(0)), [b'\x00']) self.assertEqual(cborutil.decodeall(b'\x00'), [0]) self.assertEqual(list(cborutil.streamencode(1)), [b'\x01']) self.assertEqual(cborutil.decodeall(b'\x01'), [1]) self.assertEqual(list(cborutil.streamencode(2)), [b'\x02']) self.assertEqual(cborutil.decodeall(b'\x02'), [2]) self.assertEqual(list(cborutil.streamencode(3)), [b'\x03']) self.assertEqual(cborutil.decodeall(b'\x03'), [3]) self.assertEqual(list(cborutil.streamencode(4)), [b'\x04']) self.assertEqual(cborutil.decodeall(b'\x04'), [4]) # Multiple value decode works. self.assertEqual( cborutil.decodeall(b'\x00\x01\x02\x03\x04'), [0, 1, 2, 3, 4] ) def testnegativesmall(self): self.assertEqual(list(cborutil.streamencode(-1)), [b'\x20']) self.assertEqual(cborutil.decodeall(b'\x20'), [-1]) self.assertEqual(list(cborutil.streamencode(-2)), [b'\x21']) self.assertEqual(cborutil.decodeall(b'\x21'), [-2]) self.assertEqual(list(cborutil.streamencode(-3)), [b'\x22']) self.assertEqual(cborutil.decodeall(b'\x22'), [-3]) self.assertEqual(list(cborutil.streamencode(-4)), [b'\x23']) self.assertEqual(cborutil.decodeall(b'\x23'), [-4]) self.assertEqual(list(cborutil.streamencode(-5)), [b'\x24']) self.assertEqual(cborutil.decodeall(b'\x24'), [-5]) # Multiple value decode works. self.assertEqual( cborutil.decodeall(b'\x20\x21\x22\x23\x24'), [-1, -2, -3, -4, -5] ) def testrange(self): for i in range(-70000, 70000, 10): encoded = b''.join(cborutil.streamencode(i)) self.assertEqual(encoded, cbor.dumps(i)) self.assertEqual(cborutil.decodeall(encoded), [i]) def testdecodepartialubyte(self): encoded = b''.join(cborutil.streamencode(250)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 250, 2, cborutil.SPECIAL_NONE), ) def testdecodepartialbyte(self): encoded = b''.join(cborutil.streamencode(-42)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, -42, 2, cborutil.SPECIAL_NONE), ) def testdecodepartialushort(self): encoded = b''.join(cborutil.streamencode(2 ** 15)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, 2 ** 15, 3, cborutil.SPECIAL_NONE), ) def testdecodepartialshort(self): encoded = b''.join(cborutil.streamencode(-1024)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (True, -1024, 3, cborutil.SPECIAL_NONE), ) def testdecodepartialulong(self): encoded = b''.join(cborutil.streamencode(2 ** 28)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, 2 ** 28, 5, cborutil.SPECIAL_NONE), ) def testdecodepartiallong(self): encoded = b''.join(cborutil.streamencode(-1048580)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, -1048580, 5, cborutil.SPECIAL_NONE), ) def testdecodepartialulonglong(self): encoded = b''.join(cborutil.streamencode(2 ** 32)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -8, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -7, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -6, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -5, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:7]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:8]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:9]), (True, 2 ** 32, 9, cborutil.SPECIAL_NONE), ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'input data not fully consumed' ): cborutil.decodeall(encoded[0:1]) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'input data not fully consumed' ): cborutil.decodeall(encoded[0:2]) def testdecodepartiallonglong(self): encoded = b''.join(cborutil.streamencode(-7000000000)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -8, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -7, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -6, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -5, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:7]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:8]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:9]), (True, -7000000000, 9, cborutil.SPECIAL_NONE), ) class ArrayTests(TestCase): def testempty(self): self.assertEqual(list(cborutil.streamencode([])), [b'\x80']) self.assertEqual(loadit(cborutil.streamencode([])), []) self.assertEqual(cborutil.decodeall(b'\x80'), [[]]) def testbasic(self): source = [b'foo', b'bar', 1, -10] chunks = [b'\x84', b'\x43', b'foo', b'\x43', b'bar', b'\x01', b'\x29'] self.assertEqual(list(cborutil.streamencode(source)), chunks) self.assertEqual(cborutil.decodeall(b''.join(chunks)), [source]) def testemptyfromiter(self): self.assertEqual( b''.join(cborutil.streamencodearrayfromiter([])), b'\x9f\xff' ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length uint not allowed' ): cborutil.decodeall(b'\x9f\xff') def testfromiter1(self): source = [b'foo'] self.assertEqual( list(cborutil.streamencodearrayfromiter(source)), [ b'\x9f', b'\x43', b'foo', b'\xff', ], ) dest = b''.join(cborutil.streamencodearrayfromiter(source)) self.assertEqual(cbor.loads(dest), source) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length uint not allowed' ): cborutil.decodeall(dest) def testtuple(self): source = (b'foo', None, 42) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cbor.loads(encoded), list(source)) self.assertEqual(cborutil.decodeall(encoded), [list(source)]) def testpartialdecode(self): source = list(range(4)) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (True, 4, 1, cborutil.SPECIAL_START_ARRAY), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 4, 1, cborutil.SPECIAL_START_ARRAY), ) source = list(range(23)) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (True, 23, 1, cborutil.SPECIAL_START_ARRAY), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 23, 1, cborutil.SPECIAL_START_ARRAY), ) source = list(range(24)) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 24, 2, cborutil.SPECIAL_START_ARRAY), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (True, 24, 2, cborutil.SPECIAL_START_ARRAY), ) source = list(range(256)) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (True, 256, 3, cborutil.SPECIAL_START_ARRAY), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (True, 256, 3, cborutil.SPECIAL_START_ARRAY), ) def testnested(self): source = [[], [], [[], [], []]] encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) source = [True, None, [True, 0, 2], [None], [], [[[]], -87]] encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) # A set within an array. source = [None, {b'foo', b'bar', None, False}, set()] encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) # A map within an array. source = [None, {}, {b'foo': b'bar', True: False}, [{}]] encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) def testindefinitebytestringvalues(self): # Single value array whose value is an empty indefinite bytestring. encoded = b'\x81\x5f\x40\xff' with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length bytestrings not ' 'allowed as array values', ): cborutil.decodeall(encoded) class SetTests(TestCase): def testempty(self): self.assertEqual( list(cborutil.streamencode(set())), [ b'\xd9\x01\x02', b'\x80', ], ) self.assertEqual(cborutil.decodeall(b'\xd9\x01\x02\x80'), [set()]) def testset(self): source = {b'foo', None, 42} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cbor.loads(encoded), source) self.assertEqual(cborutil.decodeall(encoded), [source]) def testinvalidtag(self): # Must use array to encode sets. encoded = b'\xd9\x01\x02\xa0' with self.assertRaisesRegex( cborutil.CBORDecodeError, 'expected array after finite set ' 'semantic tag', ): cborutil.decodeall(encoded) def testpartialdecode(self): # Semantic tag item will be 3 bytes. Set header will be variable # depending on length. encoded = b''.join(cborutil.streamencode({i for i in range(23)})) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (True, 23, 4, cborutil.SPECIAL_START_SET), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, 23, 4, cborutil.SPECIAL_START_SET), ) encoded = b''.join(cborutil.streamencode({i for i in range(24)})) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, 24, 5, cborutil.SPECIAL_START_SET), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (True, 24, 5, cborutil.SPECIAL_START_SET), ) encoded = b''.join(cborutil.streamencode({i for i in range(256)})) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (True, 256, 6, cborutil.SPECIAL_START_SET), ) def testinvalidvalue(self): encoded = b''.join( [ b'\xd9\x01\x02', # semantic tag b'\x81', # array of size 1 b'\x5f\x43foo\xff', # indefinite length bytestring "foo" ] ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length bytestrings not ' 'allowed as set values', ): cborutil.decodeall(encoded) encoded = b''.join( [ b'\xd9\x01\x02', b'\x81', b'\x80', ] ) # empty array with self.assertRaisesRegex( cborutil.CBORDecodeError, 'collections not allowed as set values' ): cborutil.decodeall(encoded) encoded = b''.join( [ b'\xd9\x01\x02', b'\x81', b'\xa0', ] ) # empty map with self.assertRaisesRegex( cborutil.CBORDecodeError, 'collections not allowed as set values' ): cborutil.decodeall(encoded) encoded = b''.join( [ b'\xd9\x01\x02', b'\x81', b'\xd9\x01\x02\x81\x01', # set with integer 1 ] ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'collections not allowed as set values' ): cborutil.decodeall(encoded) class BoolTests(TestCase): def testbasic(self): self.assertEqual(list(cborutil.streamencode(True)), [b'\xf5']) self.assertEqual(list(cborutil.streamencode(False)), [b'\xf4']) self.assertIs(loadit(cborutil.streamencode(True)), True) self.assertIs(loadit(cborutil.streamencode(False)), False) self.assertEqual(cborutil.decodeall(b'\xf4'), [False]) self.assertEqual(cborutil.decodeall(b'\xf5'), [True]) self.assertEqual( cborutil.decodeall(b'\xf4\xf5\xf5\xf4'), [False, True, True, False] ) class NoneTests(TestCase): def testbasic(self): self.assertEqual(list(cborutil.streamencode(None)), [b'\xf6']) self.assertIs(loadit(cborutil.streamencode(None)), None) self.assertEqual(cborutil.decodeall(b'\xf6'), [None]) self.assertEqual(cborutil.decodeall(b'\xf6\xf6'), [None, None]) class MapTests(TestCase): def testempty(self): self.assertEqual(list(cborutil.streamencode({})), [b'\xa0']) self.assertEqual(loadit(cborutil.streamencode({})), {}) self.assertEqual(cborutil.decodeall(b'\xa0'), [{}]) def testemptyindefinite(self): self.assertEqual( list(cborutil.streamencodemapfromiter([])), [b'\xbf', b'\xff'] ) self.assertEqual(loadit(cborutil.streamencodemapfromiter([])), {}) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length uint not allowed' ): cborutil.decodeall(b'\xbf\xff') def testone(self): source = {b'foo': b'bar'} self.assertEqual( list(cborutil.streamencode(source)), [b'\xa1', b'\x43', b'foo', b'\x43', b'bar'], ) self.assertEqual(loadit(cborutil.streamencode(source)), source) self.assertEqual(cborutil.decodeall(b'\xa1\x43foo\x43bar'), [source]) def testmultiple(self): source = { b'foo': b'bar', b'baz': b'value1', } self.assertEqual(loadit(cborutil.streamencode(source)), source) self.assertEqual( loadit(cborutil.streamencodemapfromiter(source.items())), source ) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) def testcomplex(self): source = { b'key': 1, 2: -10, } self.assertEqual(loadit(cborutil.streamencode(source)), source) self.assertEqual( loadit(cborutil.streamencodemapfromiter(source.items())), source ) encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) def testnested(self): source = {b'key1': None, b'key2': {b'sub1': b'sub2'}, b'sub2': {}} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) source = { b'key1': [], b'key2': [None, False], b'key3': {b'foo', b'bar'}, b'key4': {}, } encoded = b''.join(cborutil.streamencode(source)) self.assertEqual(cborutil.decodeall(encoded), [source]) def testillegalkey(self): encoded = b''.join( [ # map header + len 1 b'\xa1', # indefinite length bytestring "foo" in key position b'\x5f\x03foo\xff', ] ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length bytestrings not ' 'allowed as map keys', ): cborutil.decodeall(encoded) encoded = b''.join( [ b'\xa1', b'\x80', b'\x43foo', ] ) # empty array with self.assertRaisesRegex( cborutil.CBORDecodeError, 'collections not supported as map keys' ): cborutil.decodeall(encoded) def testillegalvalue(self): encoded = b''.join( [ b'\xa1', # map headers b'\x43foo', # key b'\x5f\x03bar\xff', # indefinite length value ] ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'indefinite length bytestrings not ' 'allowed as map values', ): cborutil.decodeall(encoded) def testpartialdecode(self): source = {b'key1': b'value1'} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (True, 1, 1, cborutil.SPECIAL_START_MAP), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 1, 1, cborutil.SPECIAL_START_MAP), ) source = {b'key%d' % i: None for i in range(23)} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (True, 23, 1, cborutil.SPECIAL_START_MAP), ) source = {b'key%d' % i: None for i in range(24)} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (True, 24, 2, cborutil.SPECIAL_START_MAP), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (True, 24, 2, cborutil.SPECIAL_START_MAP), ) source = {b'key%d' % i: None for i in range(256)} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (True, 256, 3, cborutil.SPECIAL_START_MAP), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (True, 256, 3, cborutil.SPECIAL_START_MAP), ) source = {b'key%d' % i: None for i in range(65536)} encoded = b''.join(cborutil.streamencode(source)) self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -4, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -3, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:3]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:4]), (False, None, -1, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:5]), (True, 65536, 5, cborutil.SPECIAL_START_MAP), ) self.assertEqual( cborutil.decodeitem(encoded[0:6]), (True, 65536, 5, cborutil.SPECIAL_START_MAP), ) class SemanticTagTests(TestCase): def testdecodeforbidden(self): for i in range(500): if i == cborutil.SEMANTIC_TAG_FINITE_SET: continue tag = cborutil.encodelength(cborutil.MAJOR_TYPE_SEMANTIC, i) encoded = tag + cborutil.encodelength(cborutil.MAJOR_TYPE_UINT, 42) # Partial decode is incomplete. if i < 24: pass elif i < 256: self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -1, cborutil.SPECIAL_NONE), ) elif i < 65536: self.assertEqual( cborutil.decodeitem(encoded[0:1]), (False, None, -2, cborutil.SPECIAL_NONE), ) self.assertEqual( cborutil.decodeitem(encoded[0:2]), (False, None, -1, cborutil.SPECIAL_NONE), ) with self.assertRaisesRegex( cborutil.CBORDecodeError, r'semantic tag \d+ not allowed' ): cborutil.decodeitem(encoded) class SpecialTypesTests(TestCase): def testforbiddentypes(self): for i in range(256): if i == cborutil.SUBTYPE_FALSE: continue elif i == cborutil.SUBTYPE_TRUE: continue elif i == cborutil.SUBTYPE_NULL: continue encoded = cborutil.encodelength(cborutil.MAJOR_TYPE_SPECIAL, i) with self.assertRaisesRegex( cborutil.CBORDecodeError, r'special type \d+ not allowed' ): cborutil.decodeitem(encoded) class SansIODecoderTests(TestCase): def testemptyinput(self): decoder = cborutil.sansiodecoder() self.assertEqual(decoder.decode(b''), (False, 0, 0)) class BufferingDecoderTests(TestCase): def testsimple(self): source = [ b'foobar', b'x' * 128, {b'foo': b'bar'}, True, False, None, [None] * 128, ] encoded = b''.join(cborutil.streamencode(source)) for step in range(1, 32): decoder = cborutil.bufferingdecoder() start = 0 while start < len(encoded): decoder.decode(encoded[start : start + step]) start += step self.assertEqual(decoder.getavailable(), [source]) def testbytearray(self): source = b''.join(cborutil.streamencode(b'foobar')) decoder = cborutil.bufferingdecoder() decoder.decode(bytearray(source)) self.assertEqual(decoder.getavailable(), [b'foobar']) class DecodeallTests(TestCase): def testemptyinput(self): self.assertEqual(cborutil.decodeall(b''), []) def testpartialinput(self): encoded = b''.join( [ b'\x82', b'\x01', ] # array of 2 elements # integer 1 ) with self.assertRaisesRegex( cborutil.CBORDecodeError, 'input data not complete' ): cborutil.decodeall(encoded) if __name__ == '__main__': import silenttestrunner silenttestrunner.main(__name__)