mercurial/pure/base85.py
author Gregory Szorc <gregory.szorc@gmail.com>
Sun, 14 Aug 2016 16:30:44 -0700
changeset 29781 2654a0aac80d
parent 27334 9007f697e8ef
child 35944 01b4d88ccb24
permissions -rw-r--r--
profiling: move profiling code from dispatch.py (API) Currently, profiling code lives in dispatch.py, which is a low-level module centered around command dispatch. Furthermore, dispatch.py imports a lot of other modules, meaning that importing dispatch.py to get at profiling functionality would often result in a module import cycle. Profiling is a generic activity. It shouldn't be limited to command dispatch. This patch moves profiling code from dispatch.py to the new profiling.py. The low-level "run a profiler against a function" functions have been moved verbatim. The code for determining how to invoke the profiler has been extracted to its own function. I decided to create a new module rather than stick this code elsewhere (such as util.py) because util.py is already quite large. And, I foresee this file growing larger once Facebook's profiling enhancements get added to it.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     1
# base85.py: pure python base85 codec
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     2
#
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     3
# Copyright (C) 2009 Brendan Cully <brendan@kublai.com>
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     4
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 7881
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9029
diff changeset
     6
# GNU General Public License version 2 or any later version.
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
     7
27334
9007f697e8ef base85: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 16598
diff changeset
     8
from __future__ import absolute_import
9007f697e8ef base85: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 16598
diff changeset
     9
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    10
import struct
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    11
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    12
_b85chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" \
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    13
            "abcdefghijklmnopqrstuvwxyz!#$%&()*+-;<=>?@^_`{|}~"
7835
2505e9f84153 Optimization of pure.base85.b85encode
Mads Kiilerich <mads@kiilerich.com>
parents: 7701
diff changeset
    14
_b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    15
_b85dec = {}
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    16
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    17
def _mkb85dec():
8632
9e055cfdd620 replace "i in range(len(xs))" with "i, x in enumerate(xs)"
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    18
    for i, c in enumerate(_b85chars):
9e055cfdd620 replace "i in range(len(xs))" with "i, x in enumerate(xs)"
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    19
        _b85dec[c] = i
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    20
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    21
def b85encode(text, pad=False):
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    22
    """encode text in base85 format"""
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    23
    l = len(text)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    24
    r = l % 4
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    25
    if r:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    26
        text += '\0' * (4 - r)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    27
    longs = len(text) >> 2
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    28
    words = struct.unpack('>%dL' % (longs), text)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    29
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8632
diff changeset
    30
    out = ''.join(_b85chars[(word // 52200625) % 85] +
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8632
diff changeset
    31
                  _b85chars2[(word // 7225) % 7225] +
7835
2505e9f84153 Optimization of pure.base85.b85encode
Mads Kiilerich <mads@kiilerich.com>
parents: 7701
diff changeset
    32
                  _b85chars2[word % 7225]
2505e9f84153 Optimization of pure.base85.b85encode
Mads Kiilerich <mads@kiilerich.com>
parents: 7701
diff changeset
    33
                  for word in words)
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    34
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    35
    if pad:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    36
        return out
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    37
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    38
    # Trim padding
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    39
    olen = l % 4
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    40
    if olen:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    41
        olen += 1
9029
0001e49f1c11 compat: use // for integer division
Alejandro Santos <alejolp@alejolp.com>
parents: 8632
diff changeset
    42
    olen += l // 4 * 5
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    43
    return out[:olen]
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    44
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    45
def b85decode(text):
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    46
    """decode base85-encoded text"""
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    47
    if not _b85dec:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    48
        _mkb85dec()
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    49
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    50
    l = len(text)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    51
    out = []
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    52
    for i in range(0, len(text), 5):
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
    53
        chunk = text[i:i + 5]
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    54
        acc = 0
8632
9e055cfdd620 replace "i in range(len(xs))" with "i, x in enumerate(xs)"
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    55
        for j, c in enumerate(chunk):
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    56
            try:
8632
9e055cfdd620 replace "i in range(len(xs))" with "i, x in enumerate(xs)"
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    57
                acc = acc * 85 + _b85dec[c]
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    58
            except KeyError:
16598
20a9d823f242 pure/base85: align exception type/msg on base85.c
Patrick Mezard <patrick@mezard.eu>
parents: 10282
diff changeset
    59
                raise ValueError('bad base85 character at position %d'
20a9d823f242 pure/base85: align exception type/msg on base85.c
Patrick Mezard <patrick@mezard.eu>
parents: 10282
diff changeset
    60
                                 % (i + j))
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    61
        if acc > 4294967295:
16598
20a9d823f242 pure/base85: align exception type/msg on base85.c
Patrick Mezard <patrick@mezard.eu>
parents: 10282
diff changeset
    62
            raise ValueError('Base85 overflow in hunk starting at byte %d' % i)
7701
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    63
        out.append(acc)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    64
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    65
    # Pad final chunk if necessary
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    66
    cl = l % 5
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    67
    if cl:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    68
        acc *= 85 ** (5 - cl)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    69
        if cl > 1:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    70
            acc += 0xffffff >> (cl - 2) * 8
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    71
        out[-1] = acc
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    72
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    73
    out = struct.pack('>%dL' % (len(out)), *out)
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    74
    if cl:
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    75
        out = out[:-(5 - cl)]
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    76
4bdead043d8d Pure python base85 fallback
Brendan Cully <brendan@kublai.com>
parents:
diff changeset
    77
    return out