i18n/posplit

localrepo: move new repo requirements into standalone function (API)

This patch extracts the code for determining requirements for a new
repo into a standalone function. By doing so, future code that will
perform an in-place repository upgrade (e.g. to generaldelta) can
examine the set of proposed new requirements and possibly take
additional actions (such as adding dotencode or fncache) when
performing the upgrade.

This patch is marked as API because _baserequirements (which was added
in b090601a80d1 so extensions could override it) has been removed and
will presumably impact whatever extension it was added for. Consumers
should be able to monkeypatch the new function to achieve the same
functionality.

The "create" argument has been dropped because the function is only
called in one location and "create" is always true in that case.

While it makes logical sense for this code to be a method so extensions
can implement a custom repo class / method to override it, this won't
actually work. This is because requirements determination occurs during
localrepository.__init__ and this is before the "reposetup"
"callback" is fired. So, the only way for extensions to customize
requirements would be to overwrite localrepo.localrepository or to
monkeypatch a function on a module during extsetup(). Since we try to
keep localrepository small, we use a standalone function. There is
probably room to offer extensions a "hook" point to alter repository
creation. But that is scope bloat.

#!/usr/bin/env python
#
# posplit - split messages in paragraphs on .po/.pot files
#
# license: MIT/X11/Expat
#

import re
import sys
import polib

def addentry(po, entry, cache):
    e = cache.get(entry.msgid)
    if e:
        e.occurrences.extend(entry.occurrences)
    else:
        po.append(entry)
        cache[entry.msgid] = entry

def mkentry(orig, delta, msgid, msgstr):
    entry = polib.POEntry()
    entry.merge(orig)
    entry.msgid = msgid or orig.msgid
    entry.msgstr = msgstr or orig.msgstr
    entry.occurrences = [(p, int(l) + delta) for (p, l) in orig.occurrences]
    return entry

if __name__ == "__main__":
    po = polib.pofile(sys.argv[1])

    cache = {}
    entries = po[:]
    po[:] = []
    findd = re.compile(r' *\.\. (\w+)::') # for finding directives
    for entry in entries:
        msgids = entry.msgid.split(u'\n\n')
        if entry.msgstr:
            msgstrs = entry.msgstr.split(u'\n\n')
        else:
            msgstrs = [u''] * len(msgids)

        if len(msgids) != len(msgstrs):
            # places the whole existing translation as a fuzzy
            # translation for each paragraph, to give the
            # translator a chance to recover part of the old
            # translation - erasing extra paragraphs is
            # probably better than retranslating all from start
            if 'fuzzy' not in entry.flags:
                entry.flags.append('fuzzy')
            msgstrs = [entry.msgstr] * len(msgids)

        delta = 0
        for msgid, msgstr in zip(msgids, msgstrs):
            if msgid and msgid != '::':
                newentry = mkentry(entry, delta, msgid, msgstr)
                mdirective = findd.match(msgid)
                if mdirective:
                    if not msgid[mdirective.end():].rstrip():
                        # only directive, nothing to translate here
                        delta += 2
                        continue
                    directive = mdirective.group(1)
                    if directive in ('container', 'include'):
                        if msgid.rstrip('\n').count('\n') == 0:
                            # only rst syntax, nothing to translate
                            delta += 2
                            continue
                        else:
                            # lines following directly, unexpected
                            print 'Warning: text follows line with directive' \
                                  ' %s' % directive
                    comment = 'do not translate: .. %s::' % directive
                    if not newentry.comment:
                        newentry.comment = comment
                    elif comment not in newentry.comment:
                        newentry.comment += '\n' + comment
                addentry(po, newentry, cache)
            delta += 2 + msgid.count('\n')
    po.save()