view i18n/posplit @ 13169:4c1fbed00224 stable

fncachestore: copy dh directory before the manifest

Before this patch, the copy order on clone was:

  requires
  00changelog.i
  store\data
  store\00manifest.d
  store\00manifest.i
  store\00changelog.d
  store\00changelog.i
  store\dh
  store\fncache

Which provides a theoretical non-zero probability of a race during clone where
a very early reader might see a repository with missing revlog files if it sees
00changelog.i before all files inside dh have been copied.

The dh directory is similar to the data directory -- just for files with long
names (which are hashed). The manifest refers to files in data *and* dh, so dh
should be copied before the manifest.

This patch improves the copy order to:

  requires
  00changelog.i
  store\data
  store\dh
  store\fncache
  store\00manifest.d
  store\00manifest.i
  store\00changelog.d
  store\00changelog.i

I'm putting fncache to before the manifest while I'm at it, since fncache
provides a mechanism to enumerate all repository files without visiting the
manifest revisions. fncache depends only on data and dh.

Note that data must be copied first, since copying data triggers the creation
of the repository write lock in the destination repo (see hg.clone).

#!/usr/bin/env python
#
# posplit - split messages in paragraphs on .po/.pot files
#
# license: MIT/X11/Expat
#

import sys
import polib

def addentry(po, entry, cache):
    e = cache.get(entry.msgid)
    if e:
        e.occurrences.extend(entry.occurrences)
    else:
        po.append(entry)
        cache[entry.msgid] = entry

def mkentry(orig, delta, msgid, msgstr):
    entry = polib.POEntry()
    entry.merge(orig)
    entry.msgid = msgid or orig.msgid
    entry.msgstr = msgstr or orig.msgstr
    entry.occurrences = [(p, int(l) + delta) for (p, l) in orig.occurrences]
    return entry

if __name__ == "__main__":
    po = polib.pofile(sys.argv[1])

    cache = {}
    entries = po[:]
    po[:] = []
    for entry in entries:
        msgids = entry.msgid.split(u'\n\n')
        if entry.msgstr:
            msgstrs = entry.msgstr.split(u'\n\n')
        else:
            msgstrs = [u''] * len(msgids)

        if len(msgids) != len(msgstrs):
            # places the whole existing translation as a fuzzy
            # translation for each paragraph, to give the
            # translator a chance to recover part of the old
            # translation - erasing extra paragraphs is
            # probably better than retranslating all from start
            if 'fuzzy' not in entry.flags:
                entry.flags.append('fuzzy')
            msgstrs = [entry.msgstr] * len(msgids)

        delta = 0
        for msgid, msgstr in zip(msgids, msgstrs):
            if msgid:
                newentry = mkentry(entry, delta, msgid, msgstr)
                addentry(po, newentry, cache)
            delta += 2 + msgid.count('\n')
    po.save()