i18n/posplit

wix: functionality to automate building WiX installers

Like we did for Inno Setup, we want to make it easier to
produce WiX installers. This commit does that.

We introduce a new hgpackaging.wix module for performing
all the high-level tasks required to produce WiX installers.
This required miscellaneous enhancements to existing code in
hgpackaging, including support for signing binaries.

A new build.py script for calling into the module APIs has been
created. It behaves very similarly to the Inno Setup build.py
script.

Unlike Inno Setup, we didn't have code in the repo previously
to generate WiX installers. It appears that all existing
automation for building WiX installers lives in the
https://bitbucket.org/tortoisehg/thg-winbuild repository - most
notably in its setup.py file. My strategy for inventing the
code in this commit was to step through the code in that repo's
setup.py and observe what it was doing. Despite the length of
setup.py in that repository, the actual amount of steps required
to produce a WiX installer is actually quite low. It consists
of a basic py2exe build plus invocations of candle.exe and
light.exe to produce the MSI.

One rabbit hole that gave me fits was locating the Visual Studio
9 C Runtime merge modules. These merge modules are only present
on your system if you have a full Visual Studio 2008 installation.
Fortunately, I have a copy of Visual Studio 2008 and was able
to install all the required updates. I then uploaded these merge
modules to a personal repository on GitHub. That is where the
added code references them from. We probably don't need to
ship the merge modules. But that is for another day.

The installs from the MSIs produced with the new automation
differ from the last official MSI in the following ways:

* Our HTML manual pages have UNIX line endings instead of Windows.
* We ship modules in the mercurial.pure package. It appears the
  upstream packaging code is not including this package due to
  omission (they supply an explicit list of packages that has
  drifted out of sync with our setup.py).
* We do not ship various distutils.* modules. This is because
  virtualenvs have a custom distutils/__init__.py that automagically
  imports distutils from its original location and py2exe gets
  confused by this. We don't use distutils in core Mercurial and
  don't provide a usable python.exe, so this omission should be
  acceptable.
* The version of the enum package is different and we ship
  an enum.pyc instead of an enum/__init__.py.
* The version of the docutils package is different and we
  ship a different set of files.
* The version of Sphinx is drastically newer and we ship a
  number of files the old version did not. (I'm not sure why
  we ship Sphinx - I think it is a side-effect of the way the
  THG code was installing dependencies.)
* We ship the idna package (dependent of requests which is a
  dependency of newer versions of Sphinx).
* The version of imagesize is different and we ship an
  imagesize.pyc instead of an imagesize/__init__.pyc.
* The version of the jinja2 package is different and the sets
  of files differs.
* We ship the packaging package, which is a dependency for Sphinx.
* The version of the pygments package is different and the sets
  of files differs.
* We ship the requests package, which is a dependency for Sphinx.
* We ship the snowballstemmer package, which is a dependency for
  Sphinx.
* We ship the urllib3 package, which is a dependency for requests,
  which is a dependency for Sphinx.
* We ship a newer version of the futures package, which includes a
  handful of extra modules that match Python 3 module names.

# no-check-commit because foo_bar naming

Differential Revision: https://phab.mercurial-scm.org/D6097

#!/usr/bin/env python
#
# posplit - split messages in paragraphs on .po/.pot files
#
# license: MIT/X11/Expat
#

from __future__ import absolute_import, print_function

import polib
import re
import sys

def addentry(po, entry, cache):
    e = cache.get(entry.msgid)
    if e:
        e.occurrences.extend(entry.occurrences)

        # merge comments from entry
        for comment in entry.comment.split('\n'):
            if comment and comment not in e.comment:
                if not e.comment:
                    e.comment = comment
                else:
                    e.comment += '\n' + comment
    else:
        po.append(entry)
        cache[entry.msgid] = entry

def mkentry(orig, delta, msgid, msgstr):
    entry = polib.POEntry()
    entry.merge(orig)
    entry.msgid = msgid or orig.msgid
    entry.msgstr = msgstr or orig.msgstr
    entry.occurrences = [(p, int(l) + delta) for (p, l) in orig.occurrences]
    return entry

if __name__ == "__main__":
    po = polib.pofile(sys.argv[1])

    cache = {}
    entries = po[:]
    po[:] = []
    findd = re.compile(r' *\.\. (\w+)::') # for finding directives
    for entry in entries:
        msgids = entry.msgid.split(u'\n\n')
        if entry.msgstr:
            msgstrs = entry.msgstr.split(u'\n\n')
        else:
            msgstrs = [u''] * len(msgids)

        if len(msgids) != len(msgstrs):
            # places the whole existing translation as a fuzzy
            # translation for each paragraph, to give the
            # translator a chance to recover part of the old
            # translation - erasing extra paragraphs is
            # probably better than retranslating all from start
            if 'fuzzy' not in entry.flags:
                entry.flags.append('fuzzy')
            msgstrs = [entry.msgstr] * len(msgids)

        delta = 0
        for msgid, msgstr in zip(msgids, msgstrs):
            if msgid and msgid != '::':
                newentry = mkentry(entry, delta, msgid, msgstr)
                mdirective = findd.match(msgid)
                if mdirective:
                    if not msgid[mdirective.end():].rstrip():
                        # only directive, nothing to translate here
                        delta += 2
                        continue
                    directive = mdirective.group(1)
                    if directive in ('container', 'include'):
                        if msgid.rstrip('\n').count('\n') == 0:
                            # only rst syntax, nothing to translate
                            delta += 2
                            continue
                        else:
                            # lines following directly, unexpected
                            print('Warning: text follows line with directive'
                                  ' %s' % directive)
                    comment = 'do not translate: .. %s::' % directive
                    if not newentry.comment:
                        newentry.comment = comment
                    elif comment not in newentry.comment:
                        newentry.comment += '\n' + comment
                addentry(po, newentry, cache)
            delta += 2 + msgid.count('\n')
    po.save()