contrib/fixpax.py

phases: really fix native phase computation

For some reason (probably rebase issue, leprechaun or badly resolved .rej)
1635579f9baf contains only half of the emailed patches and do not fix the bug.
This patch adds the other half and enable the sweet native computation for real.

As expected this provide massive speedup along the board.

revset #0: not public()
   plain         first
0) 0.011960      0.010523
1) 0.000465   3% 0.000492   4%

revset #1: (tip~1000::) - public()
   plain         first
0) 0.025700      0.025169
1) 0.002864  11% 0.001899   7%

revset #2: not public() and branch("default")
   plain         first
0) 0.022842      0.020863
1) 0.011418  49% 0.010948   52%

However, it has a less impact (even bad) on first result time in simple
situation.  This comes from the overhead of building the set and filtering it.
This is especially true on my Mercurial repository (used here) where about 1/3
of the changesets are non public and hidden. This could be mitigated by a
caching of the set and a better usage of smartset in '_notpublic'. (But this
won't happen in this patch because the win is massive everywhere else).

revset #0: not public()
   last
0) 0.000081
1) 0.000493 x6.1 <-- bad impact

revset #1: (tip~1000::) - public()
   last
0) 0.013966
1) 0.002737  19%

revset #2: not public() and branch("default")
   last
0) 0.011021
1) 0.011038

The effect mostly disappear when the number of non-public changesets is small
and/or the repo get bigger. Result for Mozilla central:

Mozilla

revset #0: not public()
   plain         first         last
0) 0.092787      0.084094      0.000080
1) 0.000054   0% 0.000083   0% 0.000083

revset #1: (tip~1000::) - public()
   plain         first         last
0) 0.215607      0.183996      0.124962
1) 0.031620  14% 0.006616   3% 0.031168  24%

revset #2: not public() and branch("default")
   plain         first         last
0) 0.092626      0.082687      0.000162
1) 0.000139   0% 0.000165   0% 0.000167

# fixpax - fix ownership in bdist_mpkg output
#
# Copyright 2015 Matt Mackall <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# MIT license (http://opensource.org/licenses/MIT)

"""Set file ownership to 0 in an Archive.pax.gz.
Suitable for fixing files bdist_mpkg output:
*.mpkg/Contents/Packages/*.pkg/Contents/Archive.pax.gz
"""

import sys, os, gzip

def fixpax(iname, oname):
    i = gzip.GzipFile(iname)
    o = gzip.GzipFile(oname, "w")

    while True:
        magic = i.read(6)
        dev = i.read(6)
        ino = i.read(6)
        mode = i.read(6)
        i.read(6) # uid
        i.read(6) # gid
        nlink = i.read(6)
        rdev = i.read(6)
        mtime = i.read(11)
        namesize = i.read(6)
        filesize = i.read(11)
        name = i.read(int(namesize, 8))
        data = i.read(int(filesize, 8))

        o.write(magic)
        o.write(dev)
        o.write(ino)
        o.write(mode)
        o.write("000000")
        o.write("000000")
        o.write(nlink)
        o.write(rdev)
        o.write(mtime)
        o.write(namesize)
        o.write(filesize)
        o.write(name)
        o.write(data)

        if name.startswith("TRAILER!!!"):
            o.write(i.read())
            break

    o.close()
    i.close()

if __name__ == '__main__':
    for iname in sys.argv[1:]:
        print 'fixing file ownership in %s' % iname
        oname = sys.argv[1] + '.tmp'
        fixpax(iname, oname)
        os.rename(oname, iname)