Mercurial > hg
view setup.py @ 30155:b7a966ce89ed
changelog: disable delta chains
This patch disables delta chains on changelogs. After this patch, new
entries on changelogs - including existing changelogs - will be stored
as the fulltext of that data (likely compressed). No delta computation
will be performed.
An overview of delta chains and data justifying this change follows.
Revlogs try to store entries as a delta against a previous entry (either
a parent revision in the case of generaldelta or the previous physical
revision when not using generaldelta). Most of the time this is the
correct thing to do: it frequently results in less CPU usage and smaller
storage.
Delta chains are most effective when the base revision being deltad
against is similar to the current data. This tends to occur naturally
for manifests and file data, since only small parts of each tend to
change with each revision. Changelogs, however, are a different story.
Changelog entries represent changesets/commits. And unless commits in a
repository are homogonous (same author, changing same files, similar
commit messages, etc), a delta from one entry to the next tends to be
relatively large compared to the size of the entry. This means that
delta chains tend to be short. How short? Here is the full vs delta
revision breakdown on some real world repos:
Repo % Full % Delta Max Length
hg 45.8 54.2 6
mozilla-central 42.4 57.6 8
mozilla-unified 42.5 57.5 17
pypy 46.1 53.9 6
python-zstandard 46.1 53.9 3
(I threw in python-zstandard as an example of a repo that is homogonous.
It contains a small Python project with changes all from the same
author.)
Contrast this with the manifest revlog for these repos, where 99+% of
revisions are deltas and delta chains run into the thousands.
So delta chains aren't as useful on changelogs. But even a short delta
chain may provide benefits. Let's measure that.
Delta chains may require less CPU to read revisions if the CPU time
spent reading smaller deltas is less than the CPU time used to
decompress larger individual entries. We can measure this via
`hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's
fulltext. Here are the results of that command on a repo using delta
chains in its changelog and on a repo without delta chains:
hg (forward)
! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25)
! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26)
hg (reverse)
! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19)
! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25)
mozilla-central (forward)
! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3)
! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3)
mozilla-central (reverse)
! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3)
! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3)
mozilla-unified (forward)
! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3)
! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3)
mozilla-unified (reverse)
! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3)
! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3)
pypy (forward)
! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8)
! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9)
pypy (reverse)
! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7)
! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9)
The data clearly shows that it takes less wall and CPU time to resolve
revisions when there are no delta chains in the changelogs, regardless
of the direction of traversal. Furthermore, not using a delta chain
means that fulltext resolution in reverse is as fast as iterating
forward. So not using delta chains on the changelog is a clear CPU win
for reading operations.
An example of a user-visible operation showing this speed-up is revset
evaluation. Here are results for
`hg perfrevset 'author(gps) or author(mpm)'`:
hg
! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6)
! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7)
mozilla-central
! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3)
! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3)
What about 00changelog.i size?
Repo Delta Chains No Delta Chains
hg 7,033,250 6,976,771
mozilla-central 82,978,748 81,574,623
mozilla-unified 88,112,349 86,702,162
pypy 20,740,699 20,659,741
The data shows that removing delta chains from the changelog makes the
changelog smaller.
Delta chains are also used during changegroup generation. This
operation essentially converts a series of revisions to one large
delta chain. And changegroup generation is smart: if the delta in
the revlog matches what the changegroup is emitting, it will reuse
the delta instead of recalculating it. We can measure the impact
removing changelog delta chains has on changegroup generation via
`hg perfchangegroupchangelog`:
hg
! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6)
mozilla-central
! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3)
! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3)
mozilla-unified
! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3)
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
pypy
! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3)
! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3)
The data shows eliminating delta chains makes the changelog part of
changegroup generation slower. This is expected since we now have to
compute deltas for revisions where we could recycle the delta before.
It is worth putting this regression into context of overall changegroup
times. Here is the rough total CPU time spent in changegroup generation
for various repos while using delta chains on the changelog:
Repo CPU Time (s) CPU Time w/ compression
hg 4.50 7.05
mozilla-central 111.1 222.0
pypy 28.68 75.5
Before compression, removing delta chains from the changegroup adds
~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central,
and 2.0% to pypy. When you factor in zlib compression, these percentages
are roughly divided by 2.
While the increased CPU usage for changegroup generation is unfortunate,
I think it is acceptable because the percentage is small, server
operators (those likely impacted most by this) have other mechanisms
to mitigate CPU consumption (namely reducing zlib compression level and
pre-generated clone bundles), and because there is room to optimize this
in the future. For example, we could use the nullid as the base revision,
effectively encoding the full revision for each entry in the changegroup.
When doing this, `hg perfchangegroupchangelog` nearly halves:
mozilla-unified
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3)
This looks very promising as a future optimization opportunity.
It's worth that the changes in test-acl.t to the changegroup part size.
This is because revision 6 in the changegroup had a delta chain of
length 2 before and after this patch the base revision is nullrev.
When the base revision is nullrev, cg2packer.deltaparent() hardcodes
the *previous* revision from the changegroup as the delta parent.
This caused the delta in the changegroup to switch base revisions,
the delta to change, and the size to change accordingly. While the
size increased in this case, I think sizes will remain the same
on average, as the delta base for changelog revisions doesn't matter
too much (as this patch shows). So, I don't consider this a regression.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 13 Oct 2016 12:50:27 +0200 |
parents | a8933d992a71 |
children | 9cc438bf7d9a |
line wrap: on
line source
# # This is the mercurial setup script. # # 'python setup.py install', or # 'python setup.py --help' for more options import sys, platform if getattr(sys, 'version_info', (0, 0, 0)) < (2, 6, 0, 'final'): raise SystemExit("Mercurial requires Python 2.6 or later.") if sys.version_info[0] >= 3: printf = eval('print') libdir_escape = 'unicode_escape' else: libdir_escape = 'string_escape' def printf(*args, **kwargs): f = kwargs.get('file', sys.stdout) end = kwargs.get('end', '\n') f.write(b' '.join(args) + end) # Solaris Python packaging brain damage try: import hashlib sha = hashlib.sha1() except ImportError: try: import sha sha.sha # silence unused import warning except ImportError: raise SystemExit( "Couldn't import standard hashlib (incomplete Python install).") try: import zlib zlib.compressobj # silence unused import warning except ImportError: raise SystemExit( "Couldn't import standard zlib (incomplete Python install).") # The base IronPython distribution (as of 2.7.1) doesn't support bz2 isironpython = False try: isironpython = (platform.python_implementation() .lower().find("ironpython") != -1) except AttributeError: pass if isironpython: sys.stderr.write("warning: IronPython detected (no bz2 support)\n") else: try: import bz2 bz2.BZ2Compressor # silence unused import warning except ImportError: raise SystemExit( "Couldn't import standard bz2 (incomplete Python install).") ispypy = "PyPy" in sys.version import ctypes import os, stat, subprocess, time import re import shutil import tempfile from distutils import log if 'FORCE_SETUPTOOLS' in os.environ: from setuptools import setup else: from distutils.core import setup from distutils.core import Command, Extension from distutils.dist import Distribution from distutils.command.build import build from distutils.command.build_ext import build_ext from distutils.command.build_py import build_py from distutils.command.build_scripts import build_scripts from distutils.command.install_lib import install_lib from distutils.command.install_scripts import install_scripts from distutils.spawn import spawn, find_executable from distutils import file_util from distutils.errors import ( CCompilerError, DistutilsError, DistutilsExecError, ) from distutils.sysconfig import get_python_inc, get_config_var from distutils.version import StrictVersion scripts = ['hg'] if os.name == 'nt': # We remove hg.bat if we are able to build hg.exe. scripts.append('contrib/win32/hg.bat') # simplified version of distutils.ccompiler.CCompiler.has_function # that actually removes its temporary files. def hasfunction(cc, funcname): tmpdir = tempfile.mkdtemp(prefix='hg-install-') devnull = oldstderr = None try: fname = os.path.join(tmpdir, 'funcname.c') f = open(fname, 'w') f.write('int main(void) {\n') f.write(' %s();\n' % funcname) f.write('}\n') f.close() # Redirect stderr to /dev/null to hide any error messages # from the compiler. # This will have to be changed if we ever have to check # for a function on Windows. devnull = open('/dev/null', 'w') oldstderr = os.dup(sys.stderr.fileno()) os.dup2(devnull.fileno(), sys.stderr.fileno()) objects = cc.compile([fname], output_dir=tmpdir) cc.link_executable(objects, os.path.join(tmpdir, "a.out")) return True except Exception: return False finally: if oldstderr is not None: os.dup2(oldstderr, sys.stderr.fileno()) if devnull is not None: devnull.close() shutil.rmtree(tmpdir) # py2exe needs to be installed to work try: import py2exe py2exe.Distribution # silence unused import warning py2exeloaded = True # import py2exe's patched Distribution class from distutils.core import Distribution except ImportError: py2exeloaded = False def runcmd(cmd, env): if (sys.platform == 'plan9' and (sys.version_info[0] == 2 and sys.version_info[1] < 7)): # subprocess kludge to work around issues in half-baked Python # ports, notably bichued/python: _, out, err = os.popen3(cmd) return str(out), str(err) else: p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) out, err = p.communicate() return out, err def runhg(cmd, env): out, err = runcmd(cmd, env) # If root is executing setup.py, but the repository is owned by # another user (as in "sudo python setup.py install") we will get # trust warnings since the .hg/hgrc file is untrusted. That is # fine, we don't want to load it anyway. Python may warn about # a missing __init__.py in mercurial/locale, we also ignore that. err = [e for e in err.splitlines() if not e.startswith(b'not trusting file') \ and not e.startswith(b'warning: Not importing') \ and not e.startswith(b'obsolete feature not enabled')] if err: printf("stderr from '%s':" % (' '.join(cmd)), file=sys.stderr) printf(b'\n'.join([b' ' + e for e in err]), file=sys.stderr) return '' return out version = '' # Execute hg out of this directory with a custom environment which takes care # to not use any hgrc files and do no localization. env = {'HGMODULEPOLICY': 'py', 'HGRCPATH': '', 'LANGUAGE': 'C'} if 'LD_LIBRARY_PATH' in os.environ: env['LD_LIBRARY_PATH'] = os.environ['LD_LIBRARY_PATH'] if 'SystemRoot' in os.environ: # Copy SystemRoot into the custom environment for Python 2.6 # under Windows. Otherwise, the subprocess will fail with # error 0xc0150004. See: http://bugs.python.org/issue3440 env['SystemRoot'] = os.environ['SystemRoot'] if os.path.isdir('.hg'): cmd = [sys.executable, 'hg', 'log', '-r', '.', '--template', '{tags}\n'] numerictags = [t for t in runhg(cmd, env).split() if t[0].isdigit()] hgid = runhg([sys.executable, 'hg', 'id', '-i'], env).strip() if numerictags: # tag(s) found version = numerictags[-1] if hgid.endswith('+'): # propagate the dirty status to the tag version += '+' else: # no tag found ltagcmd = [sys.executable, 'hg', 'parents', '--template', '{latesttag}'] ltag = runhg(ltagcmd, env) changessincecmd = [sys.executable, 'hg', 'log', '-T', 'x\n', '-r', "only(.,'%s')" % ltag] changessince = len(runhg(changessincecmd, env).splitlines()) version = '%s+%s-%s' % (ltag, changessince, hgid) if version.endswith('+'): version += time.strftime('%Y%m%d') elif os.path.exists('.hg_archival.txt'): kw = dict([[t.strip() for t in l.split(':', 1)] for l in open('.hg_archival.txt')]) if 'tag' in kw: version = kw['tag'] elif 'latesttag' in kw: if 'changessincelatesttag' in kw: version = '%(latesttag)s+%(changessincelatesttag)s-%(node).12s' % kw else: version = '%(latesttag)s+%(latesttagdistance)s-%(node).12s' % kw else: version = kw.get('node', '')[:12] if version: with open("mercurial/__version__.py", "w") as f: f.write('# this file is autogenerated by setup.py\n') f.write('version = "%s"\n' % version) try: oldpolicy = os.environ.get('HGMODULEPOLICY', None) os.environ['HGMODULEPOLICY'] = 'py' from mercurial import __version__ version = __version__.version except ImportError: version = 'unknown' finally: if oldpolicy is None: del os.environ['HGMODULEPOLICY'] else: os.environ['HGMODULEPOLICY'] = oldpolicy class hgbuild(build): # Insert hgbuildmo first so that files in mercurial/locale/ are found # when build_py is run next. sub_commands = [('build_mo', None)] + build.sub_commands class hgbuildmo(build): description = "build translations (.mo files)" def run(self): if not find_executable('msgfmt'): self.warn("could not find msgfmt executable, no translations " "will be built") return podir = 'i18n' if not os.path.isdir(podir): self.warn("could not find %s/ directory" % podir) return join = os.path.join for po in os.listdir(podir): if not po.endswith('.po'): continue pofile = join(podir, po) modir = join('locale', po[:-3], 'LC_MESSAGES') mofile = join(modir, 'hg.mo') mobuildfile = join('mercurial', mofile) cmd = ['msgfmt', '-v', '-o', mobuildfile, pofile] if sys.platform != 'sunos5': # msgfmt on Solaris does not know about -c cmd.append('-c') self.mkpath(join('mercurial', modir)) self.make_file([pofile], mobuildfile, spawn, (cmd,)) class hgdist(Distribution): pure = False cffi = ispypy global_options = Distribution.global_options + \ [('pure', None, "use pure (slow) Python " "code instead of C extensions"), ] def has_ext_modules(self): # self.ext_modules is emptied in hgbuildpy.finalize_options which is # too late for some cases return not self.pure and Distribution.has_ext_modules(self) class hgbuildext(build_ext): def build_extension(self, ext): try: build_ext.build_extension(self, ext) except CCompilerError: if not getattr(ext, 'optional', False): raise log.warn("Failed to build optional extension '%s' (skipping)", ext.name) class hgbuildscripts(build_scripts): def run(self): if os.name != 'nt' or self.distribution.pure: return build_scripts.run(self) exebuilt = False try: self.run_command('build_hgexe') exebuilt = True except (DistutilsError, CCompilerError): log.warn('failed to build optional hg.exe') if exebuilt: # Copying hg.exe to the scripts build directory ensures it is # installed by the install_scripts command. hgexecommand = self.get_finalized_command('build_hgexe') dest = os.path.join(self.build_dir, 'hg.exe') self.mkpath(self.build_dir) self.copy_file(hgexecommand.hgexepath, dest) # Remove hg.bat because it is redundant with hg.exe. self.scripts.remove('contrib/win32/hg.bat') return build_scripts.run(self) class hgbuildpy(build_py): def finalize_options(self): build_py.finalize_options(self) if self.distribution.pure: self.distribution.ext_modules = [] elif self.distribution.cffi: import setup_mpatch_cffi import setup_bdiff_cffi exts = [setup_mpatch_cffi.ffi.distutils_extension(), setup_bdiff_cffi.ffi.distutils_extension()] # cffi modules go here if sys.platform == 'darwin': import setup_osutil_cffi exts.append(setup_osutil_cffi.ffi.distutils_extension()) self.distribution.ext_modules = exts else: h = os.path.join(get_python_inc(), 'Python.h') if not os.path.exists(h): raise SystemExit('Python headers are required to build ' 'Mercurial but weren\'t found in %s' % h) def run(self): if self.distribution.pure: modulepolicy = 'py' else: modulepolicy = 'c' with open("mercurial/__modulepolicy__.py", "w") as f: f.write('# this file is autogenerated by setup.py\n') f.write('modulepolicy = "%s"\n' % modulepolicy) build_py.run(self) class buildhgextindex(Command): description = 'generate prebuilt index of hgext (for frozen package)' user_options = [] _indexfilename = 'hgext/__index__.py' def initialize_options(self): pass def finalize_options(self): pass def run(self): if os.path.exists(self._indexfilename): with open(self._indexfilename, 'w') as f: f.write('# empty\n') # here no extension enabled, disabled() lists up everything code = ('import pprint; from mercurial import extensions; ' 'pprint.pprint(extensions.disabled())') out, err = runcmd([sys.executable, '-c', code], env) if err: raise DistutilsExecError(err) with open(self._indexfilename, 'w') as f: f.write('# this file is autogenerated by setup.py\n') f.write('docs = ') f.write(out) class buildhgexe(build_ext): description = 'compile hg.exe from mercurial/exewrapper.c' def build_extensions(self): if os.name != 'nt': return if isinstance(self.compiler, HackedMingw32CCompiler): self.compiler.compiler_so = self.compiler.compiler # no -mdll self.compiler.dll_libraries = [] # no -lmsrvc90 # Different Python installs can have different Python library # names. e.g. the official CPython distribution uses pythonXY.dll # and MinGW uses libpythonX.Y.dll. _kernel32 = ctypes.windll.kernel32 _kernel32.GetModuleFileNameA.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_ulong] _kernel32.GetModuleFileNameA.restype = ctypes.c_ulong size = 1000 buf = ctypes.create_string_buffer(size + 1) filelen = _kernel32.GetModuleFileNameA(sys.dllhandle, ctypes.byref(buf), size) if filelen > 0 and filelen != size: dllbasename = os.path.basename(buf.value) if not dllbasename.lower().endswith('.dll'): raise SystemExit('Python DLL does not end with .dll: %s' % dllbasename) pythonlib = dllbasename[:-4] else: log.warn('could not determine Python DLL filename; ' 'assuming pythonXY') hv = sys.hexversion pythonlib = 'python%d%d' % (hv >> 24, (hv >> 16) & 0xff) log.info('using %s as Python library name' % pythonlib) with open('mercurial/hgpythonlib.h', 'wb') as f: f.write('/* this file is autogenerated by setup.py */\n') f.write('#define HGPYTHONLIB "%s"\n' % pythonlib) objects = self.compiler.compile(['mercurial/exewrapper.c'], output_dir=self.build_temp) dir = os.path.dirname(self.get_ext_fullpath('dummy')) target = os.path.join(dir, 'hg') self.compiler.link_executable(objects, target, libraries=[], output_dir=self.build_temp) @property def hgexepath(self): dir = os.path.dirname(self.get_ext_fullpath('dummy')) return os.path.join(self.build_temp, dir, 'hg.exe') class hginstalllib(install_lib): ''' This is a specialization of install_lib that replaces the copy_file used there so that it supports setting the mode of files after copying them, instead of just preserving the mode that the files originally had. If your system has a umask of something like 027, preserving the permissions when copying will lead to a broken install. Note that just passing keep_permissions=False to copy_file would be insufficient, as it might still be applying a umask. ''' def run(self): realcopyfile = file_util.copy_file def copyfileandsetmode(*args, **kwargs): src, dst = args[0], args[1] dst, copied = realcopyfile(*args, **kwargs) if copied: st = os.stat(src) # Persist executable bit (apply it to group and other if user # has it) if st[stat.ST_MODE] & stat.S_IXUSR: setmode = int('0755', 8) else: setmode = int('0644', 8) m = stat.S_IMODE(st[stat.ST_MODE]) m = (m & ~int('0777', 8)) | setmode os.chmod(dst, m) file_util.copy_file = copyfileandsetmode try: install_lib.run(self) finally: file_util.copy_file = realcopyfile class hginstallscripts(install_scripts): ''' This is a specialization of install_scripts that replaces the @LIBDIR@ with the configured directory for modules. If possible, the path is made relative to the directory for scripts. ''' def initialize_options(self): install_scripts.initialize_options(self) self.install_lib = None def finalize_options(self): install_scripts.finalize_options(self) self.set_undefined_options('install', ('install_lib', 'install_lib')) def run(self): install_scripts.run(self) # It only makes sense to replace @LIBDIR@ with the install path if # the install path is known. For wheels, the logic below calculates # the libdir to be "../..". This is because the internal layout of a # wheel archive looks like: # # mercurial-3.6.1.data/scripts/hg # mercurial/__init__.py # # When installing wheels, the subdirectories of the "<pkg>.data" # directory are translated to system local paths and files therein # are copied in place. The mercurial/* files are installed into the # site-packages directory. However, the site-packages directory # isn't known until wheel install time. This means we have no clue # at wheel generation time what the installed site-packages directory # will be. And, wheels don't appear to provide the ability to register # custom code to run during wheel installation. This all means that # we can't reliably set the libdir in wheels: the default behavior # of looking in sys.path must do. if (os.path.splitdrive(self.install_dir)[0] != os.path.splitdrive(self.install_lib)[0]): # can't make relative paths from one drive to another, so use an # absolute path instead libdir = self.install_lib else: common = os.path.commonprefix((self.install_dir, self.install_lib)) rest = self.install_dir[len(common):] uplevel = len([n for n in os.path.split(rest) if n]) libdir = uplevel * ('..' + os.sep) + self.install_lib[len(common):] for outfile in self.outfiles: with open(outfile, 'rb') as fp: data = fp.read() # skip binary files if b'\0' in data: continue # During local installs, the shebang will be rewritten to the final # install path. During wheel packaging, the shebang has a special # value. if data.startswith(b'#!python'): log.info('not rewriting @LIBDIR@ in %s because install path ' 'not known' % outfile) continue data = data.replace(b'@LIBDIR@', libdir.encode(libdir_escape)) with open(outfile, 'wb') as fp: fp.write(data) cmdclass = {'build': hgbuild, 'build_mo': hgbuildmo, 'build_ext': hgbuildext, 'build_py': hgbuildpy, 'build_scripts': hgbuildscripts, 'build_hgextindex': buildhgextindex, 'install_lib': hginstalllib, 'install_scripts': hginstallscripts, 'build_hgexe': buildhgexe, } packages = ['mercurial', 'mercurial.hgweb', 'mercurial.httpclient', 'mercurial.pure', 'hgext', 'hgext.convert', 'hgext.fsmonitor', 'hgext.fsmonitor.pywatchman', 'hgext.highlight', 'hgext.largefiles', 'hgext.zeroconf', 'hgext3rd'] common_depends = ['mercurial/bitmanipulation.h', 'mercurial/compat.h', 'mercurial/util.h'] osutil_ldflags = [] if sys.platform == 'darwin': osutil_ldflags += ['-framework', 'ApplicationServices'] extmodules = [ Extension('mercurial.base85', ['mercurial/base85.c'], depends=common_depends), Extension('mercurial.bdiff', ['mercurial/bdiff.c', 'mercurial/bdiff_module.c'], depends=common_depends + ['mercurial/bdiff.h']), Extension('mercurial.diffhelpers', ['mercurial/diffhelpers.c'], depends=common_depends), Extension('mercurial.mpatch', ['mercurial/mpatch.c', 'mercurial/mpatch_module.c'], depends=common_depends), Extension('mercurial.parsers', ['mercurial/dirs.c', 'mercurial/manifest.c', 'mercurial/parsers.c', 'mercurial/pathencode.c'], depends=common_depends), Extension('mercurial.osutil', ['mercurial/osutil.c'], extra_link_args=osutil_ldflags, depends=common_depends), Extension('hgext.fsmonitor.pywatchman.bser', ['hgext/fsmonitor/pywatchman/bser.c']), ] try: from distutils import cygwinccompiler # the -mno-cygwin option has been deprecated for years compiler = cygwinccompiler.Mingw32CCompiler class HackedMingw32CCompiler(cygwinccompiler.Mingw32CCompiler): def __init__(self, *args, **kwargs): compiler.__init__(self, *args, **kwargs) for i in 'compiler compiler_so linker_exe linker_so'.split(): try: getattr(self, i).remove('-mno-cygwin') except ValueError: pass cygwinccompiler.Mingw32CCompiler = HackedMingw32CCompiler except ImportError: # the cygwinccompiler package is not available on some Python # distributions like the ones from the optware project for Synology # DiskStation boxes class HackedMingw32CCompiler(object): pass packagedata = {'mercurial': ['locale/*/LC_MESSAGES/hg.mo', 'help/*.txt', 'help/internals/*.txt', 'default.d/*.rc', 'dummycert.pem']} def ordinarypath(p): return p and p[0] != '.' and p[-1] != '~' for root in ('templates',): for curdir, dirs, files in os.walk(os.path.join('mercurial', root)): curdir = curdir.split(os.sep, 1)[1] dirs[:] = filter(ordinarypath, dirs) for f in filter(ordinarypath, files): f = os.path.join(curdir, f) packagedata['mercurial'].append(f) datafiles = [] setupversion = version extra = {} if py2exeloaded: extra['console'] = [ {'script':'hg', 'copyright':'Copyright (C) 2005-2016 Matt Mackall and others', 'product_version':version}] # sub command of 'build' because 'py2exe' does not handle sub_commands build.sub_commands.insert(0, ('build_hgextindex', None)) # put dlls in sub directory so that they won't pollute PATH extra['zipfile'] = 'lib/library.zip' if os.name == 'nt': # Windows binary file versions for exe/dll files must have the # form W.X.Y.Z, where W,X,Y,Z are numbers in the range 0..65535 setupversion = version.split('+', 1)[0] if sys.platform == 'darwin' and os.path.exists('/usr/bin/xcodebuild'): version = runcmd(['/usr/bin/xcodebuild', '-version'], {})[0].splitlines() if version: version = version[0] if sys.version_info[0] == 3: version = version.decode('utf-8') xcode4 = (version.startswith('Xcode') and StrictVersion(version.split()[1]) >= StrictVersion('4.0')) xcode51 = re.match(r'^Xcode\s+5\.1', version) is not None else: # xcodebuild returns empty on OS X Lion with XCode 4.3 not # installed, but instead with only command-line tools. Assume # that only happens on >= Lion, thus no PPC support. xcode4 = True xcode51 = False # XCode 4.0 dropped support for ppc architecture, which is hardcoded in # distutils.sysconfig if xcode4: os.environ['ARCHFLAGS'] = '' # XCode 5.1 changes clang such that it now fails to compile if the # -mno-fused-madd flag is passed, but the version of Python shipped with # OS X 10.9 Mavericks includes this flag. This causes problems in all # C extension modules, and a bug has been filed upstream at # http://bugs.python.org/issue21244. We also need to patch this here # so Mercurial can continue to compile in the meantime. if xcode51: cflags = get_config_var('CFLAGS') if cflags and re.search(r'-mno-fused-madd\b', cflags) is not None: os.environ['CFLAGS'] = ( os.environ.get('CFLAGS', '') + ' -Qunused-arguments') setup(name='mercurial', version=setupversion, author='Matt Mackall and many others', author_email='mercurial@selenic.com', url='https://mercurial-scm.org/', download_url='https://mercurial-scm.org/release/', description=('Fast scalable distributed SCM (revision control, version ' 'control) system'), long_description=('Mercurial is a distributed SCM tool written in Python.' ' It is used by a number of large projects that require' ' fast, reliable distributed revision control, such as ' 'Mozilla.'), license='GNU GPLv2 or any later version', classifiers=[ 'Development Status :: 6 - Mature', 'Environment :: Console', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: GNU General Public License (GPL)', 'Natural Language :: Danish', 'Natural Language :: English', 'Natural Language :: German', 'Natural Language :: Italian', 'Natural Language :: Japanese', 'Natural Language :: Portuguese (Brazilian)', 'Operating System :: Microsoft :: Windows', 'Operating System :: OS Independent', 'Operating System :: POSIX', 'Programming Language :: C', 'Programming Language :: Python', 'Topic :: Software Development :: Version Control', ], scripts=scripts, packages=packages, ext_modules=extmodules, data_files=datafiles, package_data=packagedata, cmdclass=cmdclass, distclass=hgdist, options={'py2exe': {'packages': ['hgext', 'email']}, 'bdist_mpkg': {'zipdist': False, 'license': 'COPYING', 'readme': 'contrib/macosx/Readme.html', 'welcome': 'contrib/macosx/Welcome.html', }, }, **extra)