annotate mercurial/sslutil.py @ 24735:07200e3332a1

tags: extract .hgtags filenodes cache to a standalone file Resolution of .hgtags filenodes values has historically been a performance pain point for large repositories, where reading individual manifests can take over 100ms. Multiplied by hundreds or even thousands of heads and resolving .hgtags filenodes becomes a performance issue. This patch establishes a standalone cache file holding the .hgtags filenodes for each changeset. After this patch, the .hgtags filenode for any particular changeset should only have to be computed once during the lifetime of the repository. The introduced hgtagsfnodes1 cache file is modeled after the rev branch cache: the cache is effectively an array of entries consisting of a changeset fragment and the filenode for a revision. The file grows in proportion to the length of the repository (24 bytes per changeset) and is truncated when the repository is stripped. The file is not written unless tag info is requested and tags have changed since last time. This patch partially addresses issue4550. Future patches will split the "tags" cache file into per-filter files and will refactor the cache format to not capture the .hgtags fnodes, as these are now stored in the hgtagsfnodes1 cache. This patch is capable of standing alone. We should not have to wait on the tags cache filter split and format refactor for this patch to land.
author Gregory Szorc <gregory.szorc@gmail.com>
date Wed, 15 Apr 2015 17:42:38 -0400
parents 241d98d84aed
children 21b536f01eda
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # sslutil.py - SSL handling for mercurial
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 #
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 #
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 # This software may be used and distributed according to the terms of the
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # GNU General Public License version 2 or any later version.
22575
d7f7f1860f00 ssl: on OS X, use a dummy cert to trick Python/OpenSSL to use system CA certs
Mads Kiilerich <madski@unity3d.com>
parents: 22574
diff changeset
9 import os, sys
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11 from mercurial import util
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
12 from mercurial.i18n import _
24291
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
13
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
14 _canloaddefaultcerts = False
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15 try:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16 # avoid using deprecated/broken FakeSocket in python 2.6
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
17 import ssl
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
18 CERT_REQUIRED = ssl.CERT_REQUIRED
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
19 try:
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
20 ssl_context = ssl.SSLContext
24291
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
21 _canloaddefaultcerts = util.safehasattr(ssl_context,
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
22 'load_default_certs')
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
23
23849
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
24 def ssl_wrap_socket(sock, keyfile, certfile, cert_reqs=ssl.CERT_NONE,
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
25 ca_certs=None, serverhostname=None):
23850
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
26 # Allow any version of SSL starting with TLSv1 and
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
27 # up. Note that specifying TLSv1 here prohibits use of
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
28 # newer standards (like TLSv1_2), so this is the right way
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
29 # to do this. Note that in the future it'd be better to
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
30 # support using ssl.create_default_context(), which sets
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
31 # up a bunch of things in smart ways (strong ciphers,
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
32 # protocol versions, etc) and is upgraded by Python
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
33 # maintainers for us, but that breaks too many things to
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
34 # do it in a hurry.
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
35 sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
e1931f7cd977 sslutil: use saner TLS settings on Python 2.7.9
Augie Fackler <augie@google.com>
parents: 23849
diff changeset
36 sslcontext.options &= ssl.OP_NO_SSLv2 & ssl.OP_NO_SSLv3
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
37 if certfile is not None:
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
38 sslcontext.load_cert_chain(certfile, keyfile)
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
39 sslcontext.verify_mode = cert_reqs
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
40 if ca_certs is not None:
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
41 sslcontext.load_verify_locations(cafile=ca_certs)
24291
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
42 elif _canloaddefaultcerts:
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
43 sslcontext.load_default_certs()
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
44
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
45 sslsocket = sslcontext.wrap_socket(sock,
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
46 server_hostname=serverhostname)
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
47 # check if wrap_socket failed silently because socket had been
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
48 # closed
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
49 # - see http://bugs.python.org/issue13721
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
50 if not sslsocket.cipher():
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
51 raise util.Abort(_('ssl connection failed'))
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
52 return sslsocket
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
53 except AttributeError:
23849
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
54 def ssl_wrap_socket(sock, keyfile, certfile, cert_reqs=ssl.CERT_NONE,
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
55 ca_certs=None, serverhostname=None):
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
56 sslsocket = ssl.wrap_socket(sock, keyfile, certfile,
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
57 cert_reqs=cert_reqs, ca_certs=ca_certs,
23851
948a8ca27152 sslutil: drop defunct ssl version constants
Augie Fackler <augie@google.com>
parents: 23850
diff changeset
58 ssl_version=ssl.PROTOCOL_TLSv1)
23834
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
59 # check if wrap_socket failed silently because socket had been
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
60 # closed
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
61 # - see http://bugs.python.org/issue13721
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
62 if not sslsocket.cipher():
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
63 raise util.Abort(_('ssl connection failed'))
bf07c19b4c82 https: support tls sni (server name indication) for https urls (issue3090)
Alex Orange <crazycasta@gmail.com>
parents: 23069
diff changeset
64 return sslsocket
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65 except ImportError:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 CERT_REQUIRED = 2
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
67
14616
64dfbe576455 sslutil: Restore missing imports of socket and httplib to sslutil
Stephen Thorne <stephen@thorne.id.au>
parents: 14204
diff changeset
68 import socket, httplib
64dfbe576455 sslutil: Restore missing imports of socket and httplib to sslutil
Stephen Thorne <stephen@thorne.id.au>
parents: 14204
diff changeset
69
23849
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
70 def ssl_wrap_socket(sock, keyfile, certfile, cert_reqs=CERT_REQUIRED,
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
71 ca_certs=None, serverhostname=None):
15160
b2d4400398f3 sslutil: abort when ssl module is needed but not found
Mads Kiilerich <mads@kiilerich.com>
parents: 14667
diff changeset
72 if not util.safehasattr(socket, 'ssl'):
b2d4400398f3 sslutil: abort when ssl module is needed but not found
Mads Kiilerich <mads@kiilerich.com>
parents: 14667
diff changeset
73 raise util.Abort(_('Python SSL support not found'))
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
74 if ca_certs:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
75 raise util.Abort(_(
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 'certificate checking requires Python 2.6'))
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
77
19808
3b82d412e9e8 sslutil: make keyfile and certfile arguments consistent between 2.6+ and 2.5-
Augie Fackler <raf@durin42.com>
parents: 19806
diff changeset
78 ssl = socket.ssl(sock, keyfile, certfile)
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
79 return httplib.FakeSocket(sock, ssl)
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
80
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
81 def _verifycert(cert, hostname):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
82 '''Verify that cert (in socket.getpeercert() format) matches hostname.
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
83 CRLs is not handled.
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
85 Returns error message if any problems are found and None on success.
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
86 '''
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
87 if not cert:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
88 return _('no certificate received')
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
89 dnsname = hostname.lower()
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90 def matchdnsname(certname):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 return (certname == dnsname or
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
92 '.' in dnsname and certname == '*.' + dnsname.split('.', 1)[1])
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
93
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
94 san = cert.get('subjectAltName', [])
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
95 if san:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
96 certnames = [value.lower() for key, value in san if key == 'DNS']
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
97 for name in certnames:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
98 if matchdnsname(name):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
99 return None
14666
27b080aa880a sslutil: fall back to commonName when no dNSName in subjectAltName (issue2798)
Nicolas Bareil <nico@chdir.org>
parents: 14616
diff changeset
100 if certnames:
27b080aa880a sslutil: fall back to commonName when no dNSName in subjectAltName (issue2798)
Nicolas Bareil <nico@chdir.org>
parents: 14616
diff changeset
101 return _('certificate is for %s') % ', '.join(certnames)
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 # subject is only checked when subjectAltName is empty
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 for s in cert.get('subject', []):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 key, value = s[0]
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
106 if key == 'commonName':
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
107 try:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
108 # 'subject' entries are unicode
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
109 certname = value.lower().encode('ascii')
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
110 except UnicodeEncodeError:
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
111 return _('IDN in certificate not supported')
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
112 if matchdnsname(certname):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
113 return None
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
114 return _('certificate is for %s') % certname
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
115 return _('no commonName or subjectAltName found in certificate')
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
116
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
117
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
118 # CERT_REQUIRED means fetch the cert from the server all the time AND
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 # validate it against the CA store provided in web.cacerts.
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
120 #
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
121 # We COMPLETELY ignore CERT_REQUIRED on Python <= 2.5, as it's totally
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
122 # busted on those versions.
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
123
23042
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
124 def _plainapplepython():
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
125 """return true if this seems to be a pure Apple Python that
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
126 * is unfrozen and presumably has the whole mercurial module in the file
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
127 system
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
128 * presumably is an Apple Python that uses Apple OpenSSL which has patches
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
129 for using system certificate store CAs in addition to the provided
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
130 cacerts file
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
131 """
24614
241d98d84aed ssl: resolve symlink before checking for Apple python executable (issue4588)
Yuya Nishihara <yuya@tcha.org>
parents: 24291
diff changeset
132 if sys.platform != 'darwin' or util.mainfrozen() or not sys.executable:
23042
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
133 return False
24614
241d98d84aed ssl: resolve symlink before checking for Apple python executable (issue4588)
Yuya Nishihara <yuya@tcha.org>
parents: 24291
diff changeset
134 exe = os.path.realpath(sys.executable).lower()
23042
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
135 return (exe.startswith('/usr/bin/python') or
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
136 exe.startswith('/system/library/frameworks/python.framework/'))
2cd3fa4412dc ssl: only use the dummy cert hack if using an Apple Python (issue4410)
Mads Kiilerich <madski@unity3d.com>
parents: 22575
diff changeset
137
24288
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
138 def _defaultcacerts():
24291
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
139 """return path to CA certificates; None for system's store; ! to disable"""
24288
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
140 if _plainapplepython():
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
141 dummycert = os.path.join(os.path.dirname(__file__), 'dummycert.pem')
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
142 if os.path.exists(dummycert):
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
143 return dummycert
24291
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
144 if _canloaddefaultcerts:
760a86865f80 ssl: load CA certificates from system's store by default on Python 2.7.9
Yuya Nishihara <yuya@tcha.org>
parents: 24290
diff changeset
145 return None
24290
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
146 return '!'
24288
922e087ba158 ssl: extract function that returns dummycert path on Apple python
Yuya Nishihara <yuya@tcha.org>
parents: 23851
diff changeset
147
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
148 def sslkwargs(ui, host):
23849
58080815f667 sslutil: drop support for clients of sslutil specifying a TLS version
Augie Fackler <augie@google.com>
parents: 23834
diff changeset
149 kws = {}
22574
a00a7951b20c ssl: refactor sslkwargs - move things around a bit, preparing for next change
Mads Kiilerich <madski@unity3d.com>
parents: 19808
diff changeset
150 hostfingerprint = ui.config('hostfingerprints', host)
a00a7951b20c ssl: refactor sslkwargs - move things around a bit, preparing for next change
Mads Kiilerich <madski@unity3d.com>
parents: 19808
diff changeset
151 if hostfingerprint:
a00a7951b20c ssl: refactor sslkwargs - move things around a bit, preparing for next change
Mads Kiilerich <madski@unity3d.com>
parents: 19808
diff changeset
152 return kws
a00a7951b20c ssl: refactor sslkwargs - move things around a bit, preparing for next change
Mads Kiilerich <madski@unity3d.com>
parents: 19808
diff changeset
153 cacerts = ui.config('web', 'cacerts')
24290
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
154 if cacerts == '!':
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
155 pass
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
156 elif cacerts:
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
157 cacerts = util.expandpath(cacerts)
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
158 if not os.path.exists(cacerts):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
159 raise util.Abort(_('could not find web.cacerts: %s') % cacerts)
24290
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
160 else:
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
161 cacerts = _defaultcacerts()
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
162 if cacerts and cacerts != '!':
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
163 ui.debug('using %s to enable OS X system CA\n' % cacerts)
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
164 ui.setconfig('web', 'cacerts', cacerts, 'defaultcacerts')
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
165 if cacerts != '!':
19806
47ff9d1abfa9 sslutil: add a config knob to support TLS (default) or SSLv23 (bc) (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19749
diff changeset
166 kws.update({'ca_certs': cacerts,
47ff9d1abfa9 sslutil: add a config knob to support TLS (default) or SSLv23 (bc) (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19749
diff changeset
167 'cert_reqs': CERT_REQUIRED,
47ff9d1abfa9 sslutil: add a config knob to support TLS (default) or SSLv23 (bc) (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19749
diff changeset
168 })
47ff9d1abfa9 sslutil: add a config knob to support TLS (default) or SSLv23 (bc) (issue4038)
Augie Fackler <raf@durin42.com>
parents: 19749
diff changeset
169 return kws
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
170
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
171 class validator(object):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
172 def __init__(self, ui, host):
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
173 self.ui = ui
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
174 self.host = host
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
175
18887
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
176 def __call__(self, sock, strict=False):
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
177 host = self.host
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
178 cacerts = self.ui.config('web', 'cacerts')
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
179 hostfingerprint = self.ui.config('hostfingerprints', host)
15813
3ae04eb5e38a sslutil: handle setups without .getpeercert() early in the validator
Mads Kiilerich <mads@kiilerich.com>
parents: 15812
diff changeset
180 if not getattr(sock, 'getpeercert', False): # python 2.5 ?
3ae04eb5e38a sslutil: handle setups without .getpeercert() early in the validator
Mads Kiilerich <mads@kiilerich.com>
parents: 15812
diff changeset
181 if hostfingerprint:
3ae04eb5e38a sslutil: handle setups without .getpeercert() early in the validator
Mads Kiilerich <mads@kiilerich.com>
parents: 15812
diff changeset
182 raise util.Abort(_("host fingerprint for %s can't be "
3ae04eb5e38a sslutil: handle setups without .getpeercert() early in the validator
Mads Kiilerich <mads@kiilerich.com>
parents: 15812
diff changeset
183 "verified (Python too old)") % host)
18887
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
184 if strict:
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
185 raise util.Abort(_("certificate for %s can't be verified "
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
186 "(Python too old)") % host)
16391
9cf7c9d529d0 ui: optionally quiesce ssl verification warnings on python 2.5
Steven Stallion <sstallion@gmail.com>
parents: 15997
diff changeset
187 if self.ui.configbool('ui', 'reportoldssl', True):
9cf7c9d529d0 ui: optionally quiesce ssl verification warnings on python 2.5
Steven Stallion <sstallion@gmail.com>
parents: 15997
diff changeset
188 self.ui.warn(_("warning: certificate for %s can't be verified "
9cf7c9d529d0 ui: optionally quiesce ssl verification warnings on python 2.5
Steven Stallion <sstallion@gmail.com>
parents: 15997
diff changeset
189 "(Python too old)\n") % host)
15813
3ae04eb5e38a sslutil: handle setups without .getpeercert() early in the validator
Mads Kiilerich <mads@kiilerich.com>
parents: 15812
diff changeset
190 return
18879
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
191
15816
4bb59919c905 sslutil: work around validator crash getting certificate on failed sockets
Mads Kiilerich <mads@kiilerich.com>
parents: 15815
diff changeset
192 if not sock.cipher(): # work around http://bugs.python.org/issue13721
4bb59919c905 sslutil: work around validator crash getting certificate on failed sockets
Mads Kiilerich <mads@kiilerich.com>
parents: 15815
diff changeset
193 raise util.Abort(_('%s ssl connection error') % host)
18879
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
194 try:
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
195 peercert = sock.getpeercert(True)
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
196 peercert2 = sock.getpeercert()
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
197 except AttributeError:
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
198 raise util.Abort(_('%s ssl connection error') % host)
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
199
15817
8f377751b510 sslutil: abort properly if no certificate received for https connection
Mads Kiilerich <mads@kiilerich.com>
parents: 15816
diff changeset
200 if not peercert:
8f377751b510 sslutil: abort properly if no certificate received for https connection
Mads Kiilerich <mads@kiilerich.com>
parents: 15816
diff changeset
201 raise util.Abort(_('%s certificate error: '
8f377751b510 sslutil: abort properly if no certificate received for https connection
Mads Kiilerich <mads@kiilerich.com>
parents: 15816
diff changeset
202 'no certificate received') % host)
15814
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
203 peerfingerprint = util.sha1(peercert).hexdigest()
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
204 nicefingerprint = ":".join([peerfingerprint[x:x + 2]
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
205 for x in xrange(0, len(peerfingerprint), 2)])
15815
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
206 if hostfingerprint:
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
207 if peerfingerprint.lower() != \
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
208 hostfingerprint.replace(':', '').lower():
15997
a45516cb8d9f sslutil: more helpful fingerprint mismatch message
Matt Mackall <mpm@selenic.com>
parents: 15817
diff changeset
209 raise util.Abort(_('certificate for %s has unexpected '
a45516cb8d9f sslutil: more helpful fingerprint mismatch message
Matt Mackall <mpm@selenic.com>
parents: 15817
diff changeset
210 'fingerprint %s') % (host, nicefingerprint),
a45516cb8d9f sslutil: more helpful fingerprint mismatch message
Matt Mackall <mpm@selenic.com>
parents: 15817
diff changeset
211 hint=_('check hostfingerprint configuration'))
15815
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
212 self.ui.debug('%s certificate matched fingerprint %s\n' %
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
213 (host, nicefingerprint))
24290
b76d8c641746 ssl: set explicit symbol "!" to web.cacerts to disable SSL verification (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 24288
diff changeset
214 elif cacerts != '!':
18879
93b03a222c3e sslutil: try harder to avoid getpeercert problems
Matt Mackall <mpm@selenic.com>
parents: 16391
diff changeset
215 msg = _verifycert(peercert2, host)
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
216 if msg:
15814
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
217 raise util.Abort(_('%s certificate error: %s') % (host, msg),
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
218 hint=_('configure hostfingerprint %s or use '
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
219 '--insecure to connect insecurely') %
c3e958b50a22 sslutil: show fingerprint when cacerts validation fails
Mads Kiilerich <mads@kiilerich.com>
parents: 15813
diff changeset
220 nicefingerprint)
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
221 self.ui.debug('%s certificate successfully verified\n' % host)
18887
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
222 elif strict:
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
223 raise util.Abort(_('%s certificate with fingerprint %s not '
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
224 'verified') % (host, nicefingerprint),
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
225 hint=_('check hostfingerprints or web.cacerts '
2d7fac049d3a sslutil: abort if peer certificate is not verified for secure use
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 18879
diff changeset
226 'config setting'))
14204
5fa21960b2f4 sslutil: extracted ssl methods from httpsconnection in url.py
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
227 else:
15815
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
228 self.ui.warn(_('warning: %s certificate with fingerprint %s not '
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
229 'verified (check hostfingerprints or web.cacerts '
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
230 'config setting)\n') %
edc3a901a63d sslutil: reorder validator code to make it more readable
Mads Kiilerich <mads@kiilerich.com>
parents: 15814
diff changeset
231 (host, nicefingerprint))