hgweb: support Content Security Policy
authorGregory Szorc <gregory.szorc@gmail.com>
Tue, 10 Jan 2017 23:37:08 -0800
changeset 30766 d7bf7d2bd5ab
parent 30765 eb7de21b15be
child 30767 bbb5cc55ab8b
hgweb: support Content Security Policy Content-Security-Policy (CSP) is a web security feature that allows servers to declare what loaded content is allowed to do. For example, a policy can prevent loading of images, JavaScript, CSS, etc unless the source of that content is whitelisted (by hostname, URI scheme, hashes of content, etc). It's a nifty security feature that provides extra mitigation against some attacks, notably XSS. Mitigation against these attacks is important for Mercurial because hgweb renders repository data, which is commonly untrusted. While we make attempts to escape things, etc, there's the possibility that malicious data could be injected into the site content. If this happens today, the full power of the web browser is available to that malicious content. A restrictive CSP policy (defined by the server operator and sent in an HTTP header which is outside the control of malicious content), could restrict browser capabilities and mitigate security problems posed by malicious data. CSP works by emitting an HTTP header declaring the policy that browsers should apply. Ideally, this header would be emitted by a layer above Mercurial (likely the HTTP server doing the WSGI "proxying"). This works for some CSP policies, but not all. For example, policies to allow inline JavaScript may require setting a "nonce" attribute on <script>. This attribute value must be unique and non-guessable. And, the value must be present in the HTTP header and the HTML body. This means that coordinating the value between Mercurial and another HTTP server could be difficult: it is much easier to generate and emit the nonce in a central location. This commit introduces support for emitting a Content-Security-Policy header from hgweb. A config option defines the header value. If present, the header is emitted. A special "%nonce%" syntax in the value triggers generation of a nonce and inclusion in <script> elements in templates. The inclusion of a nonce does not occur unless "%nonce%" is present. This makes this commit completely backwards compatible and the feature opt-in. The nonce is a type 4 UUID, which is the flavor that is randomly generated. It has 122 random bits, which should be plenty to satisfy the guarantees of a nonce.
mercurial/help/config.txt
mercurial/hgweb/common.py
mercurial/hgweb/hgweb_mod.py
mercurial/hgweb/hgwebdir_mod.py
mercurial/templates/gitweb/graph.tmpl
mercurial/templates/gitweb/shortlog.tmpl
mercurial/templates/monoblue/graph.tmpl
mercurial/templates/monoblue/shortlog.tmpl
mercurial/templates/paper/graph.tmpl
mercurial/templates/paper/shortlog.tmpl
mercurial/templates/spartan/graph.tmpl
tests/test-hgweb-csp.t
--- a/mercurial/help/config.txt	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/help/config.txt	Tue Jan 10 23:37:08 2017 -0800
@@ -2111,6 +2111,20 @@
     Name or email address of the person in charge of the repository.
     (default: ui.username or ``$EMAIL`` or "unknown" if unset or empty)
 
+``csp``
+    Send a ``Content-Security-Policy`` HTTP header with this value.
+
+    The value may contain a special string ``%nonce%``, which will be replaced
+    by a randomly-generated one-time use value. If the value contains
+    ``%nonce%``, ``web.cache`` will be disabled, as caching undermines the
+    one-time property of the nonce. This nonce will also be inserted into
+    ``<script>`` elements containing inline JavaScript.
+
+    Note: lots of HTML content sent by the server is derived from repository
+    data. Please consider the potential for malicious repository data to
+    "inject" itself into generated HTML content as part of your security
+    threat model.
+
 ``deny_push``
     Whether to deny pushing to the repository. If empty or not set,
     push is not denied. If the special value ``*``, all remote users are
--- a/mercurial/hgweb/common.py	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/hgweb/common.py	Tue Jan 10 23:37:08 2017 -0800
@@ -8,9 +8,11 @@
 
 from __future__ import absolute_import
 
+import base64
 import errno
 import mimetypes
 import os
+import uuid
 
 from .. import (
     encoding,
@@ -199,3 +201,22 @@
     if req.env.get('HTTP_IF_NONE_MATCH') == tag:
         raise ErrorResponse(HTTP_NOT_MODIFIED)
     req.headers.append(('ETag', tag))
+
+def cspvalues(ui):
+    """Obtain the Content-Security-Policy header and nonce value.
+
+    Returns a 2-tuple of the CSP header value and the nonce value.
+
+    First value is ``None`` if CSP isn't enabled. Second value is ``None``
+    if CSP isn't enabled or if the CSP header doesn't need a nonce.
+    """
+    # Don't allow untrusted CSP setting since it be disable protections
+    # from a trusted/global source.
+    csp = ui.config('web', 'csp', untrusted=False)
+    nonce = None
+
+    if csp and '%nonce%' in csp:
+        nonce = base64.urlsafe_b64encode(uuid.uuid4().bytes).rstrip('=')
+        csp = csp.replace('%nonce%', nonce)
+
+    return csp, nonce
--- a/mercurial/hgweb/hgweb_mod.py	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/hgweb/hgweb_mod.py	Tue Jan 10 23:37:08 2017 -0800
@@ -19,6 +19,7 @@
     HTTP_OK,
     HTTP_SERVER_ERROR,
     caching,
+    cspvalues,
     permhooks,
 )
 from .request import wsgirequest
@@ -115,6 +116,8 @@
         # of the request.
         self.websubtable = app.websubtable
 
+        self.csp, self.nonce = cspvalues(self.repo.ui)
+
     # Trust the settings from the .hg/hgrc files by default.
     def config(self, section, name, default=None, untrusted=True):
         return self.repo.ui.config(section, name, default,
@@ -201,6 +204,7 @@
             'sessionvars': sessionvars,
             'pathdef': makebreadcrumb(req.url),
             'style': style,
+            'nonce': self.nonce,
         }
         tmpl = templater.templater.frommapfile(mapfile,
                                                filters={'websub': websubfilter},
@@ -318,6 +322,13 @@
         encoding.encoding = rctx.config('web', 'encoding', encoding.encoding)
         rctx.repo.ui.environ = req.env
 
+        if rctx.csp:
+            # hgwebdir may have added CSP header. Since we generate our own,
+            # replace it.
+            req.headers = [h for h in req.headers
+                           if h[0] != 'Content-Security-Policy']
+            req.headers.append(('Content-Security-Policy', rctx.csp))
+
         # work with CGI variables to create coherent structure
         # use SCRIPT_NAME, PATH_INFO and QUERY_STRING as well as our REPO_NAME
 
@@ -414,7 +425,9 @@
                 req.form['cmd'] = [tmpl.cache['default']]
                 cmd = req.form['cmd'][0]
 
-            if rctx.configbool('web', 'cache', True):
+            # Don't enable caching if using a CSP nonce because then it wouldn't
+            # be a nonce.
+            if rctx.configbool('web', 'cache', True) and not rctx.nonce:
                 caching(self, req) # sets ETag header or raises NOT_MODIFIED
             if cmd not in webcommands.__all__:
                 msg = 'no such method: %s' % cmd
--- a/mercurial/hgweb/hgwebdir_mod.py	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/hgweb/hgwebdir_mod.py	Tue Jan 10 23:37:08 2017 -0800
@@ -19,6 +19,7 @@
     HTTP_NOT_FOUND,
     HTTP_OK,
     HTTP_SERVER_ERROR,
+    cspvalues,
     get_contact,
     get_mtime,
     ismember,
@@ -227,8 +228,12 @@
         try:
             self.refresh()
 
+            csp, nonce = cspvalues(self.ui)
+            if csp:
+                req.headers.append(('Content-Security-Policy', csp))
+
             virtual = req.env.get("PATH_INFO", "").strip('/')
-            tmpl = self.templater(req)
+            tmpl = self.templater(req, nonce)
             ctype = tmpl('mimetype', encoding=encoding.encoding)
             ctype = templater.stringify(ctype)
 
@@ -466,7 +471,7 @@
                     sortcolumn=sortcolumn, descending=descending,
                     **dict(sort))
 
-    def templater(self, req):
+    def templater(self, req, nonce):
 
         def motd(**map):
             if self.motd is not None:
@@ -510,6 +515,7 @@
             "staticurl": staticurl,
             "sessionvars": sessionvars,
             "style": style,
+            "nonce": nonce,
         }
         tmpl = templater.templater.frommapfile(mapfile, defaults=defaults)
         return tmpl
--- a/mercurial/templates/gitweb/graph.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/gitweb/graph.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -45,7 +45,7 @@
 <ul id="graphnodes"></ul>
 </div>
 
-<script>
+<script{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};
@@ -108,7 +108,7 @@
 | {changenav%navgraph}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,
--- a/mercurial/templates/gitweb/shortlog.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/gitweb/shortlog.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -40,7 +40,7 @@
 {changenav%navshort}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH
--- a/mercurial/templates/monoblue/graph.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/monoblue/graph.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -40,7 +40,7 @@
         <ul id="graphnodes"></ul>
     </div>
 
-    <script>
+    <script{if(nonce, ' nonce="{nonce}"')}>
     <!-- hide script content
 
     document.getElementById('noscript').style.display = 'none';
@@ -104,7 +104,7 @@
         | {changenav%navgraph}
     </div>
 
-    <script type="text/javascript">
+    <script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,
--- a/mercurial/templates/monoblue/shortlog.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/monoblue/shortlog.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -41,7 +41,7 @@
     {changenav%navshort}
     </div>
 
-    <script type="text/javascript">
+    <script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH
--- a/mercurial/templates/paper/graph.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/paper/graph.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -59,7 +59,7 @@
 <ul id="graphnodes"></ul>
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};
@@ -121,7 +121,7 @@
 | rev {rev}: {changenav%navgraph}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}graph/{rev}?revcount=%next%&style={style}',
             {revcount}+60,
--- a/mercurial/templates/paper/shortlog.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/paper/shortlog.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -72,7 +72,7 @@
 | rev {rev}: {changenav%navshort}
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
     ajaxScrollInit(
             '{url|urlescape}shortlog/%next%{sessionvars%urlparameter}',
             '{nextentry%"{node}"}', <!-- NEXTHASH
--- a/mercurial/templates/spartan/graph.tmpl	Tue Jan 10 20:47:48 2017 -0800
+++ b/mercurial/templates/spartan/graph.tmpl	Tue Jan 10 23:37:08 2017 -0800
@@ -36,7 +36,7 @@
 <ul id="graphnodes"></ul>
 </div>
 
-<script type="text/javascript">
+<script type="text/javascript"{if(nonce, ' nonce="{nonce}"')}>
 <!-- hide script content
 
 var data = {jsdata|json};
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tests/test-hgweb-csp.t	Tue Jan 10 23:37:08 2017 -0800
@@ -0,0 +1,129 @@
+#require serve
+
+  $ cat > web.conf << EOF
+  > [paths]
+  > / = $TESTTMP/*
+  > EOF
+
+  $ hg init repo1
+  $ cd repo1
+  $ touch foo
+  $ hg -q commit -A -m initial
+  $ cd ..
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid >> $DAEMON_PIDS
+
+repo index should not send Content-Security-Policy header by default
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+
+static page should not send CSP by default
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+
+repo page should not send CSP by default, should send ETag
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  etag: W/"*" (glob)
+
+  $ killdaemons.py
+
+Configure CSP without nonce
+
+  $ cat >> web.conf << EOF
+  > [web]
+  > csp = script-src https://example.com/ 'unsafe-inline'
+  > EOF
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid > $DAEMON_PIDS
+
+repo index should send Content-Security-Policy header when enabled
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+
+static page should send CSP when enabled
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+
+repo page should send CSP by default, include etag w/o nonce
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  content-security-policy: script-src https://example.com/ 'unsafe-inline'
+  etag: W/"*" (glob)
+
+nonce should not be added to html if CSP doesn't use it
+
+  $ get-with-headers.py localhost:$HGPORT repo1/graph/tip | egrep 'content-security-policy|<script'
+  <script type="text/javascript" src="/repo1/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/repo1/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript">
+  <script type="text/javascript">
+
+Configure CSP with nonce
+
+  $ killdaemons.py
+  $ cat >> web.conf << EOF
+  > csp = image-src 'self'; script-src https://example.com/ 'nonce-%nonce%'
+  > EOF
+
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --web-conf web.conf
+  $ cat hg.pid > $DAEMON_PIDS
+
+nonce should be substituted in CSP header
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT '' content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce should be included in CSP for static pages
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+repo page should have nonce, no ETag
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT repo1 content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce should be added to html when used
+
+  $ get-with-headers.py localhost:$HGPORT repo1/graph/tip content-security-policy | egrep 'content-security-policy|<script'
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+  <script type="text/javascript" src="/repo1/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/repo1/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript" nonce="*"> (glob)
+  <script type="text/javascript" nonce="*"> (glob)
+
+hgweb_mod w/o hgwebdir works as expected
+
+  $ killdaemons.py
+
+  $ hg -R repo1 serve -p $HGPORT -d --pid-file=hg.pid --config "web.csp=image-src 'self'; script-src https://example.com/ 'nonce-%nonce%'"
+  $ cat hg.pid > $DAEMON_PIDS
+
+static page sends CSP
+
+  $ get-with-headers.py --headeronly localhost:$HGPORT static/mercurial.js content-security-policy etag
+  200 Script output follows
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+
+nonce included in <script> and headers
+
+  $ get-with-headers.py localhost:$HGPORT graph/tip content-security-policy  | egrep 'content-security-policy|<script'
+  content-security-policy: image-src 'self'; script-src https://example.com/ 'nonce-*' (glob)
+  <script type="text/javascript" src="/static/mercurial.js"></script>
+  <!--[if IE]><script type="text/javascript" src="/static/excanvas.js"></script><![endif]-->
+  <script type="text/javascript" nonce="*"> (glob)
+  <script type="text/javascript" nonce="*"> (glob)