mercurial/httpconnection.py
author Boris Feld <boris.feld@octobus.net>
Thu, 22 Nov 2018 18:44:07 +0100
changeset 40751 41b6245c3fc4
parent 40663 c53f0ead5781
child 41759 aaad36b88298
permissions -rw-r--r--
vfs: treat 'undo.' file the same as 'journal.' file They are the same kind of file, they are protected by the store lock, but directly lives inside the '.hg' directory. No warnings were ever raised about them because `vfs.rename` is not audited. Something we are trying to change.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     1
# httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     2
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     3
# Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     4
# Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     5
# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     6
# Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     7
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     8
# This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     9
# GNU General Public License version 2 or any later version.
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    10
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    11
from __future__ import absolute_import
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    12
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    13
import os
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    14
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    15
from .i18n import _
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    16
from . import (
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
    17
    pycompat,
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    18
    util,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    19
)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    20
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    21
urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    22
urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    23
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    24
# moved here from url.py to avoid a cycle
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    25
class httpsendfile(object):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    26
    """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    27
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    28
    Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    29
    It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    30
    might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    31
    """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    32
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    33
    def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    34
        self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    35
        self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    36
        self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    37
        self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    38
        self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    39
        self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    40
        self._pos = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    41
        # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    42
        # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    43
        # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    44
        # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    45
        # the user and maybe the push succeeds suddenly at 50%.
38393
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    46
        self._progress = ui.makeprogress(_('sending'), unit=_('kb'),
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    47
                                         total=(self.length // 1024 * 2))
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    48
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    49
    def read(self, *args, **kwargs):
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    50
        ret = self._data.read(*args, **kwargs)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    51
        if not ret:
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    52
            self._progress.complete()
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    53
            return ret
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    54
        self._pos += len(ret)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    55
        self._progress.update(self._pos // 1024)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    56
        return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    57
30142
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    58
    def __enter__(self):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    59
        return self
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    60
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    61
    def __exit__(self, exc_type, exc_val, exc_tb):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    62
        self.close()
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    63
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    64
# moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
    65
def readauthforuri(ui, uri, user):
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
    66
    uri = pycompat.bytesurl(uri)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    67
    # Read configuration
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
    68
    groups = {}
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    69
    for key, val in ui.configitems('auth'):
31935
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
    70
        if key in ('cookiefile',):
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
    71
            continue
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
    72
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    73
        if '.' not in key:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    74
            ui.warn(_("ignoring invalid [auth] key '%s'\n") % key)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    75
            continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    76
        group, setting = key.rsplit('.', 1)
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
    77
        gdict = groups.setdefault(group, {})
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    78
        if setting in ('username', 'cert', 'key'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    79
            val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    80
        gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    81
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    82
    # Find the best match
25206
18a032704f0a httpconnection: drop Python 2.4 specify hack
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 19809
diff changeset
    83
    scheme, hostpath = uri.split('://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    84
    bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    85
    bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    86
    bestauth = None
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
    87
    for group, auth in groups.iteritems():
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    88
        if user and user != auth.get('username', user):
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    89
            # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    90
            # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    91
            continue
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    92
        prefix = auth.get('prefix')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    93
        if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    94
            continue
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    95
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    96
        prefixurl = util.url(prefix)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    97
        if prefixurl.user and prefixurl.user != user:
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    98
            # If a username was set in the prefix, it must match the username in
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    99
            # the URI.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   100
            continue
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   101
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   102
        # The URI passed in has been stripped of credentials, so erase the user
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   103
        # here to allow simpler matching.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   104
        prefixurl.user = None
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   105
        prefix = bytes(prefixurl)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   106
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   107
        p = prefix.split('://', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   108
        if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   109
            schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   110
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   111
            schemes = (auth.get('schemes') or 'https').split()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   112
        if (prefix == '*' or hostpath.startswith(prefix)) and \
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   113
            (len(prefix) > bestlen or (len(prefix) == bestlen and \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   114
                not bestuser and 'username' in auth)) \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   115
             and scheme in schemes:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   116
            bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   117
            bestauth = group, auth
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   118
            bestuser = auth.get('username')
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   119
            if user and not bestuser:
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   120
                auth['username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   121
    return bestauth