mercurial/httpconnection.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 08 Oct 2019 01:47:35 -0400
changeset 43123 69301b79fd36
parent 43106 d783f945a701
child 44298 0e8b28fb751b
permissions -rw-r--r--
perf: document `perfdirstatedirs`
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     1
# httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     2
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     3
# Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     4
# Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     5
# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     6
# Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     7
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     8
# This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     9
# GNU General Public License version 2 or any later version.
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    10
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    11
from __future__ import absolute_import
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    12
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    13
import os
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    14
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    15
from .i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
    16
from .pycompat import open
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    17
from . import (
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
    18
    pycompat,
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    19
    util,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
    20
)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    21
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    22
urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    23
urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
    24
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    25
# moved here from url.py to avoid a cycle
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    26
class httpsendfile(object):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    27
    """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    28
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    29
    Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    30
    It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    31
    might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    32
    """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    33
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    34
    def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    35
        self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    36
        self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    37
        self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    38
        self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    39
        self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    40
        self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    41
        self._pos = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    42
        # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    43
        # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    44
        # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    45
        # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    46
        # the user and maybe the push succeeds suddenly at 50%.
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    47
        self._progress = ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    48
            _(b'sending'), unit=_(b'kb'), total=(self.length // 1024 * 2)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    49
        )
38393
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    50
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    51
    def read(self, *args, **kwargs):
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    52
        ret = self._data.read(*args, **kwargs)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    53
        if not ret:
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    54
            self._progress.complete()
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    55
            return ret
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    56
        self._pos += len(ret)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
    57
        self._progress.update(self._pos // 1024)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    58
        return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    59
30142
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    60
    def __enter__(self):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    61
        return self
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    62
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    63
    def __exit__(self, exc_type, exc_val, exc_tb):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    64
        self.close()
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
    65
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    66
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    67
# moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
    68
def readauthforuri(ui, uri, user):
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
    69
    uri = pycompat.bytesurl(uri)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    70
    # Read configuration
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
    71
    groups = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    72
    for key, val in ui.configitems(b'auth'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    73
        if key in (b'cookiefile',):
31935
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
    74
            continue
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
    75
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    76
        if b'.' not in key:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    77
            ui.warn(_(b"ignoring invalid [auth] key '%s'\n") % key)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    78
            continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    79
        group, setting = key.rsplit(b'.', 1)
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
    80
        gdict = groups.setdefault(group, {})
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    81
        if setting in (b'username', b'cert', b'key'):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    82
            val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    83
        gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    84
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    85
    # Find the best match
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    86
    scheme, hostpath = uri.split(b'://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    87
    bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    88
    bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    89
    bestauth = None
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
    90
    for group, auth in pycompat.iteritems(groups):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    91
        if user and user != auth.get(b'username', user):
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    92
            # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    93
            # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    94
            continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
    95
        prefix = auth.get(b'prefix')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    96
        if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    97
            continue
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    98
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
    99
        prefixurl = util.url(prefix)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   100
        if prefixurl.user and prefixurl.user != user:
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   101
            # If a username was set in the prefix, it must match the username in
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   102
            # the URI.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   103
            continue
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   104
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   105
        # The URI passed in has been stripped of credentials, so erase the user
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   106
        # here to allow simpler matching.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   107
        prefixurl.user = None
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   108
        prefix = bytes(prefixurl)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
   109
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   110
        p = prefix.split(b'://', 1)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   111
        if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   112
            schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   113
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   114
            schemes = (auth.get(b'schemes') or b'https').split()
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   115
        if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   116
            (prefix == b'*' or hostpath.startswith(prefix))
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   117
            and (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   118
                len(prefix) > bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   119
                or (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   120
                    len(prefix) == bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   121
                    and not bestuser
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   122
                    and b'username' in auth
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   123
                )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   124
            )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   125
            and scheme in schemes
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   126
        ):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   127
            bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   128
            bestauth = group, auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   129
            bestuser = auth.get(b'username')
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
   130
            if user and not bestuser:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
   131
                auth[b'username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   132
    return bestauth