annotate mercurial/httpconnection.py @ 51096:8ec2de9c6770

revlog: move the_revisioncache on the inner object The goal for this inner object is to compute that things we cache, it make more sense to have the inner object handle it directly.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Thu, 19 Oct 2023 03:33:59 +0200
parents 642e31cb55f0
children 493034cc3265
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 44298
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Olivia Mackall <olivia@selenic.com>
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # GNU General Public License version 2 or any later version.
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
10
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
11
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
12 import os
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
14 from .i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
15 from .pycompat import open
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
16 from . import (
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
17 pycompat,
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
18 util,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
19 )
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
20 from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
21 urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
22 )
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
23
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
25 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
26 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
27
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 # moved here from url.py to avoid a cycle
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
29 class httpsendfile:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30 """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
32 Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
33 It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
34 might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37 def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39 self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
40 self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41 self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
42 self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
43 self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
44 self._pos = 0
44298
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
45 self._progress = self._makeprogress()
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
46
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
47 def _makeprogress(self):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50 # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51 # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 # the user and maybe the push succeeds suddenly at 50%.
44298
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
53 return self.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
54 _(b'sending'), unit=_(b'kb'), total=(self.length // 1024 * 2)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
55 )
38393
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
56
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
57 def read(self, *args, **kwargs):
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
58 ret = self._data.read(*args, **kwargs)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
59 if not ret:
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
60 self._progress.complete()
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
61 return ret
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
62 self._pos += len(ret)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
63 self._progress.update(self._pos // 1024)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65
30142
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
66 def __enter__(self):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
67 return self
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
68
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
69 def __exit__(self, exc_type, exc_val, exc_tb):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
70 self.close()
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
71
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
72
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73 # moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
74 def readauthforuri(ui, uri, user):
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
75 uri = pycompat.bytesurl(uri)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 # Read configuration
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
77 groups = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
78 for key, val in ui.configitems(b'auth'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
79 if key in (b'cookiefile',):
31935
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
80 continue
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
81
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
82 if b'.' not in key:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
83 ui.warn(_(b"ignoring invalid [auth] key '%s'\n") % key)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
85 group, setting = key.rsplit(b'.', 1)
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
86 gdict = groups.setdefault(group, {})
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
87 if setting in (b'username', b'cert', b'key'):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
88 val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
89 gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 # Find the best match
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
92 scheme, hostpath = uri.split(b'://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
93 bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
94 bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
95 bestauth = None
48913
f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
96 for group, auth in groups.items():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
97 if user and user != auth.get(b'username', user):
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
98 # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
99 # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
100 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
101 prefix = auth.get(b'prefix')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 continue
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
104
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
105 prefixurl = urlutil.url(prefix)
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
106 if prefixurl.user and prefixurl.user != user:
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
107 # If a username was set in the prefix, it must match the username in
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
108 # the URI.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
109 continue
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
110
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
111 # The URI passed in has been stripped of credentials, so erase the user
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
112 # here to allow simpler matching.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
113 prefixurl.user = None
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
114 prefix = bytes(prefixurl)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
115
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
116 p = prefix.split(b'://', 1)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
117 if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
118 schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
120 schemes = (auth.get(b'schemes') or b'https').split()
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
121 if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
122 (prefix == b'*' or hostpath.startswith(prefix))
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
123 and (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
124 len(prefix) > bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
125 or (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
126 len(prefix) == bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
127 and not bestuser
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
128 and b'username' in auth
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
129 )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
130 )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
131 and scheme in schemes
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
132 ):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
133 bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
134 bestauth = group, auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
135 bestuser = auth.get(b'username')
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
136 if user and not bestuser:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
137 auth[b'username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
138 return bestauth