annotate mercurial/httpconnection.py @ 50650:4d84b6d52e93 stable

delta-find: move final debug processing in a `_dbg_process_data` method This will help us making sure we can call the necessary debugging code in alternative codepath.
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Thu, 08 Jun 2023 01:52:35 +0200
parents 642e31cb55f0
children 493034cc3265
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
1 # httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
2 #
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 44298
diff changeset
3 # Copyright 2005, 2006, 2007, 2008 Olivia Mackall <olivia@selenic.com>
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 # Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6 # Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
7 #
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8 # This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 # GNU General Public License version 2 or any later version.
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
10
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
11
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
12 import os
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
14 from .i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
15 from .pycompat import open
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
16 from . import (
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
17 pycompat,
27521
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
18 util,
b1adf32b0605 httpconnection: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26347
diff changeset
19 )
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
20 from .utils import (
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
21 urlutil,
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
22 )
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
23
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
25 urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
26 urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28526
diff changeset
27
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 # moved here from url.py to avoid a cycle
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
29 class httpsendfile:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30 """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
32 Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
33 It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
34 might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
35 """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
36
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
37 def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
38 self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
39 self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
40 self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
41 self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
42 self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
43 self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
44 self._pos = 0
44298
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
45 self._progress = self._makeprogress()
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
46
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
47 def _makeprogress(self):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
48 # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
49 # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
50 # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
51 # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
52 # the user and maybe the push succeeds suddenly at 50%.
44298
0e8b28fb751b httpconnection: allow `httpsendfile` subclasses to suppress the progressbar
Matt Harbison <matt_harbison@yahoo.com>
parents: 43106
diff changeset
53 return self.ui.makeprogress(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
54 _(b'sending'), unit=_(b'kb'), total=(self.length // 1024 * 2)
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
55 )
38393
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
56
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
57 def read(self, *args, **kwargs):
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
58 ret = self._data.read(*args, **kwargs)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
59 if not ret:
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
60 self._progress.complete()
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
61 return ret
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
62 self._pos += len(ret)
5f9d436cd3b7 httpconnection: use progress helper
Martin von Zweigbergk <martinvonz@google.com>
parents: 36651
diff changeset
63 self._progress.update(self._pos // 1024)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
64 return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
65
30142
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
66 def __enter__(self):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
67 return self
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
68
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
69 def __exit__(self, exc_type, exc_val, exc_tb):
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
70 self.close()
3dcaf1c4e90d largefiles: use context for file closing
Mads Kiilerich <madski@unity3d.com>
parents: 29250
diff changeset
71
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
72
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73 # moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
74 def readauthforuri(ui, uri, user):
36651
6b1eb4c610b4 httpconnection: convert url to bytes in readauthforuri
Augie Fackler <augie@google.com>
parents: 36426
diff changeset
75 uri = pycompat.bytesurl(uri)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 # Read configuration
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
77 groups = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
78 for key, val in ui.configitems(b'auth'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
79 if key in (b'cookiefile',):
31935
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
80 continue
566cb89050b7 httpconnection: allow a global auth.cookiefile config entry
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31488
diff changeset
81
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
82 if b'.' not in key:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
83 ui.warn(_(b"ignoring invalid [auth] key '%s'\n") % key)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
85 group, setting = key.rsplit(b'.', 1)
31300
0c8a042b193d httpconnection: rename config to groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 31299
diff changeset
86 gdict = groups.setdefault(group, {})
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
87 if setting in (b'username', b'cert', b'key'):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
88 val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
89 gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
90
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
91 # Find the best match
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
92 scheme, hostpath = uri.split(b'://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
93 bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
94 bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
95 bestauth = None
48913
f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
96 for group, auth in groups.items():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
97 if user and user != auth.get(b'username', user):
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
98 # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
99 # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
100 continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
101 prefix = auth.get(b'prefix')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 continue
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
104
46907
ffd3e823a7e5 urlutil: extract `url` related code from `util` into the new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46819
diff changeset
105 prefixurl = urlutil.url(prefix)
40663
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
106 if prefixurl.user and prefixurl.user != user:
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
107 # If a username was set in the prefix, it must match the username in
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
108 # the URI.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
109 continue
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
110
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
111 # The URI passed in has been stripped of credentials, so erase the user
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
112 # here to allow simpler matching.
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
113 prefixurl.user = None
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
114 prefix = bytes(prefixurl)
c53f0ead5781 http: allow 'auth.prefix' to have a username consistent with the URI
Matt Harbison <matt_harbison@yahoo.com>
parents: 38393
diff changeset
115
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
116 p = prefix.split(b'://', 1)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
117 if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
118 schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
119 else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
120 schemes = (auth.get(b'schemes') or b'https').split()
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
121 if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
122 (prefix == b'*' or hostpath.startswith(prefix))
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
123 and (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
124 len(prefix) > bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
125 or (
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
126 len(prefix) == bestlen
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
127 and not bestuser
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
128 and b'username' in auth
43075
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
129 )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
130 )
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
131 and scheme in schemes
57875cf423c9 style: run a patched black on a subset of mercurial
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
132 ):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
133 bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
134 bestauth = group, auth
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
135 bestuser = auth.get(b'username')
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
136 if user and not bestuser:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43075
diff changeset
137 auth[b'username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
138 return bestauth