tests: use stringutil.pprint instead of pprint.pprint in test-minirst.py Stabilizes some output on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3476

tests: port test-pathencode.py to Python 3 # skip-blame uninteresting changes, mainly b prefixes Differential Revision: https://phab.mercurial-scm.org/D3475

tests: port test-hgwebdir-paths.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3474

tests: port test-walkrepo.py to Python 3 # skip-blame just b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3473

tests: port test-minifileset.py to Python 3 # skip-blame all b prefixes, nothing interesting Differential Revision: https://phab.mercurial-scm.org/D3472

minifileset: fix on Python 3 Found by porting test-minifileset.py to Python 3. Differential Revision: https://phab.mercurial-scm.org/D3471

tests: port test-url.py to Python 3 # skip-blame it's all b prefixes and pycompat.bytestr() wrappers Differential Revision: https://phab.mercurial-scm.org/D3470

tests: fix error case in test-url.py's doctest This required some careful attention, so I wanted to split it out from the uninteresting bits that'll be in the next change. Differential Revision: https://phab.mercurial-scm.org/D3469

util: make util.url __repr__ consistent on Python 2 and 3 Differential Revision: https://phab.mercurial-scm.org/D3468

sslutil: fix some edge cases in Python 3 support Detected by fixing up test-url.py on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3467

tests: port test-hybridencode.py to unittest This was done predominantly through keyboard macros in emacs, so it's not always pretty, but it's enough to make the test easier to work with. There's a ton of room for improvement in this file, but it would be labor intensive and error-prone, so I went with the dumbest option that could work, so as to avoid transcription errors. Paranoia: $ egrep '^ def test' tests/test-hybridencode.py | wc -l 44 $ egrep '^ def test' tests/test-hybridencode.py | sort | uniq | wc -l so I'm pretty confident there aren't any shadowed test methods. This fixes the test on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3466

tests: make test-check-interfaces.py work on Python 3 # skip-blame just a bunch of bytes prefixes Differential Revision: https://phab.mercurial-scm.org/D3465

scmutil: clean up bytes/string cache decorator mess on Python 3 again The previous fix to this area worked, but was dropping bytes in __dict__ on Python 3. This was causing subtle breakage in test-check-interfaces.py, and probably other things too. Fixed now. Differential Revision: https://phab.mercurial-scm.org/D3464

directaccess: use resolvehexnodeidprefix() instead of _partialmatch() Same reasoning as previous commit: I want to make resolvehexnodeidprefix() move complex and don't want to duplicate that code in directaccess. Differential Revision: https://phab.mercurial-scm.org/D3463

revset: use resolvehexnodeidprefix() in id() predicate (BC) We now have a public method for this purpose, so we don't need to access the private revlog._partialmatch(). Also, I'll probably make some changes to resolvehexnodeidprefix() later, and I want those to be reflected by the id() predicate. Note that this breaks a test case, because we now resolve the prefix in the unfiltered repo and get an ambiguous lookup, which results in no revision being added to the revset. The test case was already documented to be broken even though it wasn't. It's important to note that {shortest(node)} already uses the unfiltered repo, so we're not going to break people who get the prefix from there. I think we may not want to ever use shortest() in the filtered repo. It seems unlikely to be enough of a win to matter much. For example, in my hg repo, it would save me only 0.2 hex digits. In another repo that only I modify, it saves a little more, but it's still only 0.29 hex digits. It seems unlikely that people will prune enough commits that only 1/16 of the commits are visible (which is what it would take a to save a single hex digit). Instead, I'm working on another approach: allow ambiguous matches to be disambiguated within a user-specified revset. Whether or not that pans out, I hope we're okay with this little change in behavior for now and we can decide what to do about it later. Differential Revision: https://phab.mercurial-scm.org/D3311

revset: make id() an empty set for ambiguous nodeid (BC) As Yuya pointed out in the review of D3311, id() (and rev()) does not raise an error when the input is an unknown identifier, so it doesn't make sense for it to do that when the input is ambiguous with a filtered node. However, it turned out that it already does raise an error when the input is ambiguous among the visible nodes. So let's start by fixing that. Differential Revision: https://phab.mercurial-scm.org/D3462

shortest: don't keep checking for longer prefix if node doesn't exist (API) If revlog.shortest() is called with an invalid nodeid, we keep checking if longer and longer prefixes are valid. We call revlog._partialmatch() for each prefix. That function will give us None if the node doesn't exist (and a RevlogError if it's ambiguous), so there's no need to keep checking. This patch instead makes revlog.shortest() raise a LookupError is the node does not exist, and updates the caller to handle it. Before this patch, revlog.shortest() would return the full hexnode for nonexistent nodeids. By the same reasoning as in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), it's not revlog.shortest() that should decide how to present nonexistent nodeids, so that's now moved to the template function. This should speed up cases where {shortest()} is applied to an invalid nodeid, but I couldn't think of a reasonable case where that would happen. Differential Revision: https://phab.mercurial-scm.org/D3461

shortest: extract function for checking if a prefix is a revnum Much of isvalid() was about testing if a prefix is a valid revnum. I want to reuse that soon, so let's move it out. There is no significant slowdown from the function call overhead. Differential Revision: https://phab.mercurial-scm.org/D3460

shortest: rename "test" variable to "prefix" Sorry if this is considered churn, but "prefix" just seems much clearer to me. Differential Revision: https://phab.mercurial-scm.org/D3459

shortest: move some safe code out of exception block The RevlogError and WdirUnsupported could be raised by _partialmatch(), but not by the rest of isvalid(), so let's move the rest out to make it clearer. Differential Revision: https://phab.mercurial-scm.org/D3458

revlog: don't say "not found" on internal error If index_node() returned NULL, then index_find_node() and and nt_partialmatch() used to return -2 to signal that the node was not found. However, we were passing in a revnum to index_node() that we knew should exist, so the only reason it could return NULL was due to some internal error or perhaps out of memory. Let's not use "not found" for these cases. I suppose we never noticed this because these error never happen in practice. I think there are more places where we should error out instead of reporting that the node was not found, but the cases mentioned above were all I cared about right now (because using the same error code for all failures simplified some future patches). Differential Revision: https://phab.mercurial-scm.org/D3457

revlog: extract function for getting node from known-to-exist rev Many of the calls to index_node() (which converts a rev to a nodeid) are done with a rev that's know to exist. If the function fails, there's something really wrong and we should just abort. This was done in only one place. This patch starts by extracting that code to a function that we can reuse in later patches. Differential Revision: https://phab.mercurial-scm.org/D3456

shortest: make {shortest("fffffffff")} work again {shortest("fffffffff")} should shorten it to the shortest unambiguous prefix for the working directory. It used to do that until I broke it in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), when we started returning the full hex nodeid for any working directory prefix shorter than 40 hex digits. This patch fixes it by catching WdirUnsupported specifically. Differential Revision: https://phab.mercurial-scm.org/D3455

revlog: use radix tree also for matching keys shorter than 4 hex digits I don't know what the reason for the 4-digit limit was, and I can't think of any real disadvantages of using the radix tree also when the requested minimum length is short. This speeds up `hg log -T '{shortest(node,1)}\n'` from 2m16s to 4.5s by making that not fall back to pure code. Differential Revision: https://phab.mercurial-scm.org/D3453

context: convert to hex for error message only for 20-byte changeid Now that 20-byte strings unambiguously mean binary (or a bug), we can specialize the conversion to hex for that case. Differential Revision: https://phab.mercurial-scm.org/D3452

context: clarify that only one attempt is made to interpret changeid We can now tell what type of revision specifier we have just by looking at it (we no longer attempt to interpret it in one way after the other -- that's now in scmutil.revsymbol()). Let's clarify this in the code by swithing to if/elif. Differential Revision: https://phab.mercurial-scm.org/D3451

context: only bother looking for broken dirstate for 20-byte changeid If we fail to look up a changeid in changectx.__init__, we check if it exactly matches any of the dirstate parents, and if it does, we print a more specific message ("working directory has unknown parent '...'!" instead of "unknown revision '...'"). The dirstate parents are always 20 bytes, so there's no need to check for a match when the given changeid is not 20 bytes. (And now that all the other allowed forms of changeid have been moved out of the constructor, there's no risk that a changeid that did match a dirstate parent was actually a valid bookmark.) Differential Revision: https://phab.mercurial-scm.org/D3450

revset: pass in lookup function to matchany() (issue5879) Silly mistake in f83cb91b052e.

test-hgweb: add test for foo-bar name lookup This is broken since f83cb91b052e "revset: pass in lookup function instead of repo (API)."

context: drop support for looking up context by ambiguous changeid (API) This removes support for using the changectx constructor (and thereby repo[x]) for looking up contexts by a stringified int, a namespace key (e.g. a bookmark), or a partial hex nodeid. This means that e.g. repo[<hex nodeid>] will now fail even if a bookmark with the same name exists (which is a good thing IMO). It also means that doing repo[<non-existent node>] no longer ends up loading namespaces (which was a surprising side-effect of creating of failing to create a context object that I recently ran into while debugging something unrelated to this series). Differential Revision: https://phab.mercurial-scm.org/D3449

tests: pass parent revision as integer to repo[x] in test-context-metadata.t Support for looking up by stringified int is going away. It's already deprecated, but I think I didn't notice this case because the test case pipes through `grep`. Differential Revision: https://phab.mercurial-scm.org/D3448

bookmarks: cache reverse mapping (issue5868) I chose a simpler implementation. If the initial cost of building reverse mapping is significant, we'll have to move it under @propertycache. The nodemap could be a dict of sets, but I think keeping a sorted list is better since each node is likely to have zero/one bookmark. Micro-benchmark with 1001 bookmarks and 1001 revisions: $ for n in `seq 0 1000`; do touch $n; hg book book$n; hg ci -qAm$n; done $ hg bookmarks --time > /dev/null (orig) time: real 0.040 secs (user 0.050+0.000 sys 0.000+0.000) (new) time: real 0.040 secs (user 0.040+0.000 sys 0.010+0.000) $ hg log -T '{bookmarks}\n' --time > /dev/null (orig) time: real 0.160 secs (user 0.160+0.000 sys 0.000+0.000) (new) time: real 0.090 secs (user 0.100+0.000 sys 0.000+0.000)

bookmarks: make argument names of _set/_del() more specific

bookmarks: extract function that looks up bookmark names by node

bookmarks: hide dict behind bmstore class This should make it clearer that the bmstore doesn't expose all dict APIs.

tests: remove pid file by default Previously, killdaemons.py would kill PIDs listed in a file then leave the file lingering around. If the PIDs are killed, then there's no point leaving the PID file around. In the worst case, a later invocation of killdaemons.py (run-tests.py invokes killdaemons.py after running a test) could kill a separate process whose PID conflicted with a previously-killed process. By removing the PID file, we eliminate this possibility. Some tests were manually removing the PID file after calling killdaemons.py. So we update these tests to not do this. Differential Revision: https://phab.mercurial-scm.org/D3443

tests: remove #require killdaemons The killdaemons hghave feature has returned True since it was introduced in 448d0c452140. As such, "#require killdaemons" has no effect and is superfluous. So we remove instances of it. Differential Revision: https://phab.mercurial-scm.org/D3442

pycompat: export queue module instead of symbols in module (API) Previously, pycompat and util re-exported individual symbols from the queue module. This had the side-effect of forcing the loading of the queue module whenever pycompat/util was imported. These symbols aren't used very often. So importing the module to get a handle on the symbols is wasteful. This commit changes pycompat so it no longer exports the individual symbols in the queue module. Instead, we make the imported module a "public" symbol. We drop the individual symbol aliases from the util module. All consumers are updated to use pycompat.queue.* instead. This change makes 300 invocations of `hg log -r. -T '{rev}\n'` a little faster: before: 18.44s after: 17.87s Differential Revision: https://phab.mercurial-scm.org/D3441

demandimport: make module ignores a set (API) The list of modules to ignore is used for membership testing. Yet it is defined as a list. Sets are more efficient for membership testing. So this commit converts the module list to a set. Since we took an API hit, I renamed the variable to further clarify the change. This appears to reduce the CPU time for running 300 invocations of `hg log -r. -T '{rev}'` on my i7-6700K: before: 18.64s after: 18.44s Differential Revision: https://phab.mercurial-scm.org/D3440

bundle2: mark the bundle2 part as advisory (issue5872) It blocks old clients to read bundle including this part. Differential Revision: https://phab.mercurial-scm.org/D3481

debugbundle: also display if a part is mandatory or advisory Most parts are mandatory but when introducing new parts, they should be advisory if included by default or old clients won't be able to process it. Differential Revision: https://phab.mercurial-scm.org/D3480

cmdutil: remove unused 'confirmopts' constant Follows up f10cb49951e1.

formatter: ditch namedtuple in favor of attr

log: consume --stat/patch options at constructor of changesetprinter The variable name, self.diffopts, was confusing. Let's split it to two booleans.

log: cache diffopts instance It appears that calling patch.diff*opts() repeatedly has some cost. $ hg log -T '{rev}\n' -R mercurial --time > /dev/null (orig) time: real 4.430 secs (user 4.370+0.000 sys 0.050+0.000) (new) time: real 1.950 secs (user 1.880+0.000 sys 0.060+0.000) 'diffopts or {}' isn't necessary as patch.diff*opts() accepts opts=None.

merge with stable

Added signature for changeset 6614cac550ae

Added tag 4.6 for changeset 6614cac550ae

filelog: don't crash on invalid copy metadata (issue5748) "copy" and "copyrev" are both supposed to appear next to each other. However, a user report demonstrated a crash that indicates that something in the wild is producing "copy" without "copyrev" (probably `hg convert`). While we should definitely fix the source of the bad metadata, the bad code causing the crash is already in the wild and who knows how many repositories are impacted. So let's be more defensive when accessing the file revision metadata.

httppeer: detect redirect to URL without query string (issue5860) 197d10e157ce subtly changed the HTTP peer's handling of HTTP redirects. Before that changeset, we instantiated an HTTP peer instance and performed the capabilities lookup with that instance. The old code had the following relevant properties: 1) The HTTP request layer would automatically follow HTTP redirects. 2) An encountered HTTP redirect would update a peer instance variable pointing to the repo URL. 3) The peer would automagically perform a "capabilities" command request if a caller requested capabilities but capabilities were not yet defined. The first HTTP request issued by a peer is for ?cmd=capabilities. If the server responds with an HTTP redirect to a ?cmd=capabilities URL, the HTTP request layer automatically followed it, retrieved a valid capabilities response, and the peer's base URL was updated automatically so subsequent requests used the proper URL. In other words, things "just worked." In the case where the server redirected to a URL without the ?cmd=capabilities query string, the HTTP request layer would follow the redirect and likely encounter HTML. The peer's base URL would be updated and the unexpected Content-Type would raise a RepoError. We would catch RepoError and immediately call between() (testing the case for pre 0.9.1 servers not supporting the "capabilities" command). e.g. try: inst._fetchcaps() except error.RepoError: inst.between([(nullid, nullid)]) between() would eventually call into _callstream(). And _callstream() made a call to self.capable('httpheader'). capable() would call self.capabilities(), which would see that no capabilities were set (because HTML was returned for that request) and call the "capabilities" command to fetch capabilities. Because the base URL had been updated from the redirect, this 2nd "capabilities" command would succeed and the client would immediately call "between," which would also succeed. The legacy handshake succeeded. Only because "capabilities" was successfully executed as a side effect did the peer recognize that it was talking to a modern server. In other words, this all appeared to work accidentally. After 197d10e157ce, we stopped calling the "capabilities" command on the peer instance. Instead, we made the request via a low-level opener, detected the redirect as part of response handling code, and passed the redirected URL into the constructed peer instance. For cases where the redirected URL included the query string, this "just worked." But for cases where the redirected URL stripped the query string, we threw RepoError and because we removed the "between" handshake fallback, we fell through to the "is a static HTTP repo" check and performed an HTTP request for .hg/requires. While 197d10e157ce was marked as backwards incompatible, the only intended backwards incompatible behavior was not performing the "between" fallback. It was not realized that the "between" command had the side-effect of recovering from an errant redirect that dropped the query string. This commit restores the previous behavior and allows clients to handle a redirect that drops the query string. In the case where the request is redirected and the query string is dropped, we raise a special case of RepoError. We then catch this special exception in the handshake code and perform another "capabilities" request against the redirected URL. If that works, all is well. Otherwise, we fall back to the "is a static HTTP repo" check. The new code is arguably better than before 197d10e157ce, as it is explicit about the expected behavior and we avoid performing a "between" request, saving a server round trip. Differential Revision: https://phab.mercurial-scm.org/D3433

hgweb: prevent triggering dummy href="#" handler Follow up for the previous patch.

paper: add href="#" to links with click handlers This restores the styling that was accidentally removed by the previous change to these files. Differential Revision: https://phab.mercurial-scm.org/D3438

paper: don't register click handlers with inline javascript (issue5812) The use of inline href="javascript:" undermines CSP policies that don't allow inline javascript. This commit changes the registering of the diffstat and line wrapping toggle handlers to the the global DOMContentLoaded handler, thus eliminating all inline javascript from the paper template. Differential Revision: https://phab.mercurial-scm.org/D3437

hgweb: allow Content-Security-Policy header on 304 responses (issue5844) A side-effect of 98baf8dea553 was that the Content-Security-Policy header was set on all HTTP responses by default. This header wasn't in our list of allowed headers for HTTP 304 responses. This would trigger a ProgrammingError when a 304 response was issued via hgwebdir. This commit adds Content-Security-Policy to the allow list of headers for 304 responses so we no longer encounter the error. Differential Revision: https://phab.mercurial-scm.org/D3436

hgweb: discard Content-Type header for 304 responses (issue5844) A side-effect of 98baf8dea553 was that hgwebdir always sets a global default for the Content-Type header. HTTP 304 responses don't allow the Content-Type header. So a side-effect of this change was that HTTP 304 responses served via hgwebdir resulted in a ProgrammingError being raised. This commit teaches our 304 response issuing code to drop the Content-Type header. Differential Revision: https://phab.mercurial-scm.org/D3435

tests: add tests demonstrating ISE for HTTP 304 responses with hgwebdir There are two separate failures here. One for the Content-Type header. Another for the Content-Security-Policy header. Differential Revision: https://phab.mercurial-scm.org/D3434