hgweb: register web.comparisoncontext to the config table This was caught in some server side logging added to debug py3 issues.

Added signature for changeset 1c8c54cf9725

Added tag 4.8.1 for changeset 1c8c54cf9725

rebase: fix path auditing to audit path relative to repo root (issue5818) Before this patch, when rebasing a file called "foo/bar", we would check e.g. if "/foo" (i.e. rooted at the file system root) was a symlink. Differential Revision: https://phab.mercurial-scm.org/D5361

tests: show bad path auditing in in-memory rebase Thanks to Yuya for providing this test case in https://bz.mercurial-scm.org/show_bug.cgi?id=5818. Differential Revision: https://phab.mercurial-scm.org/D5368

tests: add a missing "cd .." to test-rebase-inmemory.t Differential Revision: https://phab.mercurial-scm.org/D5367

rust: fix possible out-of-bounds read through index_get_parents() index_get_parents() is an internal function, which doesn't check if the specified rev is valid. If rustlazyancestors() were instantiated with an invalid stoprev, it would access to invalid memory region. This is NOT a security fix as there's no Python code triggering the bug, but included in this series to not give a notion about the memory issue fixed by the previous patch.

revlog: fix out-of-bounds access by negative parents read from revlog (SEC) 82d6a35cf432 wasn't enough. Several callers don't check negative revisions but for -1 (nullrev), which would directly lead to out-of-bounds read, and buffer overflow could follow. RCE might be doable with carefully crafted revlog structure, though I don't think this would be useful attack surface.

rebase: fix dir/file conflict detection when using in-mem merge Differential Revision: https://phab.mercurial-scm.org/D5360

tests: show that in-mem rebase does not find path dir/file conflicts Differential Revision: https://phab.mercurial-scm.org/D5359

extdiff: register the configuration generated commands with a help category Otherwise, 'extdiff' shows up under file management and the rest of the commands are at the bottom under 'Uncategorized'.

rebase: abort in-mem rebase if there's a dirty merge state In-memory merge uses the on-disk merge state, so we should not allow it run in-memory merge when the merge state is not clean. We should probably not use the on-disk merge state when running in-memory merge, but chaning that is not suitable for the stable branch. Differential Revision: https://phab.mercurial-scm.org/D5357

rebase: preserve working copy when redoing in-mem rebase on disk When in-memory rebase runs into conflicts, we retry it on disk. But before we do that, we abort the in-memory rebase. That is done because even though it's mostly in memory, there are still a few state files written (e.g. the merge state). We should make it not write those files so we don't need to abort, but for the stable branch, let's explicitly clear the state we need to clear instead of running the usual abort code. Differential Revision: https://phab.mercurial-scm.org/D5356

tests: show that in-mem rebase falling back loses state Both working copy changes and the merge state is lost when in-memory rebase falls back to on-disk mode. Differential Revision: https://phab.mercurial-scm.org/D5355

commandserver: get around ETIMEDOUT raised by selectors2 selector.select() should exits with an empty event list on timed out, but selectors2 raises OSError if timeout expires while recovering from EINTR. Spotted while debugging new chg feature.

selectors2: backport minimal fix of timeout handling from 2.0.1 The original code would raise TypeError since OSError() doesn't support keyword arguments. We can't simply import the selectors 2.0.1, which still spawns "uname -p" through platform.system(). We could switch to the unreleased version, but I decided to not right now to minimize the change.

tests: sniff for libfuzzer actually being available in test-fuzz-targets.t When I upgraded the FreeBSD buildbot to 11.2 it seems we picked up clang6, but the default clang on FreeBSD doesn't include libfuzzer. I can't find a way to sniff for libfuzzer without running a compile, so here we are. Differential Revision: https://phab.mercurial-scm.org/D5270

tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t This isn't as robust as it probably should be, but for now it'll get the job done on the buildbots. Differential Revision: https://phab.mercurial-scm.org/D5269

tests: stabilize test-inherit-mode.t on FreeBSD and macOS (issue6026) Symbolic links are funny permissions-wise, but on the linked issue Yuya has convinced me that we can ignore this permissions issue on macOS (FreeBSD allows setting permissions bits but ignores them) and we'll be in fine shape.

wireprotov2peer: wait for initial object before resolving future As part of rolling out wireprotov2 with redirect support, I encountered an edge case with regards to future resolution. Essentially, the initial response frame from the server did not fully decode the initial CBOR object. The frame wasn't marked as EOS. In the previous code, we resolved the future for the request to response.objects(), which mapped to the commandresponse instance which would eventually produce a redirect. Upon receiving subsequent data, the initial CBOR object containing the redirect would be decoded and we'd process the redirect. However, the future would already have been resolved with the initial commandresponse.objects() and the client iterating over the objects wouldn't receive any objects from the redirect because the redirect was populating a different commandresponse instance! This commit changes the logic so we don't resolve futures until the initial CBOR response object is fully decoded or until EOS occurs. In cases where there is an empty or partial frame associated with a redirect, the future will now resolve with the commandresponse containing the proper series of decoded objects.

wireprotov2peer: always return a bool from _processredirect() Without this, we may stop servicing the redirect response if the future has already been resolved. And the future will often be resolved very early, since many consumers iterate the decoded CBOR object stream and expect data to lazily arrive.

tests: stabilize the recent checkexec changes on Windows This goes with bd0874977a5e.

checkexec: create destination directory if necessary Since 460733327640, a "share" use the cache of the source repository. A side effect is that no `.hg/cache` directory exists in the "share" anymore. As a result, the checkexec logic can't use it to create its temporary file and have to use the working copy for that. This is suboptimal, it pollutes the working copy and prevents them to keep the file around in cache. We do not want to use the cache directory for the share target, it might be on a different file system. So instead, we (try to) create the directory if it is missing. This is a simple change that fixes the current behavior regression on stable. On default, we should probably ensure the proper directories are created when initializing the repository. We should also introduce a 'wcache' directory to hold cache file related to the working copy. This would clarify the cache situation regarding shares. The tests catch a couple of other affected cases.

graft: do not try to skip rev derived from ancestor more than once (issue6024) We check 'x in revs' in other cases, so let's do the same. The test case credits to Tom Prince.

subrepo: print the status line before creating the peer for better diagnostics I ran into a problem where I tried updating to a different branch, and the process appeared to hang. It turned out that the subrepo revision wasn't available locally, and I must have originally cloned it from an `hg serve -S` on a machine that currently wasn't serving anything. It took 2+ minutes to timeout, and didn't mention what it was connecting to even then. There are a couple of other issues in this scenario too. - The repo is dirty after the failed checkout because the top level repo is updated first. We should probably make 2 passes- top down to pull everything needed, and then do an update once everything is in place. - Something must be reading .hgsubstate from wdir because if the same merge command is run after the timeout, a prompt is issued that the local and remote subrepo diverged, instead of hanging. But it lists the local version and remote version as having the same hash.

tests: allow for 100% of profiled time in sleep in test-profile.t I'm getting an annoying failure in this test on our builder, and I *think* what's happening is that the profiler is taking _just_ long enough to start that we're spending 100% of the profiled time in the sleep function, which was causing the leading space to not be printed since the 100 was in the first column of output. Differential Revision: https://phab.mercurial-scm.org/D5272

copystore: provide unit to ui.makeprogress()

verify: provide unit to ui.makeprogress()

tests: fix wireproto redirection test on systems without tls1.2 Our automated package builder has some ancient configuration that lacks modern TLS, which is how we noticed this. Tested: the test now passes on both macOS High Sierra (has tls1.2) and Ubuntu Trusty (which does not).

phabricator: ensure the command summaries are available in extension help Previously, `hg help phabricator` listed the 3 supported commands at the bottom of the extension help, but said "no help text available".

hgweb: cast bytearray to bytes PEP-3333 seems to indicate that bytes is the only allowed type that can be used to express the output of a WSGI application. And some WSGI environments seem to enforce this (mod_wsgi does). This commit universally casts bytearray instances to bytes to appease the WSGI specification. I found this because wireprotov2 is emitting bytearray instances. I'd like to keep things that way because the way it builds a data structure, bytearray is more efficient. I'd rather keep the low-level code efficient (and using bytearray) and cast at the edges than impose a performance penalty on code that may run outside WSGI contexts.

help: unjumble the list of default config values for `internals.config`

tweakdefaults: remove commands.resolve.mark-check=abort, it is too broken See issue6020 for the current case. I don't want to continue attempting to fix this on the stable branch, so I'm removing from tweakdefaults and will send fixes meant for the default branch and 4.9. Differential Revision: https://phab.mercurial-scm.org/D5225

resolve: when resolve.mark-check=abort, downgrade to warning if pats specified Previously, with --config resolve.mark-check=abort, running `hg resolve -m foo` would abort and emit a message saying to use --all. This command does not work, though: `hg resolve -m foo --all`, and it's really weird for --all to be the "--force" flag. My original goal with the option was to make it so that `hg resolve -m` (no filename arguments) was safer, which is why --all is used; in my mind, `hg resolve -m foo` should always mark it as resolved, and `--all` is how you specify "all the files", so that's why I chose `hg resolve -m --all` as the way out of `hg resolve -m` aborting. This commit makes all of this work the way it was meant to in my head :) Differential Revision: https://phab.mercurial-scm.org/D5218

Added signature for changeset a91a2837150b

Added tag 4.8 for changeset a91a2837150b

rust: fix signature of rustlazyancestors_init() function Obviously, sizeof(int) != mem::size_of::<usize>() on amd64, though the argument would be passed in 64-bit register anyway.

tests: require SQLite 3.8.3+ as sqlitestore relies on "WITH" clause The test fails on gcc112 because the SQLite is too old. https://sqlite.org/changes.html#version_3_8_3

relnotes: various tweaks for release notes Stop filtering out commits that are expected to be covered by releasenotes extension: now we want two lists, one for WhatsNew and one for ReleaseX.Y. Use `only(stoprev, startrev)` to make `relnotes -h` output be actually true about what revisions are included. More filter rules, mostly obvious. More classifying rules to have less things in "unsorted". Looks like nargs=1 was just making args.startrev and args.stoprev be lists for no reason. BC and API sections are renamed to what we're using on the WhatsNew page, and also just skipped if empty.

delta: skip "empty delta" optimisation for non-general case (issue6006) Non-general delta repository cannot delta against anything than prev. So even if the delta to prev is empty we should use it. This is similar to the change made in bafa1c4bb7a8. Differential Revision: https://phab.mercurial-scm.org/D5201

narrow: fix copies._fullcopytracing() narrowspec filtering in graft case I broke this too in 707c3804e607 (narrow: move copies overrides to core, 2018-09-28). Hopefully I'm done fixing things broken by that commit now. Differential Revision: https://phab.mercurial-scm.org/D5213

tests: demonstrate broken copies._fullcopytracing() Turns out copies._fullcopytracing() was also broken. Differential Revision: https://phab.mercurial-scm.org/D5212

narrow: make copies.pathcopies() filter with narrowspec again I broke this in 707c3804e607 (narrow: move copies overrides to core, 2018-09-28). Differential Revision: https://phab.mercurial-scm.org/D5203

tests: demonstrate broken copies.pathcopies() Differential Revision: https://phab.mercurial-scm.org/D5202

setup: explain to distutils how we write rc versions When we use a rc version number (e.g. 4.8rc0), bdist_msi is using distutils.StrictVersion to parse it into a tuple of numbers. By default, StrictVersion.version_re only recognizes [ab] for alpha/beta, where mercurial may use '-rc' or 'rc'. This change makes StrictVersion parse correctly our version numbers, so that bdist_msi doesn't fail on rc versions.

changegroup: restore default node ordering (issue6001) Changeset db5501d9 changed the default node ordering from "storage" to "linearize". While the new API is more explicit and cleaner, the "linearize" order is problematic on certain repositories like netbeans where it makes bundling slower the more nodes we bundle. Pushing and pulling 100 changesets was ~20% slower and pushing and pulling 1000 changesets was ~600% slower. A very quick analysis of profile traces showed that the pull operation was taking more time creating the delta. Putting back the old default order seems to be the safe option. With more time during the next cycle, we can understand better the impact of sorting with the DAG order by default, the source of the regression and how to mitigate it. /!\ We are still waiting for the full performance impact but with this patch, bundling and pulling locally (not on the performance workstation) 1000 changesets on the netbeans repository is as fast as before the regression. Differential Revision: https://phab.mercurial-scm.org/D5196

changegroup: introduce an explicit linear sorting We still need to linearize the revisions in some cases, introduce an explicit `linear` sorting before changing back the default order. Differential Revision: https://phab.mercurial-scm.org/D5195

fix: disable use of thread-based worker getfixes() accesses to repo, changectx, filectx, etc., so I believe there are code paths triggering data race. Mercurial API isn't thread safe in general.

configitems: rename the config to prevent adding an alias in future Right now the config option looks like: [experimental.server] stream-narrow-clones= which does not match how config options are generally defined in core. So let's rename this to: [experimental] server.stream-narrow-clones= before the new release so that we don't have to add an alias in future for this. Differential Revision: https://phab.mercurial-scm.org/D5198

sparse-revlog: only refine delta candidates in the sparse case (issue6006) Starting with 5aef5afa8654, a valid delta parent might be "refined". This allows repository using sparse-revlog to produce better delta chain by using better intermediate snapshot base. However, this refining step was performed in all cases, including for repository not using sparse-revlog. This could produce a strange chain in the general delta case and corrupted repository in the non-general delta case. We now skip this step unless sparse-revlog is in use. In issue 6006, Yuya Nishihara provided a test case using an external repository, so we did not include it. Finding "laboratory" condition to reproduce this case and implementing an efficient test reproducing it is a bit tricky. We do not foresee to have the time to provide one by the release date. Differential Revision: https://phab.mercurial-scm.org/D5197

http: work around custom http client classes that refuse extra attrs I have no idea what is going on with our custom http client code at Google, but it chokes on these extra attributes we're tucking on http clients. Since it feels more than a little wrong to just stuff extra data on a client, let's degrade gracefully when the client class refuses the attributes.

crecord: make nextsametype() check that parent item exists (issue6009) Items that represent files in curses interface don't have parents.

help: describe what ui.tweakdefaults changes, concretely Currently, one has to look at the code. A couple things are suboptimal: - probably not translatable - lines don't get wrapped (a couple are a bit too long) but it seems to better this way than without help at all. Differential Revision: https://phab.mercurial-scm.org/D5187

logexchange: convert paths to unix when detecting the active path This fixes the problem in the tests[1] where Windows was showing the whole path as the remotename for local repositories. Somebody with a better understanding of this extension should probably take a deeper look. There may be other cases that need to be converted- specifically the `elif not instance` and the missing `else` cases in activepath(). I also noticed when adding debug prints that the absolute path is stored in the file, probably not normalized. (It's wrapped up in $TESTTMP.) [1] https://buildbot.mercurial-scm.org/builders/Win7%20x86_64%20hg%20tests/builds/1042/steps/run-tests.py%20%28python%202.7.13%29/logs/stdio

help: update the default value specified for `profiling.time-track` I tried conditionalizing this in a `.. container::` block, but that seemed to add an extra blank line between the main text and the parenthetical.

profiling: revert the default mode back to 'cpu' on Windows On Windows, os.times() only returns user and system times. Real elapsed time is 0. That results in no actual times reported, an end wall time of 0.000000, and seemingly randomly sorted stack frames. This at least provides test stability in test-profile.t. I kind of think that `default=pycompat.iswindows and 'cpu' or 'real'` would be a better way to set the default in configitems, but I didn't see any other examples of this, and thought maybe there's a reason for that. That might allow plugging the value into the help text automatically- the documented default wasn't updated in db0dba2d157d.