subrepo: mask out passwords embedded in the messages displaying a URL I noticed the password in maintenance logs for the "no changes since last push" and "pushing to" messages when pushing with an explicit path. But the test case here with :pushurl was also affected. I didn't see that cloning or pulling subrepos on demand had this problem, but it seems safer to just mask that too. There's a bit of a disconnect here, because it looks like clone is slicing off the password (makes sense not to store it in the hgrc in cleartext). But not shearing it off of an explicit path causes the subrepo not to realize that it already pushed the latest stuff. This is the easiest fix, however.

buildrpm: remove TODO that looks done to me Differential Revision: https://phab.mercurial-scm.org/D4494

buildrpm: fix embarassing bug in shell variable expansions `$distance_$node` looks up and concatenates `$distance_` and `$node`. `$distance_` is empty, so we were getting the node without the distance. Using the curly braces makes our intent explicit and produces better-versioned RPMS. Differential Revision: https://phab.mercurial-scm.org/D4493

manifest: fix leak on error return from lazymanifest_filtercopy() Spotted by ASAN. free(copy->lines) and Py_DECREF(copy->pydata) are replaced by Py_XDECREF(copy), which should call lazymanifest_dealloc(). Freeing half-initialized copy->lines is safe since copy->numlines holds a valid value.

manifest: incref/decref copy->pydata to clarify 'copy' holds a reference

dirstate: use tuple interface to fix leak in pack_dirstate() Spotted by ASAN. Unlike PyTuple_GET_ITEM(), PySequence_ITEM() returns a new reference. This bug could be fixed by inserting Py_CLEAR() and Py_XDECREF() appropriately, but I think requiring a tuple object is simpler and less error-prone. The cext version is jumped to 10 since 6..9 are used in the default branch. We'll need to bump it again at merge.

xdiff: fix leak in hunk_consumer() Spotted by ASAN. Since PyList_Append() does not "steal" a reference, Py_DECREF() is always required. Perhaps, this is the largest leak in this series.

base85: fix leak on error return from b85decode() Spotted by ASAN. We don't need to initialize 'out' to NULL, but I decided to do that for clarity.

Added signature for changeset ede3bf31fe63

Added tag 4.7.1 for changeset ede3bf31fe63

hgweb: load revcount + 1 entries to fill nextentry in log page (issue5972) "revcount + 1" is moved to the call site to make it clearer.

scmutil: avoid quadratic membership testing (issue5969) tr.changes['revs'] is an xrange, which has an O(n) __contains__ implementation. The `rev not in newrevs` lookup a few lines below will therefore be O(n^2) if all incoming changesets are public. This issue isn't present on @ because 45e05d39d9ce introduced a custom type implementing an xrange primitive with O(1) contains and switched tr.changes['revs'] to be an instance of that type. We work around the problem on the stable branch by casting the xrange to a set. This is a bit hacky because it requires allocating memory to hold each integer in the range. But we are already holding the full set of pulled revision numbers in memory multiple times (such as in `tr.changes['phases']`). So this is a relatively minor problem. This issue has been present since the phases reporting code was introduced in the 4.7 cycle by eb9835014d20. This change should be reverted/ignored when stable is merged into default. On the mozilla-unified repository with 483492 changesets, `hg clone` time improves substantially: before: 1843.700s user; 29.810s sys after: 461.170s user; 29.360s sys

phases: drop dead code in `newheads` There are new code earlier in the function achieving the same result. Spotted by Yuya Nishihara.

phases: fix `nullid` reference in newheads Spotted by Yuya Nishihara.

beautifygraph: don't warn about busted terminal if HGPLAIN is set This was breaking some automation for users that discovered the extension and turned it on. Differential Revision: https://phab.mercurial-scm.org/D4338

merge: do not delete untracked files silently (issue5962) 37450a122128, 2a774cae3a03, and 656ac240f392 weren't enough to prevent data loss. No unknown "files" weren't deleted before 7a8a16f8ea22, "context: also consider path conflicts when clearing unknown files."

perf: fix typo in perfphasesremote()

remotephase: avoid full changelog iteration (issue5964) Changeset 88efb7d6bcb6 introduced a performance regression by triggering a full ancestors walk. This changeset reworks this logic so that we no longer walk down the full changelog. The motivation for 88efb7d6bcb6, issue5939, is still fixed. mercurial compared to a draft repository ---------------------------------------- 8eeed92475d5: 0.012637 seconds 88efb7d6bcb6: 0.202699 seconds (x16) 46da52f4b820: 0.215551 seconds (+6%) this code: 0.008397 seconds (-33% from base) The payload size reduction we see in `test-bookmarks-pushpull.t` comes from a more aggressive filter of nullid and is harmless.

remotephase: fast path newheads computation in simple case (issue5964) Changeset 88efb7d6bcb6 fixed the logic of `phases.newheads` but greatly regressed its performance (up to many order of magnitude). The first step to fix the regression is to exit early when there is no work to do. If there are no heads to filter or not roots to filter them, we don't have to do any work. This fixes the regression when talking to an all public changeset. The performance is even better than before. pypy, compared to an all public repo ------------------------------------ 8eeed92475d5: 0.005758 seconds 88efb7d6bcb6: 0.602517 seconds (x104) this code: 0.001508 seconds (-74% from base) mercurial compared to an all public repo ---------------------------------------- 8eeed92475d5: 0.000577 seconds 88efb7d6bcb6: 0.185316 seconds (x321) this code: 0.000150 seconds (-74% from base) The performance of newheads, when actual computations are required, is fixed in the next changeset.

perf: add a perfphasesremote command This command measure the time we spend analysing remote phase during push and pull and display some information relevant to this computation. The `test-contrib-perf.t` expected output has to be updated but I do need these module for this perf command.

sparse-revlog: fix delta validity computation When considering the validity of a delta with sparse-revlog, we check the size of the largest read. To do so, we use some regular logic with the extra delta information. Some of this logic was not handling this extra delta properly, confusing it with "nullrev". This confusion with nullrev lead to wrong results for this computation but preventing a crash. Changeset 781b2720d2ac on default revealed this error, crashing. This changeset fixes the logic on stable so that the computation is correct (and the crash is averted). The fix is made on stable as this will impact 4.7 clients interacting with sparse-revlog repositories (eg: created by later version).

convert: don't drop missing or corrupt tag entries Cleaning up the tags file could be a useful feature in some cases, so maybe there should be a switch for this. However, the default hg -> hg convert tries to maintain identical hashes (thus convert.hg.saverev is off by default, but is on by default for other source types). It looks like _rewritesubstate() has a `continue` in it, and therefore a similar problem. I ran into this conversion divergence when a coworker "merged" two repositories by copy/pasting all of the files from the source repo and massaging the code, and forgetting to revert the .hg* files. That silently emptied the .hgtags file after the conversion. (This isn't the manifest node bug Yuya has been helping with- this occurred well after the bzr -> hg conversion and wasn't a merge commit, which made it extra puzzling. That bug is still an issue.)

hgweb: catch ParseError that's raised by revset.match() Some queries, like the demonstrated "first(::)", fail earlier than we call mfunc(), and that results in a "500 Internal Server Error". To prevent it, revset.match() also needs to be in a try-except block.

tests: update test expectations in pre-2.7.9 branch of this test As far as I can tell this is the only spot that got overlooked. Rough test methodology (run inside an hg working copy): docker run --rm -v `pwd`:/hg -it debian:wheezy bash apt-get update apt-get install python python-dev build-essential unzip mercurial \ wget libbz2-dev make testpy-2.7.8 You could try and use the system Python, but it's 2.7.3 and has lots of issues for hg these days that are not worth fixing. Differential Revision: https://phab.mercurial-scm.org/D4070

packaging: always execute builddeb from source root Without this, dockerdeb and the make targets that invoke it may fail depending on the pwd at the time of invocation.

Added signature for changeset 33ac6a72308a

Added tag 4.7 for changeset 33ac6a72308a

revlog: fix descendant deprecated method Fix the descendant deprecated method introduced earlier in this cycle. This was caught by Yuya, thank you.

ui: remove commands.status.terse=u from ui.tweakdefaults commands.status.terse=u can add significant overhead when operating on large repositories. Using the Firefox repository: HGRCPATH= hg --time status time: real 1.340 secs (user 0.960+0.000 sys 0.380+0.000) HGRCPATH= hg --time --config commands.status.terse=u status time: real 2.420 secs (user 2.070+0.000 sys 0.360+0.000) HGRCPATH= hg --time --config extensions.fsmonitor= status time: real 0.080 secs (user 0.050+0.010 sys 0.040+0.000) HGRCPATH= ~/src/hg/hg --time --config extensions.fsmonitor= --config commands.status.terse=u status time: real 2.470 secs (user 2.080+0.000 sys 0.390+0.000) The performance regression - especially when fsmonitor is being used - is too much to stomach for the 4.7 release. We've decided to remove commands.status.terse=u from ui.tweakdefaults until we can improve its performance, hopefully in the 4.8 cycle. This commit effectively backs out 6acf41bb8d40.

context: safegaurd against 'lx' being passed as file flag in manifest Subversion can have a file as executable link. When using hgsubversion, we will have both islink and isexec True. This will lead to _flags being set to `lx`. However, manifest expects flag to be one-byte so it will crash if 'lx' is passed. Also it's impossible to have an executable link. This patch will safegaurd us from having 'lx' being a possible value. This was authored by Ivan Lezhankin from Yandex. Differential Revision: https://phab.mercurial-scm.org/D3985

localrepo: unconditionally enable general delta with sparse revlogs This come as an extra security, better safe than sorry.

sparse-revlog: also use sparse-revlog config as a general delta trigger Sparse revlog rely on general delta, so we should make sure it is used.

fileset: suppress EACCES while reading arbitrary paths via filectx API On Windows, EACCES is raised in place of EISDIR. This patch simply adds EACCES to the list of errors to be ignored since I think it's okay for filesets to treat inaccessible working-copy files as not existing.

merge: do the trivial resolution after updating sparse checkout In merge, we do trivial resolution for files which were deleted on one side and changed on other. When sparse extension in involved that file might not be present in wdir and trivial resolution can lead to file not found error. This patch make sure we updates the sparse checkout before doing the trivial resolution. This fixes the test failure demonstrated in previous patch. Differential Revision: https://phab.mercurial-scm.org/D3984

sparse: add test showing `hg merge` is broken while using sparse extension This patch adds a test to demonstrate that `hg merge` is broken in some cases while using sparse extension. The case is when you have a file which is: * modified between current wdir parent and merge base * deleted between merge base and merge destination * excluded from sparse checkout Doing `hg merge` results in file not found error. Next patch will fix this. Differential Revision: https://phab.mercurial-scm.org/D3983

context: use hex nodeid in error about filtered node I went a little too far in my cleanup in 9231148ea599 (context: convert to hex for error message only for 20-byte changeid, 2018-04-06). I missed that the case where a binary nodeid refers to a filtered node. Differential Revision: https://phab.mercurial-scm.org/D3987

doc: fix underline length for config title (issue5949) This was an error in the initial commit. Thanks go to Marcel Svitalský for reporting the bug.

clone: process 'lookup' return as an arbitrary symbol In theory, checkout is expected to be a node here because it was returned by peer.lookup. In practice, multiple important extensions (like hg-git, hg-subversion) use peers not backed by a mercurial repository where lookup cannot return a node. Allowing arbitrary symbols is necessary to make these extensions working with 4.7. We should probably introduce a new API in Core to have these extensions to work without abusing the lookup API. In the meantime, a small change to restore compatibility in 4.7 seems in order.

tests: use inline Python instead of sed to add trailing whitespace The sed invocation was failing on OS X and FreeBSD. I'm far too lazy to diagnose that, so just use some inline Python to fix the build.

context: add missing b prefix This fixes mercurial/context.py:593: SyntaxError: cannot mix bytes and nonbytes literals (context.py, line 593) in Python 3.

doctest: convert matcher root to native path Otherwise it wouldn't be caught by a fast path of pathutil.canonpath(), and fall back to file identity checks.

test-fileset: make con.xml in output conditionally available

test-obsmarker-template: run mkcommit in subshell to isolate envvars I don't know if it is a feature or a bug, but it appears that environment variables passed to a shell function persist on /bin/sh.

test-obsmarker-template: add missing HGENCODING=latin-1 It just worked by accident on some Linux sh.

gitweb: add link to graph error.tmpl wasn't consistent with other templates in gitweb in that it was missing a "graph" link. This commit makes it consistent.

hgweb: strip trailing '/' in apppath before appending '/static/' (issue5943) Differential Revision: https://phab.mercurial-scm.org/D3978

config: rename `revlog` section into `storage` The idea was suggested by Gregory Szorc on IRC. It is more generic and seems better. It is probably best to rename the section before it ever makes into an official (non-rc) release. The only config option currently in this section have been prefixed with `revlog` to clarify it applies to `revlog` related storage.

Added signature for changeset e90130af47ce

Added tag 4.7rc0 for changeset e90130af47ce

configitems: restore alias for format.aggressivemergedeltas 913ca175c4ae broke BC by dropping support for reading format.aggressivemergedeltas. Let's restore it. Differential Revision: https://phab.mercurial-scm.org/D3966

worker: call selector.close() to release polling resources

release: merge default into stable for 4.7 release freeze

histedit: drop --no-backup option Dropping this option because now we have a better option than passing --no-backup flag every time, now user can set a config in hgrc: [ui] history-editing-backup = False This config aims to operate on every history editing command and it is still work in progress. As yuya suggessted it probably to late to add full support this config, so making this as an experimental config. Differential Revision: https://phab.mercurial-scm.org/D3965

aggressivemergedelta: document rename and move to `revlog` section The config does not follow our naming guideline and "Aggressive" is probably a word to keep away from users. The option does not truly fit in the `format` section. It can be turned on and off for existing repository without much consequence regarding compatibility. A new `revlog` option is created to control behavior related to revlog writing and reading. We can see multiple other config options that could be migrated there. * format.maxchainlen * experimental.mmapindexthreshold * experimental.sparse-read.density-threshold (in an updated form) * experimental.sparse-read.min-gap-size (in an updated form) In addition, we can foresee at least a couple of sparse-revlog related option coming too (to reduce delta chain length and increase snapshot reuse) These two extra options might fit there too. Unless we want to create a section dedicated to caches and performance. * format.chunkcachesize * format.manifestcachesize For now, we only migrate `optimize-delta-parent-choice` since it is getting out of experimental. It is too close to the release to move the other one. In addition, we still lack proper the prioritization of alias that would help renaming them without bad consequence for users. (Not fully happy about the `revlog` name but could not find better).

aggressivemergedeltas: rename variable internally The "aggressivemergedeltas" name is not great. First, it is quite long, second, we would rather have less "Aggressive" names within the project. We are about to rename the config option, so it seems the appropriate time to rename the internal variable.

config: document the purpose of the `format` config section The config section now have a top level documentation to clarify its intend and usage. In particular, user are now explain when the option are taken in account and how to convert repository. There are an handful of experimental options in this section that does not match its definition. They should be relocated to other section before getting out of experimental. (see next changeset for one example).