parser: use %d instead of %s for interpolating error position Error position is an int, so we should use %d instead of %s. Fixes failures on Python 3.

manifest: unbreak pure-python manifest parsing on Python 3

context: use portable construction to verify int parsing

ui: portably bytestring-ify url object

scmutil: fix key generation to portably bytestringify integer

branchmap: stringify int in a portable way We actually need a bytes in Python 3, and thanks to our nasty source loader this will portably do the right thing.

branchmap: don't use buffer() on Python 3 This is certainly slower than the Python 2 code, but it works, and we can revisit it later if it's a problem.

py3: use bytearray() instead of array('c', ...) constructions Portable from 2.6-3.6.

summary: don't explicitly str() something we're about to %s str() is wrong on Python 3 here, and %s implicitly calls str() anyway, so this was just extra dancing for no reason.

context: implement both __bytes__ and __str__ for Python 3 They're very similar, for obvious reasons.

context: work around `long` not existing on Python 3 I can't figure out what this branch is even trying to accomplish, and it was introduced in ac89a23ca814 which doesn't really shed any insight into why longs are treated differently from ints.

phases: explicitly evaluate list returned by map On Python 3 map() returns a generator, which bool()s to true even if it had an empty input set. Work around this by using list() on the map() result.

ui: check for --debugger in sys.argv using r-string to avoid bytes on py3 Our source loader was errantly turning this --debugger into a bytes, which was then causing me to still get a pager when I was using the debugger on py3. That made life hard.

minirst: use bytes.strip instead of str.strip bytes.strip exists in Python 2.6 and Python 2.7 also.

smcposix: pass unicode as first argument to array.array This is an instance where we can safely convert the first argument, rest are the cases except one where we are using 'c' which is not there in Python 3. So that needs to be handled differently. This will help in making `hg help` run on Python 3.

util: pass encoding.[encoding|encodingmode] as unicodes We need to pass str to encode() and decode().

win32text: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

transplant: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

shelve: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

share: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

mq: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

largefiles: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

keyword: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

journal: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

histedit: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

eol: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

convert: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

test-bundle2-remote-changegroup: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

subrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

repair: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

merge: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

hg-mod: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

commands: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

cmdutil: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

localrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

minirst: make encoding.encoding unicodes to pass into encode() and decode()

minirst: make regular expressions bytes

setup: convert setupversion to unicode Something deep in the bowels of distutils expects "version" passed to setup() to be a str/unicode. So, convert the type. This still works on Python 2 because the string is ascii and an implicit coercion back to str/bytes should work without issue. If it does cause problems, we can always make the unicode conversion dependent on running Python 3. This change makes `python3.5 setup.py install` work.

util: add debugstacktrace depth limit Useful when you don't care about the start of the stack, but only want to see the last entries.

util: strip trailing newline from debugstacktrace message This makes the function more convenient to use as drop-in replacement for ui.write & co.

tests: rework util.debugstacktrace tests Prepare for adding another test.

vfs: use repo.vfs.rename

vfs: use repo.vfs.unlinkpath

merge: use repo.wvfs.unlinkpath

vfs: use repo.wvfs.unlinkpath

policy: try and always have a bytes for module policy debuginstall now runs cleanly in Python 3.

init: zstd is already python3-ready, so don't run it through our importer

config: guard against setconfig specifying unicode values on py3 This was leading to some difficult to trace problems because the values were set in one place, but then blew up much later in the program. Exploding violently with an assertion seems reasonable here.

dispatch: enforce bytes when converting boolean flags to config items This fixes --verbose on Python 3.

py3: add a test that proves %include and missing-extension code works

extensions: tapdance to get reasonable import error formatting I'm not thrilled with this, but it seems to work.

rebase: abort if *any* commit in rebase set is public

merge with stable

httpconnection: rename config to groups Because that is what it is.

httpconnection: don't use dict() Not sure the history here. But we don't do this elsewhere.

formatter: set _first on formatter, not ui The _first field is used for tracking when to emit a separator between items. It seems like it's clearly formatter state, not ui state, so let's move it there.

rebase: unhide original working directory node as well (issue5219) By including the working directory revision at the start of rebase in the repo._rebaseset, we make sure it's not hidden when we update back to it at the end of the rebase. This feels like abusing the set a bit given its name (_rebaseset), but I couldn't think of another name that's clearly better.

rebase: pass in a regular set to _setrebasesetvisibility() Trivial refactoring to simplify the next patch.

tests: demonstrate broken rebase from obsolete commit This shows the bug I reported in issue 5219.

treemanifest: make node reuse match flat manifest behavior In a flat manifest, a node with the same content but different parents is still considered a new node. In the current tree manifests however, if the content is the same, we ignore the parents entirely and just reuse the existing node. In our external treemanifest extension, we want to allow having one treemanifest for every flat manifests, as a way of easeing the migration to treemanifests. To make this possible, let's change the root node treemanifest behavior to match the behavior for flat manifests, so we can have a 1:1 relationship. While this sounds like a BC breakage, it's not actually a state users can normally get in because: A) you can't make empty commits, and B) even if you try to make an empty commit (by making a commit then amending it's changes away), the higher level commit logic in localrepo.commitctx() forces the commit to use the original p1 manifest node if no files were changed. So this would only affect extensions and automation that reached passed the normal localrepo.commit() logic straight into the manifest logic.

help: update help.internalstable for new censor docs

parsers: drop old nonnormalentries method This is okay to do because the Python will fall back transparently if the method is missing.

parsers: avoid leak of nonnset and otherpset Py_BuildValue increments the refcount, rather than stealing the reference, which I missed in code review.

sslutil: issue warning when [hostfingerprint] is used Mercurial 3.9 added the [hostsecurity] section, which is better than [hostfingerprints] in every way. One of the ways that [hostsecurity] is better is that it supports SHA-256 and SHA-512 fingerprints, not just SHA-1 fingerprints. The world is moving away from SHA-1 because it is borderline secure. Mercurial should be part of that movement. This patch adds a warning when a valid SHA-1 fingerprint from the [hostfingerprints] section is being used. The warning informs users to switch to [hostsecurity]. It even prints the config option they should set. It uses the SHA-256 fingerprint because recommending a SHA-1 fingerprint in 2017 would be ill-advised. The warning will print itself on every connection to a server until it is fixed. There is no way to suppress the warning. I admit this is annoying. But given the security implications of sticking with SHA-1, I think this is justified. If this patch is accepted, I'll likely send a follow-up to start warning on SHA-1 certificates in [hostsecurity] as well. Then sometime down the road, we can drop support for SHA-1 fingerprints. Credit for this idea comes from timeless in issue 5466.

setup: use setuptools on Windows (issue5400) We've had a long, complicated history with setuptools. We want to make it the universal default. But when we do, it breaks things. `python setup.py build` is broken on Windows today. Forcing the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things. Since the previous bustage with making setuptools the default was on !Windows, it seems safe to move ahead with the setuptools transition on Windows. So this patch does that.

schemes: use br'' literal to define bytes regexp

help: fix layout of pre-formatted text

help: fix example of revs() fileset

filecache: make 'join' abstract All subclasses redefine this method, so we can make it abstract.

filecache: explicitly test 'repofilecache' The tests is actually about testing a repofilecache (it uses a fake repo). We make this clear to prevent blockers while cleaning theses API.

repofilecache: directly use 'repo.vfs.join' The 'vfs' attribute already have all methods we need, the value of going through the repository for this is low. so we removes it.

repofilecache: define a 'join' method We are about to turn the 'join' method of the base class Abstract, so we need on to be defined in the localrepo. The ultimate goal here is to be able to stop relying for the 'localrepo' class to have a 'join' methods (there is above one hundred methods on 'localrepo'. This change make te 'repo' file cache have its own code so that we can prepare this change to the repostory class. explicite join

wix: add censor docs to installer script Spotted by Matt Harbison.

internals: add some brief documentation about censor

localrepo: rename proxycls to filteredrepo When debugging in a Python shell, the type of "repo" is "proxycls", which could confuse new people. In [1]: repo Out[1]: <mercurial.localrepo.proxycls at 0x7f65d4b976d0> Let's rename it to "filteredrepo" to make it clearer.

dirstate: track otherparent files same as nonnormal Calling dirstate.setparents() is expensive in a large repo because it iterates over every file in the dirstate. It does so to undo any merge state or otherparent state files. Merge state files are already covered by dirstate._nonnormalset, so we just need to track otherparent files in a similar manner to avoid the full iteration here. Fixing this shaves 20-25% off histedit in large repos. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

hgweb: use patch.diffhunks in webutil.diffs to simplify the algorithm Function patch.diffhunks yields items for a "block" (i.e. a file) as a whole so take advantage of this to simplify the algorithm and avoid parsing diff lines to determine whether we're starting a new "block" or not. Thus we drop to external block counter and rely on diffhunks iterations instead. We also take advantage of the fact that patch.diffhunks() yields *lines* of hunks (instead of a string) to avoid building a list that is ''.join-ed into a string that is then split. As lines in 'header' returned by patch.diffhunks() have no trailing new line, we need to insert it ourselves to match template expectations.

hgweb: start enumerate at 1 in webutil.diffs's inner function prettyprintlines

patch: add a diffhunks function yielding (diffheaders, hunks) trydiff function now yield (header, hunks) tuple that are processed by diffhunks(). Then diff() is a wrapper around diffhunks().

mdiff: let unidiff return (diffheader, hunks) This will be used to make it possible to filter diff hunks based on this range information. Now unidiff returns a 'hunks' generator that yield tuple (hunkrange, hunklines) coming from _unidiff() with 'newline at end of file' processing.

mdiff: extract a checknonewline inner function in unidiff()

mdiff: distinguish diff headers from hunks in unidiff() Let unidiff return the list of headers it produces (lines '--- <original>' and '+++ <new>') apart from diff hunks. In patch.diff(), we combine headers generated there (not specific to unified format) with those from unidiff(). By returning a list of header lines, we do not append new lines in datetag inner function of unidiff() so that all header lines are '\n'.join-ed in a similar way.

test: end printed diff "hunks" with an empty string in test-context.py So that the resulting diff is correct and does not include a spurious empty line between lines "diff --git a/foo b/foo" and "--- a/foo".

mdiff: let _unidiff yield hunks as (<range information>, <hunk lines>) Now _unidiff yields each hunk lines packed into a tuple with the "range information" `(s1, l1, s2, l2)` that is used to build the typical hunk header '@@ -s1,l1 +s2,l2 @@'. This will be used to make it possible to filter diff hunks based on this range information. The new "range information" is ignored in unidiff() (only caller of _unidiff) for now.

mdiff: turn the comment above _unidiff into a docstring

mdiff: compute newlines-splitted texts within _unidiff There is no reason to compute splitted texts l1, l2 in unidiff() before calling _unidiff as they are only used with the latter function.

dispatch: add pagination of two more help cases I missed these in the last round anf Yuya spotted them in review. Thanks!

help: avoid mutating passed-in `keep` list in `formattedhelp`

extensions: use [0:1] slice on config path instead of [0] This behaves the same in Python 2 and Python 3, even though the path is a bytes.

extensions: use inspect module instead of func_code.co_argcount Fixes the extsetup argspec check on Python 3.

treemanifest: add tests covering hg diff of partial trees Previously the hg files tests also covered the logic (i.e. treemanifest.matches) that governed how hg diff limited its diff. In a future patch we will be switching treemanifest.diff() to have a custom implementation, so let's go ahead and add equivalent test coverage for hg diff.

context: remove uses of manifest.matches This removes the uses of manifest.matches in context.py in favor of the new manifest.diff(match) api. This is part of removing manifest.matches since it is O(manifest). To drop the dependency on ctx._manifestmatches(s) we transfer responsibilty for creating status oriented manifests over to ctx._buildstatusmanifest(s). This already existed for workingctx, we just need to implement a simple version for basectx. The old _manifestmatches functionality is basically identical to the _buildstatusmanifest functionality (minus the matching part), so no behavior should be lost.

context: remove assumptions about manifest creation during _buildstatus Previously we called self.manifest() in some cases to preload the first manifest. This relied on the assumption that the later _manifestmatches() call did not duplicate any work the original self.manifest() call did. Let's remove that assumption, since it bit me during my refactors of this area and is easy to remove.

context: move _manifest from committablectx to workingctx committablectx had a _manifest implementation that was only used by the derived workingctx class. The other derived versions, like memctx and metadataonlyctx, define their own _manifest functions. Let's move the function down to workingctx, and let's break it into two parts, the _manifest part that reads from self._status, and the part that actually builds the new manifest. This separation will let us reuse the builder code in a future patch to answer _buildstatus with varying status inputs, since workingctx has special behavior for _buildstatus that the other ctx's don't have.

status: handle more node indicators in buildstatus There are several different node markers that indicate different working copy states. The context._buildstatus function was only handling one of them, and this patch makes it handle all of them (falling back to file content comparisons when in one of these states). This affects a future patch where we get rid of context._manifestmatches as part of getting rid of manifest.matches(). context._manifestmatches is currently hacky in that it uses the newnodeid for all added and modified files, which is why the current newnodeid check is sufficient. When we get rid of this function and use the normal manifest.diff function, we start to see the other indicators in the nodes, so they need to be handled or else the tests fail.

merge: remove uses of manifest.matches This gets rid of the manifest.matches calls in merge.py in favor of the new api. This is part of getting rid of manifest.matches since it is O(manifest).