context: add overlayfilectx.cmp() Differential Revision: https://phab.mercurial-scm.org/D1058

filemerge: use arbitraryfilectx for backups With in-memory merge, backup files might be overlayworkingfilectxs stored in memory. But they could also be real files if the user's backup directory is outside the working dir. Rather than have two code paths everywhere, let's use arbitraryfilectx so they can be consistent. Differential Revision: https://phab.mercurial-scm.org/D1057

registrar: add support for storing the type of command in func object This patch adds support for storing the type of command which is going to run in the func object. For this it does the following: 1) Add three possible values as attributes to the registrar.command class 2) Add a new argument to registrar.command._doregister function 3) Add a new attribute cmdtype to the func object The type of command will be helpful in deciding what level of access on hidden commits it can has. Differential Revision: https://phab.mercurial-scm.org/D736

releasenotes: fix documentation of similaritycheck() The function document says that it returns true when the fragment can be merged, but if you see the function just above it which is similar(), it writes already exists thing if return value from similaritycheck() is False which is just opposite of the doc. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D1119

configitems: register acl config section

configitems: register the 'extdata.*.diffargs' config

configitems: register the 'exdiff.opts.*' config

configitems: register the 'hostsecurity.*:verifycertsfile' config

configitems: register the 'hostsecurity.*:fingerprints' config

configitems: register the 'hostsecurity.*:ciphers' config

configitems: register the 'hostsecurity.*:minimumprotocol' config

configitems: register the test 'fakepatchtime.fakenow' config

configitems: register the test 'fakedirstatewritetime.fakenow' config

configitems: register the 'gpg' arbitraty key section

configitems: register the 'extdata' section

configitems: fix registration of extensions config The new 'itemregister' class should be used when loading the extension config. This was and oversight when itemregister was introduced.

configitems: register the test config in 'test-push-race.t'

configitems: register the test 'ui.ioerrors' config

configitems: register the test 'failafterfinalize.fail' config

configitems: register the test 'logtoprocess.foo' config

configitems: register the test 'failpush.reason' config

configitems: register the 'patchbomb.reply-to' config

configitems: register the 'patchbomb.cc' config

configitems: register the 'patchbomb.bcc' config

configitems: register the 'web.staticurl' config

configitems: register the 'experimental.uncommitondirtywdir' config

configitems: register the 'largefiles.usercache' config

configitems: register the 'largefiles.patterns' config

configitems: register the 'largefiles.minsize' config

configitem: register the 'notify.changegroup' config

configitem: register the 'notify.outgoing' config

configitem: register the 'notify.incoming' config

configitems: register the 'hgweb-paths' section

contrib-perf: register the 'parentscount' config item

contrib-perf: update the config registration There are various issue with the previous code, all hidden by the try-except clause. These are fixed now.

configitems: register the 'hostfingerprints' section

configitems: register the 'hostsecurity.minimumprotocol' config

configitems: register the 'progress.format' config

configitems: register the 'blackbox.maxfiles' config

hgweb: fix decodevaluefromheaders to always return a bytes value That's more in line with what we want, and we know it's ASCII data since that's all HTTP technically allows in headers anyway. Differential Revision: https://phab.mercurial-scm.org/D1112

hgweb: more "headers are native strs" cleanup I'll fix the decodevaluefromheaders function in an upcoming change. Differential Revision: https://phab.mercurial-scm.org/D1111

hgweb: when unpacking args from request form, convert to bytes We assume http-originated values are ASCII, which should be safe based on the RFC. Differential Revision: https://phab.mercurial-scm.org/D1110

wireproto: more strkwargs cleanup Differential Revision: https://phab.mercurial-scm.org/D1109

hgweb: more "http headers are native strs" cleanup Differential Revision: https://phab.mercurial-scm.org/D1108

wireproto: bounce kwargs to/from bytes/str as needed Differential Revision: https://phab.mercurial-scm.org/D1107

configitems: fixup default value of annotate config option It turned out that configbool is used for most of them so the registered default value was wrong.

transaction-summary: use a revset to filter obsoletes in reportnewcs() Thanks to Yuya Nishihara for suggesting.

releasenotes: add pager support when printing to the ui Differential Revision: https://phab.mercurial-scm.org/D1098

configitems: fixup default value of diff config option It turned out that configbool is used for most of them so the registered default value was wrong.

devel-warn: add 'warn-' to 'devel.empty-changegroup' config Rename 'devel.empty-changegroup' to 'devel.warn-empty- changegroup' in order to clarify it controls a warning message. No alias is installed since the previous configuration option was not documented.

peer: ensure command names are always ascii bytestrs Differential Revision: https://phab.mercurial-scm.org/D1104

httppeer: always produce native str header keys and values Differential Revision: https://phab.mercurial-scm.org/D1103

wireproto: use %d to encode int, not %s Differential Revision: https://phab.mercurial-scm.org/D1102

wireproto: use a proper exception instead of `assert False` Differential Revision: https://phab.mercurial-scm.org/D1101

wireproto: use listcomp instead of map() The latter returns a generator object on Python 3, which breaks various parts of hg that expected a list. Differential Revision: https://phab.mercurial-scm.org/D1100

dagutil: use a listcomp instead of a map() In Python 3, the map() returns a generator object instead of a list, and some parts of hg depend on this being consumable more than once or sortable in place. Differential Revision: https://phab.mercurial-scm.org/D1099

peer: when collecting method names for batch calls, bytes-ify __name__ This will explode violently if we have a non-ascii command name. That shouldn't ever happen in core, and seems unlikely even in third-party code. Regardless, it'll explode violently, so we can revisit things in the future if we need to change the encoding here. Differential Revision: https://phab.mercurial-scm.org/D1092

python3: replace im_{self,func} with __{self,func}__ globally These new names are portable back to Python 2.6. Differential Revision: https://phab.mercurial-scm.org/D1091

httppeer: extract content-type from headers using native str Differential Revision: https://phab.mercurial-scm.org/D1090

httppeer: convert request url back to bytes before inspecting it Differential Revision: https://phab.mercurial-scm.org/D1089

templater: explode if we try to emit a str Without this if branch, we infinitely recurse in _flatten, which is very confusing. Something in an hgweb template is trying to write out a string instead of a bytes on Python 3, and this at least makes it crash politely. Differential Revision: https://phab.mercurial-scm.org/D1088

hgweb: fill in content-type and content-length as native strings This lets me actually get a capabilities response from hgweb over http. Differential Revision: https://phab.mercurial-scm.org/D1087

hgweb: mimetype guessing needs a unicode path Differential Revision: https://phab.mercurial-scm.org/D1086

hgweb: set sent_headers attr as early as practical While doing Python 3 porting work, I've seen exceptions happen in parts of hgweb we normally assume are robust. It won't hurt anything to set this attribute significantly earlier, so let's do so and save confusing during the porting process. Differential Revision: https://phab.mercurial-scm.org/D1085

hgweb: detect Python 3-era libraries and use modern attribute names Differential Revision: https://phab.mercurial-scm.org/D1084

server: indent block that's about to get conditionalized Differential Revision: https://phab.mercurial-scm.org/D1083

templates: fix missed space between instability labels

templates: introduce labelcset() function in map-cmdline.default as example

templater: load aliases from [templatealias] section in map file This seems sometimes useful as an alias can be a function, but a template fragment can't.

templater: load template fragments from [templates] section in map file This allows us to %include map-cmdline.<style> file in our .hgrc files. The syntax is slightly different as hgrc doesn't support loading an external template file, but map-cmdline files don't use this feature, so the syntax can be considered identical in practice. Unnamed section is remapped for backward compatibility.

config: allow remapping the default section The next patch depends on it. It doesn't make sense that the default section can't be remapped with {'': whatever}.

templater: simplify merge of __base__ dicts by reading it first

phase: add a dedicated pretxnclose-phase hook This new hook mirror the newly introduced 'txnclose-phase' but can abort the transaction.

phase: add a dedicated txnclose-phase hook The new 'txnclose-phase' hook expose the phase movement information stored in 'tr.changes['phases]'. To provide a simple and straightforward hook API to the users, we introduce a new hook called for each revision affected. Since a transaction can affect the phase of multiple changesets, updating the existing 'txnclose' hook to expose that information would be more complex. The data for all moves will not fit in environment variables and iterations over each move would be cumbersome. So the introduction of a new dedicated hook is preferred in this changesets. This does not exclude the addition of the full phase movement information to the existing 'txnclose' in the future to help write more complex hooks.

bookmark: add a dedicated pretxnclose-bookmark hook This new hook mirror the newly introduced 'txnclose-bookmark' but can abort the transaction.

bookmark: add a dedicated txnclose-bookmark hook The new 'txnclose-bookmark' hook expose the bookmark movement information stored in 'tr.changes['bookmarks]'. To provide a simple and straightforward hook API to the users, we introduce a new hook called for each bookmark touched. Since a transaction can affect multiple bookmarks, updating the existing 'txnclose' hook to expose that information would be more complex. The data for all moves might not fit in environment variables and iterations over each move would be cumbersome. So the introduction of a new dedicated hook is preferred in this changeset. This does not exclude the addition to the full bookmark information to the existing 'txnclose' in the future to help write more complex hooks.

tweakdefaults: make commands.update.check be `noconflict` This is the second-safest option we have to offer in `updatecheck`, with `abort` being the safest and `linear` being the default. At the sprint we discussed how much `none` and `linear` make us all uncomfortable, and how we'd like to move the default behavior if we can. I'm not sure we can get away with actually changing the out of the box default behavior, but we can at *least* do this. Differential Revision: https://phab.mercurial-scm.org/D1062

hgweb: fix logging to use native strings as appropriate Kind of a tangled mess, but now logging works in both Python 2 and 3. # no-check-commit because of the interface required by Python's HTTP server code. Differential Revision: https://phab.mercurial-scm.org/D1080

config: graduate experimental.updatecheck to commands.update.check .. feature:: New `commands.update.check` feature to adjust constraints on when `hg update` will allow updates with a dirty working copy. also .. bc:: The `experimental.updatecheck` name for the new `commands.update.check` feature is now deprecated, and will be removed after this release. Differential Revision: https://phab.mercurial-scm.org/D1070

hgweb: rewrite most obviously-native-strings to be native strings This clearly won't be everything, but it unblocks a fair amount of stuff here. Differential Revision: https://phab.mercurial-scm.org/D1079

hgweb: use native strings consistently for querystring parsing Differential Revision: https://phab.mercurial-scm.org/D1078

hgweb: rewrite two X or Y and Z ad-hoc ternaries with Y if X else Z Just easier to muddle through for my brain now that I don't see the old pattern much anymore. Differential Revision: https://phab.mercurial-scm.org/D1077

httppeer: use native strings for headers On Python 3, we need to use unicodes, rather than bytes. This lets test-pull.t get a lot further along. Differential Revision: https://phab.mercurial-scm.org/D887

hgweb: more native string treatment in query string parsing Differential Revision: https://phab.mercurial-scm.org/D1076

httppeer: pass url to urllib as native str, not bytes Differential Revision: https://phab.mercurial-scm.org/D1075

tersestatus: rework dirnode and tersedir docstrings Follow-up on refactorings 3d6d4b12128e and 5d98674df18a of the original changeset 7e3001b74ab3 by updating the docstrings of dirnode class and tersedir function: * rewrite dirnode.iterfilepaths()'s docstring (the method got renamed and reimplemented in 5d98674df18a); * simplify and update dirnode.tersewalk() to remove reference to 'self' and 'tersedict'; * use the imperative form of verbs in the first sentence of all docstrings. Differential Revision: https://phab.mercurial-scm.org/D1072

bitmanipulation: reformat with clang-format Mostly un-wrapping over-wrapped definitions. Differential Revision: https://phab.mercurial-scm.org/D1069

hghave: add a check for clang-format Differential Revision: https://phab.mercurial-scm.org/D1068

python3: use our bytes-only version of cgi.escape everywhere As suggested by Yuya in D965. Differential Revision: https://phab.mercurial-scm.org/D1067

url: add cgi.escape equivalent for bytestrings This seems like a sensible enough place to put it. Differential Revision: https://phab.mercurial-scm.org/D1066

python3: move from using func_name to __name__ Previously reviewed as D964, but required some fixups and therefore seems to need a new revision. Differential Revision: https://phab.mercurial-scm.org/D1065

tests: add globs for Windows

test-conflicts: conditionalize for no-symlink platforms It seems better to create standin files for the symlinks, rather than blacklist the entire tests. Especially since link vs file doesn't seem to affect the tests. There is more instability in test-pathconflicts-{basic,merge}.t that I can't figure out.

test-extdata: conditonalize error message for Windows

test-audit-path: fill in missing output for no-symlink case It looks like this was meant to go with 989e884d1be9.

test-i18n: guard gettext test appropriately

hook: add a 'hashook' function to test for hook existence Preparing the data for some hooks can be expensive. Add a function to check if a hook exists so we can skip useless preparation if no hook is configured.

build: for the bootstrap phase of a deb/rpm build pure-py mercurial is enough When bootstrapping a deb/rpm build, packagelib.sh starts performing a local build for the sole purpose of parsing the output of "hg version". Then it "hg archive"s the source code, and builds everything again. For that initial step, we are perfectly good in using a pure python mercurial, without compiling the c modules (base85, bdiff, zstdlib, ...). On my personal system, this cuts down 22 seconds for a package build (the bootstrapping build goes from ~30 to ~8 seconds).

context: add a fast-comparision for arbitraryfilectx and workingfilectx Differential Revision: https://phab.mercurial-scm.org/D1056

tersestatus: avoid modifying tersedict Turn dirnode's methods into generators which can be used to update "tersedict" in caller. So instead of passing the "tersedict" to be mutated here and there, it's now clearer where it is updated as it's purely a local variable to tersedir() function. While I was here, I renamed _processtersestatus to tersewalk and _addfilestotersed to iterfilepaths. Differential Revision: https://phab.mercurial-scm.org/D1043

tersestatus: make methods part of the dirnode class Differential Revision: https://phab.mercurial-scm.org/D1042

tersestatus: re-implement the functionality to terse the status The previous terse status implementation was hacking around os.listdir() and was flaky. There have been a lot of instances of mercurial buildbots failing and google's internal builds failing because of the hacky implementation of terse status. Even though I wrote the last implementation but it was hard for me to find the reason for the flake. The new implementation can be slower than the old one but is clean and easy to understand. In this we create a node object for each directory and create a tree like structure starting from the root of the working copy. While building the tree like structure we store some information on the nodes which will be helpful for deciding later whether we can terse the dir or not. Once the whole tree is build we traverse and built the list of files for each status with required tersing. There is no behaviour change as the old test, test-status-terse.t passes with the new implementation. Differential Revision: https://phab.mercurial-scm.org/D985

context: add is `isinmemory()` to filectx This will make it easier to skip certain behavior when running an in-memory merge. Differential Revision: https://phab.mercurial-scm.org/D1061

merge: ensure that we always commit the mergestate In future patches, we may halt the merge process based on configuration or user requests by raising exceptions. We need to ensure that the mergestate is unconditionally committed even when such an exception is raised. Depends on D930. Differential Revision: https://phab.mercurial-scm.org/D931

merge: add tests to show current behavior on failed filemerges Currently, failed file merges will nevertheless force the user to continue attempting to merge all additional unresolved files. Future patches will allow the user to halt the merge process instead. This patch first introduces a test demonstrating the current bejhavior so the upcoming changes are more obvious. Differential Revision: https://phab.mercurial-scm.org/D930

dirstate: move the _dirfoldmap to dirstatemap Now that dirstatemap is the source of truth for the list of directories, let's move _dirfoldmap on to it. This pattern of moving cached variables onto the dirstate map makes it easier to invalidate them, as seen by how the cache invalidation functions are slowly shrinking to just be recreating the dirstatemap instance. Differential Revision: https://phab.mercurial-scm.org/D983

dirstate: remove _dirs property cache Now that dirs is source of truthed on the dirstatemap, let's get rid of the _dirs propertycache on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D982

dirstate: remove _filefoldmap property cache Now that the filefoldmap is source of truthed on the dirstatemap, let's get rid of the property cache on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D981

dirstate: move identity to dirstatemap Moving the identity function to the dirstatemap class will allow alternative dirstate implementations to replace the implementation. Differential Revision: https://phab.mercurial-scm.org/D980

dirstate: move nonnormal and otherparent sets to dirstatemap As part of separating dirstate business logic from storage, let's move the nonnormal and otherparent storage to the dirstatemap class. This will allow alternative dirstate storage to persist these sets instead of recomputing them. Differential Revision: https://phab.mercurial-scm.org/D979

dirstate: move write into dirstatemap As part of separating the dirstate business logic from the dirstate storage logic, let's move the serialization code down into dirstatemap. Differential Revision: https://phab.mercurial-scm.org/D978

dirstate: move _read into dirstatemap As part of separating the dirstate business logic from the storage, let's move the read code into the new dirstatemap class. Differential Revision: https://phab.mercurial-scm.org/D977

configitems: register the 'templates' section

configitems: register the 'paths' config section

configitems: register the 'pager.attend-.*' options

configitems: register the 'hooks' config section

configitems: register the 'extensions' section

configitems: register the 'defaults' section

configitems: register the 'committemplate' section

configitems: register the 'color' section