stringutil: edit comment to reflect actual data type name In development the data type used to hold an email/name pair was called a "mailmaptup" since it was implemented as a namedtuple. The implementation has since been changed to use an @attr.s decorated class named mailmapping. This commit changes a comment to reflect this change. Differential Revision: https://phab.mercurial-scm.org/D3004

stringutil: improve check for failed mailmap line parsing The existing check for a bad mailmap file entry fails with inputs like b'>@<'. This commit adds a function to check if a sufficient amount of information has been parsed from a mailmap file entry. At minimum, one email must be found (assumed to be the commit email). If email is not empty and no names are found, then there must be two emails. If there are at least one email and name, the mapping is valid. Differential Revision: https://phab.mercurial-scm.org/D3003

stringutil: rename local email/names variables to their plural forms email and name variables are renamed to emails and names (respectively). This is because the email variable name shadows the email function within the stringutil module. Since we are renaming email, we also rename name for consistency. Differential Revision: https://phab.mercurial-scm.org/D3002

templatefuncs: remove redundant "or author" from mailmap return statement Differential Revision: https://phab.mercurial-scm.org/D3001

lfs: add the 'Content-Type' header called out in the file transfer spec https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md#uploads

lfs: improve the client message when the server signals an object error Two things here. First, the previous message included a snippet of JSON, which tends to be long (and in the case of lfs-test-server, has no error message). Instead, give a concise message where possible, and leave the JSON to a debug output. Second, the server can signal issues other than a missing individual file. This change shows a corrupt file, but I'm debating letting the corrupt file get downloaded, because 1) the error code doesn't really fit, and 2) having it locally makes forensics easier. Maybe need a config knob for that.

check-code: tighten the check for `ls -R` Otherwise, this was flagging `... lfs.serve=False -R server ...` in the tests.

infinitepush: add tests for `hg pull -r <rev>` This patch adds test for `hg pull -r <rev>` when the infinitepush extension is not present on the client side and the server by defaults pushes all the incoming push to bundlestore. As the tests show, if a the changeset was applied to the server, that can be pulled, but if a changeset is in the bundlestore we cannot pull that yet. We should support that. Differential Revision: https://phab.mercurial-scm.org/D2998

infinitepush: use bundleoperation.source instead of hacking on tr Previous patches added a soutce attribute to bundle2.bundleoperation class which stores the command which leads to current bundleoperation. Let's use that to decide whether a processing a `hg unbundle` or not instead of hacking on transaction. Differential Revision: https://phab.mercurial-scm.org/D2997

bundle2: make source a mandatory argument for bundle2.applybundle() (API) Currently all the callers in the core pass the source argument, making it mandatory will help us storing right source value in bundleoperation() class. Differential Revision: https://phab.mercurial-scm.org/D2996

bundleoperation: pass the source argument from all the users We now have a source attribute to the bundle2.bundleoperation class which stores the operation which leads to the current bundling/unbundling. Let's make sure we pass source argument from all the users of the command. Differential Revision: https://phab.mercurial-scm.org/D2995

bunlde2: add 'source' as an optional argument to processbundle() This will help us to pass the source variable to bundleoperation class. Differential Revision: https://phab.mercurial-scm.org/D2994

bundle2: add 'source' atrribute to bundleoperation class This will help us in easily finding out which command leads to the current operation without hacking on the transaction. Differential Revision: https://phab.mercurial-scm.org/D2993

infinitepush: don't force ipv6 while connecting to mysql server Facebook internally enforces this but looks like we can't force this for pur users. Differential Revision: https://phab.mercurial-scm.org/D2992

infinitepush: drop the default value of config options which are registered Differential Revision: https://phab.mercurial-scm.org/D2991

infinitepush: replace `remotenames.hoist` with `remotenames.hoistedpeer` The remotenames.hoist config option was renamed to remotenames.hoistedpeer while moving to core as an extension. Let's start using the config option provided by the in-core extension. Differential Revision: https://phab.mercurial-scm.org/D2990

templater: drop global exception catcher from runfilter() (API) Now all built-in template filters declare input data types or handle type errors by themselves. .. api:: Template filters should declare input data type and/or catch AttributeError, ValueError, TypeError, etc. as needed. See the doc of ``registrar.templatefilters`` for details.

templatefilters: raise ProgrammingError if unencodable type passed to json() This shouldn't happen for any template data types (though I know it does because of some templater bugs.) Let's clarify it is a bug.

templatefilters: handle TypeError by count() Prepares for removing the weird exception catcher from runfilter().

keyword: declare input type of date filters as date Dropped redundant dateutil.parsedate(), which is now handled by the templater.

templatefilters: declare input type as date where appropriate I'm not sure if the templateutil.date type can be a thing. Currently it's just a constant.

templater: attach hint to input-type error of runfilter() Tests will be added by the next patch.

templatefuncs: use evaldate() where seems appropriate This means date("today") is allowed. Also fixes evaldate() to forcibly use the custom error message if specified.

templater: factor out function that parses argument as date tuple

templatefilters: declare input type as bytes where appropriate Some test outputs changed since input is now coerced to a byte string. I think that's okay. Maybe {date} should have some readable representation?

templatefilters: allow declaration of input data type Currently filters take an unwrapped value, which should have no hybrid magic but actually it does because stringify() relies on it. The 'intype' allows us to pre-process the magic by .e.g. evalstring() keeping filter functions as simple as they are now. stringify() is ported as an example. More follow.

hgweb: use registrar to declare "websub" template filter Prepares for adding 'intype' attribute.

procutil: redirect ui.fout to stderr while stdio is protected The new behavior seems slightly nicer as we can at least read the output. And this is similar to what the sshserver is doing, so we can probably reuse protectstdio() instead of the weird hook.redirect(True) hack.

procutil: unroll uin/uout loop in protectstdio() I'll change uout to be redirected to stderr.

context: drop support for changeid of type long (API?) I don't see a reason to support type long. It's pretty much the same type as int. There was some discussion about it on the mailing list around the time of ff2f90503d64 (context: work around `long` not existing on Python 3, 2017-03-11), but I couldn't find a good reason to keep it. There was some mention of hgtk doing "repo[long(rev)]", but that was in 2012. Differential Revision: https://phab.mercurial-scm.org/D2989

lfs: drop a duplicate blob verification method

server: minor code cleanup Suggested by Yuya after 77f9e95fe3c4, this is mostly using named values for stdio descriptors. The lockfile is also opened in binary mode when reading back content from the child.

server: refactor 'daemon_postexec' instructions into a dictionary

cbor: import CBORDecoder and CBOREncoder And format the imports so it is cleaner. Differential Revision: https://phab.mercurial-scm.org/D2978

py3: fix fix doctests to be bytes-safe

server: add an error feedback mechanism for when the daemon fails to launch There's a recurring problem on Windows where `hg serve -d` will randomly fail to spawn a detached process. The reason for the failure is completely hidden, and it takes hours to get a single failure on my laptop. All this does is redirect stdout/stderr of the child to a file until the lock file is freed, and then the parent dumps it out if it fails to spawn. I chose to put the output into the lock file because that is always cleaned up. There's no way to report errors after that anyway. On Windows, killdaemons.py is roughly `kill -9`, so this ensures that junk won't pile up. This may end up being a case of EADDRINUSE. At least that's what I saw spit out a few times (among other odd errors and missing output on Windows). But I also managed to get the same thing on Fedora 26 by running test-hgwebdir.t with --loop -j10 for several hours. Running `netstat` immediately after killing that run printed a wall of sockets in the TIME_WAIT state, which were gone a couple seconds later. I couldn't match up ports that failed, because --loop doesn't print out the message about the port that was used. So maybe the fix is to rotate the use of HGPORT[12] in the tests. But, let's collect some more data first.

tests: conditionalize test-stream-bundle-v2 for Windows

templatefuncs: add mailmap template function This commit adds a template function to support the .mailmap file in Mercurial repositories. The .mailmap file comes from git, and can be used to map new emails and names for old commits. The general use case is that someone may change their name or author commits under different emails and aliases, which would make these commits appear as though they came from different persons. The file allows you to specify the correct name that should be used in place of the author field specified in the commit. The mailmap file has 4 possible formats used to map old "commit" names to new "proper" names: 1. <proper@email.com> <commit@email.com> 2. Proper Name <commit@email.com> 3. Proper Name <proper@email.com> <commit@email.com> 4. Proper Name <proper@email.com> Commit Name <commit@email.com> Essentially there is a commit email present in each mailmap entry, that maps to either an updated name, email, or both. The final possible format allows commits authored by a person who used both an old name and an old email to map to a new name and email. To parse the file, we split by spaces and build a name out of every element that does not start with "<". Once we find an element that does start with "<" we concatenate all the name elements that preceded and add that as a parsed name. We then add the email as the first parsed email. We repeat the process until the end of the line, or a comment is found. We will be left with all parsed names in a list, and all parsed emails in a list, with the 0 index being the proper values and the 1 index being the commit values (if they were specified in the entry). The commit values are added as the keys to a dict, and with the proper fields as the values. The mapname function takes the mapping object and the commit author field and attempts to look for a corresponding entry. To do so we try (commit name, commit email) first, and if no results are returned then (None, commit email) is also looked up. This is due to format 4 from above, where someone may have a mailmap entry with both name and email, and if they don't it is possible they have an entry that uses only the commit email. Differential Revision: https://phab.mercurial-scm.org/D2904

extdiff: document that it copies modified files back to working directory Differential Revision: https://phab.mercurial-scm.org/D2976

zope: force module import by importing symbols Previously, we tried to import a module and handle the ImportError. Our lazy module importer doesn't verify the module exists and returns a dummy object representing the module. Only once we attempt to load a symbol in the module does the ImportError get raises. This means that simple imports inside `try..except ImportError` don't detect missing modules. This commit changes imports in zope.interface to access symbols, thus forcing module load and triggering ImportError. This fixes zope.interface for pure builds. Differential Revision: https://phab.mercurial-scm.org/D2980

fix: remove unused imports Differential Revision: https://phab.mercurial-scm.org/D2977

infinitepush: introduce server option to route every push to bundlestore This patch introduces a new config option for server `infinitepush.pushtobundlestore` which if sets to True, the server will route each incoming push to the bundlestore and store all the parts i.e. changegroups, phases, obsmarkers in the bundlestore and won't be applied to the revlog. This config option does not need any client side wrapping and does not need any custom bundle2 part or stream level parameter to decide where the push should go. This is very useful for Mozilla CI use case where they have a central server that recieves pushes to trigger code-reviews, trigger a test run of CI, run static analysis etc. The server using the new config option can stash standalone bundles to the bundlestore and server can get access to individual revisions on demand. A new test file which has related tests are added for the config option. Differential Revision: https://phab.mercurial-scm.org/D2958

infinitepush: don't wrap bundle2.processparts while calling `hg unbundle` This patch adds dirty logic to check whether we are processing `hg unbundle` instead of an `hg incoming` to prevent the wrapping of bundle2.processparts function. Differential Revision: https://phab.mercurial-scm.org/D2957

inifnitepush: use utils.{stringutil|procutil}.* instead of util.* Recently in core, util.py is splitted into various modules in mercurial/utils/. Differential Revision: https://phab.mercurial-scm.org/D2956

infinitepush: delete the non-forward-move flag for hg push Differential Revision: https://phab.mercurial-scm.org/D2955

infinitepush: drop the wrapping of update command on client side The extension wraps update command to client side to add functionality to pull and checkout to a revision if it does not present locally. There is a twist to that, only changesets which can resolve to remotenames can be pulled using this. Differential Revision: https://phab.mercurial-scm.org/D2954

infinitepush: delete infinitepushcommands.py and related tests This patch drops infinitepushcommands.py and the tests which were related to the command `debugfillinfinitepushmetadata` introduced in the commit. The patch also drops a config option which was related to debuginfinitepushmetadata command. Differential Revision: https://phab.mercurial-scm.org/D2953

infinitepush: delete infinitepush.fillmetadatabranchpattern config option This patch deletes the config option infinitepush.fillmetadatabranchpattern which if set to true sets a background process which will save metadata in infinitepush index. This series is meant to have a state where we can use it for CI purposes. Differential Revision: https://phab.mercurial-scm.org/D2952