show: pass the minimum length for nodes as a template keyword This will allow us to make the displayed length configurable and/or dynamic. Differential Revision: https://phab.mercurial-scm.org/D556

cmdutil: allow extra properties to be added to each context The changeset displayer allows setting extra keywords to be available to the templating layer. This patch adds an argument to displaygraph() to pass a dict of extra properties to be available to every changeset. Differential Revision: https://phab.mercurial-scm.org/D555

dirstate: perform transactions with _map using single call, where possible This is in the same style as https://phab.mercurial-scm.org/D493. In general, this replaces patterns such as: ``` f in self._map: entry = self._map[f] ``` with: ``` entry = self._map.get(f): if entry is not None: # use entry ``` Test Plan: `make tests` Differential Revision: https://phab.mercurial-scm.org/D663

extensions: register config item early Config items are likely to be used in during extensions setup. So we much register them before that. For example this apply to the 'win32text.warn' options.

extensions: factor extra data loading out Some of the extra data need to be registered earlier than they currently are (eg: config items). We first factor out the logic to registered them in a small function before reusing it in the next changeset.

configitems: register the 'win32text.warn' config

configitems: register the 'mq.secret' config

configitems: register the 'mq.plain' config

configitems: register the 'mq.keepchanges' config

configitems: register the 'mq.git' config

configitems: register the 'win32mbcs.encoding' config

copytrace: move fast heuristic copytracing algorithm to core copytrace extension in fb-hgext has a heuristic implementation of copy tracing which is faster than the current copy tracing. The heuristic limits the search of copies to just files that are either: 1) Renames in the same directory 2) Moved to other directory with same name The default copytrace implementation is very slow as it finds all the new files that were added from merge base up to the head commit and for each file it checks whether it this was copied or moved version of a different file. Stash@fb did analysis for the above heuristics on the fb repo and found that among 2,443,768 moves/copies there are only 32,234 moves/copies which does not fall under the above heuristics which is approx. 0.013 of total copies. This patch moves the heuristics algorithm under config `experimental.copytrace=heuristics`. While moving fbext to core, this patch removes couple of less useful config options named `sourcecommitlimit` and `maxmovescandidatestocheck`. Tests are also added for the heuristics algorithm, which are basically copied from fbext/tests/test-copytrace.t. The tests follow a pattern creating a server repo and then cloning to a local repo to create public and draft changesets, the distinction which will be useful in upcoming patches. After this patch `experimental.copytrace` has the following behaviour: 1) `off`: turns off copytracing 2) `heuristics`: use the heuristic algorithm added in this patch. 3) everything else: use the full copytracing algorithm .. feature:: A new fast heuristic algorithm for copytracing which assumes that the files moves are either:: 1) Renames in the same directory 2) Moves in other directories with same names You can use this algorithm by setting `experimental.copytrace=heuristics`. Differential Revision: https://phab.mercurial-scm.org/D623

configitems: register the 'convert.svn.startrev' config

configitems: register the 'convert.svn.debugsvnlog' config

configitems: register the 'convert.skiptags' config

configitems: register the 'convert.p4.startrev' config

configitems: register the 'convert.localtimezone' config

configitems: register the 'convert.ignoreancestorcheck' config

configitems: register the 'convert.hg.usebranchnames' config

configitems: register the 'convert.hg.tagsbranch' config

configitems: register the 'convert.hg.startrev' config

configitems: register the 'convert.hg.sourcename' config

configitems: register the 'convert.hg.saverev' config

configitems: register the 'convert.hg.revs' config

configitems: register the 'convert.hg.ignoreerrors' config

configitems: register the 'convert.hg.clonebranches' config

configitems: register the 'convert.git.skipsubmodules' config

configitems: register the 'convert.git.similarity' config

configitems: register the 'convert.git.saverev' config

configitems: register the 'convert.git.renamelimit' config

configitems: register the 'convert.git.remoteprefix' config

configitems: register the 'convert.git.findcopiesharder' config

configitems: register the 'convert.git.extrakeys' config

configitems: register the 'convert.git.committeractions' config

configitems: register the 'convert.cvsps.mergeto' config

configitems: register the 'convert.cvsps.mergefrom' config

configitems: register the 'convert.cvsps.fuzz' config

configitems: register the 'convert.cvsps.cache' config The convert extensions has code a bit all around, but it seems simpler to use a central declaration of options at the root first.

bundle2: move exception handling into part iterator As part of separating the part iteration logic from the part handling logic, let's move the exception handling to the part iterator class. Differential Revision: https://phab.mercurial-scm.org/D705

bundle2: move part counter to partiterator As part of moving the part iterator logic to a separate class, let's move the part counting logic and the output for it. Differential Revision: https://phab.mercurial-scm.org/D704

bundle2: move part iterator a separate class Currently, the part iterator logic is tightly coupled with the part handling logic, which means it's hard to replace the part handling logic without duplicating the part iterator bits. In a future diff we'll want to be able to replace all part handling, so let's begin refactoring the part iterator logic to it's own class. Differential Revision: https://phab.mercurial-scm.org/D703

changegroup: add source parameter to generatemanifests Extensions, like remotefilelog, will want to look at the source of a pull when determining what manifests to add to a changegroup. For instance, on push they will include everything, while on pull they won't. Differential Revision: https://phab.mercurial-scm.org/D686

changegroup: remove changegroup dependency from revlog.addgroup Previously revlog.addgroup would accept a changegroup and a linkmapper and use it to iterate of the deltas. As part of untangling the revlog-changegroup interdependency, let's move the changegroup delta iteration logic to it's own function and pass the simple iterator to the revlog instead. This will make it easier to introduce non-revlogs stores in the future, without reinventing any changegroup specific logic. Differential Revision: https://phab.mercurial-scm.org/D688

revlog: refactor chain variable Previously the addgroup loop would set chain to be the result of self._addrevision(node,...). Since _addrevision now always returns the passed in node, we can drop that behavior and just always set chain = node in the loop. This will be useful in a future patch where we refactor the cg.deltachunk logic to another function and therefore chain disappears entirely from this function. Differential Revision: https://phab.mercurial-scm.org/D699

scmutil: don't append .orig to backups in origbackuppath (BC) When ui.origbackuppath is set, .orig files are stored outside of the working copy, however they still have a .orig suffix appended to them. This can cause unexpected conflicts, particularly when tracked files or directories have .orig at the end. This change removes the .orig suffix from files stored in an out-of-tree origbackuppath. Test Plan: Update and run unit tests. Differential Revision: https://phab.mercurial-scm.org/D679

merge: move cwd-missing detection to helper functions This will exist in two places with defered writes, so we want to avoid duplication. Differential Revision: https://phab.mercurial-scm.org/D626

doctest: enable tests by default on Python 3 Still several tests fail mostly because of the string issues, sigh. I'll fix them one by one.

doctest: normalize b'', u'' and exception output on Python 3 The idea is described in the following page. https://dirkjan.ochtman.nl/writing/2014/07/06/single-source-python-23-doctests.html # no-check-commit

doctest: coerce dict.keys() to list Otherwise it would be printed as odict_keys([...]) on Python 3.

doctest: upgrade old-style "except" clause

doctest: use print_function and convert bytes to unicode where needed

doctest: do not embed non-ascii characters in docstring Since the outer docstring is parsed as a unicode on Python 3, we have to either double-escape or construct non-ascii string from ascii string.

doctest: pass encoding name as system string

doctest: replace str() with bytes()

doctest: replace chr() with pycompat.bytechr()

doctest: replace .iteritems() with .items()

doctest: bulk-replace string literals with b'' for Python 3 Our code transformer can't rewrite string literals in docstrings, and I don't want to make the transformer more complex.

debuginstall: do not pass exception object to formatter (issue5676)

debuginstall: use codecs.lookup() to detect invalid encoding encoding.fromlocal() never tries to decode an ascii string since 853574db5b12, and there's no universal non-ascii string which can be decoded as any valid character set.

extensions: fix wrapcommand/function of class instance 5361771f9714 changed _updatewrapper() to copy the __name__ attribute, but not all callable objects has __name__. Spotted by loading mq with extdiff.

changegroup: avoid creating empty changegroup part Previously this check happened in the changegroup code itself. Since its refactor, this logic needs to move out to callers that care about it, such as this one. Otherwise we get empty bundle devel-warnings in certain extensions. Differential Revision: https://phab.mercurial-scm.org/D690

tests: split test-revset.t in half This test has gotten so large that running it can exceed the normal timeout on systems under load (like if we're running all the tests in parallel). This patch splits the test cleanly in half. Differential Revision: https://phab.mercurial-scm.org/D694

merge: flush any deferred writes just before recordupdates() ``recordupdates`` calls into the dirstate which requires the files to be there, so this is the last possible moment we can flush anything. Differential Revision: https://phab.mercurial-scm.org/D673

merge: flush any deferred writes before, and after, running any workers Since we fork to create workers, any changes they queue up will be lost after the worker terminates, so the easiest solution is to have each worker flush the writes they accumulate--we are close to the end of the merge in any case. To prevent duplicated writes, we also have the master processs flush before forking. In an in-memory merge (M2), we'll instead disable the use of workers. Differential Revision: https://phab.mercurial-scm.org/D628

filemerge: flush if using deferred writes when running a merge tool Since merge tools might read from the filesystem, we need to write out our deferred writes here. No-ops if not using deferred writes. Differential Revision: https://phab.mercurial-scm.org/D627

merge: pass wctx to premerge, filemerge In the in-memory merge branch. we'll need to call a function (``flushall``) on the wctx inside of _xmerge. This prepares the way so it can be done without hacks like ``fcd.ctx()``. Differential Revision: https://phab.mercurial-scm.org/D449

cmdutil: fix amend when passing a date Following https://phab.mercurial-scm.org/D636, passing the same date that the changeset to amend would results in no new commits but the output changed from: $ hg amend -d '0 0' nothing changed [1] to: $ hg amend -d '0 0' Restore the old behavior by parsing the date passed as parameter so the condition "date == old.date()" correctly works in cases both dates are identical. Add a test for covering this regression. This bug was found thanks to Evolve test suite. Differential Revision: https://phab.mercurial-scm.org/D691

merge with stable

configitems: register the 'eol.only-consistent' config

configitems: register the 'eol.native' config

configitems: register the 'eol.fix-trailing-newline' config

configitems: register the 'patchbomb.publicurl' config

configitems: register the 'patchbomb.intro' config

configitems: register the 'patchbomb.from' config

configitems: register the 'patchbomb.flagtemplate' config

configitems: register the 'patchbomb.confirm' config

configitems: register the 'patchbomb.bundletype' config

blackbox: remove _bbvfs state `_bbvfs` is redundant because it could be calcualted from `_bbrepo`. Differential Revision: https://phab.mercurial-scm.org/D651

blackbox: do not cache file objects Having the blackbox file objects cached in `ui._bbfp` could in theory be troublesome if multiple processes (ex. chg servers) have file objects referring to a same file. (Although I spent some time and failed to build a convincing test case) This patch makes blackbox re-open the file every time to make the situation better. Ideally we also need proper locking. The caching logic traces back to the commit introducing blackbox (18242716a). That commit does not have details about why caching is necessary. Consider the fact that blackbox logs are not many, it seems fine to remove the fp cache to be more confident. Differential Revision: https://phab.mercurial-scm.org/D650

blackbox: inline _bbwrite There is no need to make it a separate method. This makes the next change easier to read. Differential Revision: https://phab.mercurial-scm.org/D649

blackbox: fix rotation with chg The added test will show: $ $PYTHON showsize.py .hg/blackbox* .hg/blackbox.log: < 500 .hg/blackbox.log.1: < 500 .hg/blackbox.log.2: < 500 .hg/blackbox.log.3: < 500 .hg/blackbox.log.4: < 500 .hg/blackbox.log.5: >= 500 with previous code. The issue is caused by blackbox caching file objects *by path*, and the rotation size check could run on a wrong file object (i.e. it should check "blackbox.log", but `filehandles["blackbox.log"]` contains a file object that has been renamed to "blackbox.log.5"). This patch removes the "filehandlers" global cache added by 45313f5a3a8c to solve the issue. I think the original patch was trying to make different ui objects use a same file object if their blackbox.log path is the same. In theory it could also be problematic in the rotation case. Anyway, that should become unnecessary after D650. Differential Revision: https://phab.mercurial-scm.org/D648

test-blackbox: make it compatible with chg Differential Revision: https://phab.mercurial-scm.org/D647

ssh: fix flakey ssh errors on BSD systems There's been a persistent issue with flakiness on BSD systems (like OSX) where the 'no suitable response from remote hg' message would sometimes not appear. This was caused by one of the earlier calls failing with a "IOError: Broken pipe". Catching those errors and printing the same message removes the flakiness. Differential Revision: https://phab.mercurial-scm.org/D687

context: add overlayworkingcontext and overlayworkingfilectx These two classes will be used extensively in the first in-memory merge milestone. Differential Revision: https://phab.mercurial-scm.org/D616

changegroup: rename getsubsetraw to makestream Now that nothing uses getsubsetraw except makestream, let's move the functionality into the makestream. This removes the last remaining excess changegroup creation function, getsubsetraw. Differential Revision: https://phab.mercurial-scm.org/D671

changegroup: remove external uses of getbundler Now that makestream and makechangegroup are the primary creation methods for changegroups, let's get rid of this rogue use of getbundler and getsubsetraw. Differential Revision: https://phab.mercurial-scm.org/D670

changegroup: replace getchangegroup with makechangegroup As part of reducing the number of changegroup creation APIs, let's replace getchangegroup with calls to makechangegroup. This is mostly a drop in replacement, but it does change the version specifier to be required, so it's more obvious which callers are creating old version 1 changegroups still. Differential Revision: https://phab.mercurial-scm.org/D669

changegroup: replace changegroup with makechangegroup As part of reducing the number of changegroup creation APIs, let's replace the changegroup function with makechangegroup. This pushes the responsibility of creating the outgoing set to the caller, but that seems like a simple and reasonable concept for the caller to be aware of. Differential Revision: https://phab.mercurial-scm.org/D668

changegroup: delete getlocalchangegroup As part of reducing the number of changegroup creation APIs, let's go ahead and delete this unused function. It appears to have been deprecated in the last release anyway. Differential Revision: https://phab.mercurial-scm.org/D667

changegroup: replace getlocalchangegroupraw with makestream As part of reducing the number of changegroup creation apis, let's replace calls to getlocalchangegroupraw with calls to makestream. Aside from one case of checking if there are no outgoing commits and returning None, this is pretty much a drop in replacement. Differential Revision: https://phab.mercurial-scm.org/D666

changegroup: replace changegroupsubset with makechangegroup As part of getting rid of all the permutations of changegroup creation, let's remove changegroupsubset and call makechangegroup instead. This moves the responsibility of creating the outgoing set to the caller, but that seems like a relatively reasonable unit of functionality for the caller to have to care about (i.e. what commits should be bundled). Differential Revision: https://phab.mercurial-scm.org/D665

changegroup: replace getsubset with makechangegroup The current changegroup APIs are a bit of a mess. Currently you can use getsubsetraw, getsubset, changegroupsubset, getlocalchangegroupraw, getchangegroup, and getlocalchangroup to produce changegroups. This patch is the beginning of a refactor to boil all of that away to just makechangegroup and makestream. The first step adds the new functions and replaces getsubset function with them. Differential Revision: https://phab.mercurial-scm.org/D664

rebase: remove unnecessary '.unfiltered()' calls Now we have a clear centric place to control whether `rbsrt.repo` is unfiltered or not, we can drop `unfiltered()` in other places. Differential Revision: https://phab.mercurial-scm.org/D645

rebase: remove complex unhiding code This is similar to Martin von Zweigbergk's previous patch [1]. Previous patches are adding more `.unfiltered()` to the rebase code. So I wonder: are we playing whack-a-mole regarding on `unfiltered()` in rebase? Thinking about it, I believe most of the rebase code *should* just use an unfiltered repo. The only exception is before we figuring out a `rebasestate`. This patch makes it so. See added comment in code for why that's more reasonable. This would make the code base cleaner (not mangling the `repo` object), faster (no need to invalidate caches), simpler (less LOC), less error-prone (no need to think about what to unhide, ex. should we unhide wdir p2? how about destinations?), and future proof (other code may change visibility in an unexpected way, ex. directaccess may make the destination only visible when it's in "--dest" revset tree). [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/094277.html Differential Revision: https://phab.mercurial-scm.org/D644

rebase: use unfiltered repo when loading state Before this patch, `rebase --abort` may fail to do the cleanup: $ hg rebase --abort rebase aborted (no revision is removed, only broken state is cleared) The added test case makes sure `--abort` works in this case. Differential Revision: https://phab.mercurial-scm.org/D643

rebase: do not crash rebasing merge with a parent having hidden successor The added test will crash with previous code. Differential Revision: https://phab.mercurial-scm.org/D640