httppeer: expose capabilities for each command This will help code using peers to sniff out exactly what servers support. Differential Revision: https://phab.mercurial-scm.org/D4436

narrow: intersect provided matcher with narrowmatcher in `hg diff` This provides significant speedups when running diff, and no change in behavior that I'm aware of (or that the tests found). I tested with a repo that I started using narrow in after it was created and attempted to run `hg diff -c .` and similar commands in it on a commit that had files not in the narrowspec. Timing numbers below, using a similar setup as my previous commits. before=9db85644, m-u is mozilla-unified at eb39298e432d (flatmanifest) and 0553b7f29eaf (treemanifest). l-d-r is a repo simulating a situation I've encountered where there's one directory with 30k+ subdirectories. N means narrow, T means treemanifest. The narrowspec is pretty small when in use, and importantly the narrowspec is applied *after* doing the initial checkout (without narrowing), so all of these files exist in the filesystem, which is not normally the case if someone has been using narrow for the entire life of the clone. Anything less than a 5% difference in performance is most likely noise. diff --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.292 s +- 0.009 s | 1.295 s +- 0.010 s | 100.2% m-u | | x | 1.296 s +- 0.042 s | 1.299 s +- 0.026 s | 100.2% m-u | x | | 1.292 s +- 0.010 s | 1.297 s +- 0.021 s | 100.4% m-u | x | x | 84.2 ms +- 1.2 ms | 83.6 ms +- 0.2 ms | 99.3% l-d-r | | | 188.7 ms +- 2.7 ms | 188.8 ms +- 2.0 ms | 100.1% l-d-r | | x | 189.9 ms +- 1.5 ms | 189.4 ms +- 1.2 ms | 99.7% l-d-r | x | | 97.1 ms +- 1.0 ms | 87.1 ms +- 1.0 ms | 89.7% <-- l-d-r | x | x | 96.9 ms +- 0.8 ms | 87.2 ms +- 0.7 ms | 90.0% <-- diff -c . --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 231.6 ms +- 3.1 ms | 228.9 ms +- 1.6 ms | 98.8% m-u | | x | 150.5 ms +- 1.7 ms | 150.7 ms +- 1.4 ms | 100.1% m-u | x | | 233.7 ms +- 2.4 ms | 232.2 ms +- 1.9 ms | 99.4% m-u | x | x | 126.1 ms +- 1.2 ms | 126.8 ms +- 1.2 ms | 100.6% l-d-r | | | 82.1 ms +- 2.0 ms | 81.8 ms +- 1.4 ms | 99.6% l-d-r | | x | 3.732 s +- 0.020 s | 3.746 s +- 0.027 s | 100.4% l-d-r | x | | 83.1 ms +- 0.8 ms | 107.6 ms +- 2.4 ms | 129.5% <-- l-d-r | x | x | 758.2 ms +- 38.8 ms | 188.5 ms +- 1.8 ms | 24.9% <-- rebase -r . --keep -d .^^: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 5.532 s +- 0.087 s | 5.496 s +- 0.016 s | 99.3% m-u | | x | 5.554 s +- 0.061 s | 5.532 s +- 0.013 s | 99.6% m-u | x | | 5.602 s +- 0.134 s | 5.508 s +- 0.035 s | 98.3% m-u | x | x | 582.2 ms +- 15.2 ms | 572.9 ms +- 12.0 ms | 98.4% l-d-r | | | 629.5 ms +- 12.3 ms | 622.5 ms +- 7.3 ms | 98.9% l-d-r | | x | 6.173 s +- 0.062 s | 6.185 s +- 0.076 s | 100.2% l-d-r | x | | 274.5 ms +- 10.0 ms | 272.1 ms +- 6.2 ms | 99.1% l-d-r | x | x | 4.835 s +- 0.056 s | 4.826 s +- 0.034 s | 99.8% status --change . --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 214.4 ms +- 1.4 ms | 212.2 ms +- 1.7 ms | 99.0% m-u | | x | 130.9 ms +- 1.2 ms | 131.7 ms +- 1.1 ms | 100.6% m-u | x | | 215.0 ms +- 2.1 ms | 214.9 ms +- 2.7 ms | 100.0% m-u | x | x | 109.5 ms +- 2.3 ms | 107.8 ms +- 0.9 ms | 98.4% l-d-r | | | 79.6 ms +- 0.9 ms | 79.8 ms +- 1.6 ms | 100.3% l-d-r | | x | 3.799 s +- 0.037 s | 3.928 s +- 0.021 s | 103.4% <--? l-d-r | x | | 82.7 ms +- 0.7 ms | 83.2 ms +- 1.0 ms | 100.6% l-d-r | x | x | 746.8 ms +- 6.1 ms | 739.0 ms +- 4.2 ms | 99.0% status --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.884 s +- 0.012 s | 1.885 s +- 0.013 s | 100.1% m-u | | x | 1.897 s +- 0.027 s | 1.909 s +- 0.077 s | 100.6% m-u | x | | 1.886 s +- 0.021 s | 1.891 s +- 0.030 s | 100.3% m-u | x | x | 92.0 ms +- 0.7 ms | 92.4 ms +- 0.4 ms | 100.4% l-d-r | | | 570.3 ms +- 18.7 ms | 552.2 ms +- 4.5 ms | 96.8% l-d-r | | x | 568.9 ms +- 16.1 ms | 567.2 ms +- 11.9 ms | 99.7% l-d-r | x | | 171.1 ms +- 2.5 ms | 170.4 ms +- 1.2 ms | 99.6% l-d-r | x | x | 171.6 ms +- 3.4 ms | 171.5 ms +- 1.7 ms | 99.9% update $rev^; ~/src/hg/hg{hg}/hg update $rev: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 3.107 s +- 0.017 s | 3.116 s +- 0.012 s | 100.3% m-u | | x | 2.943 s +- 0.010 s | 2.945 s +- 0.019 s | 100.1% m-u | x | | 3.116 s +- 0.033 s | 3.118 s +- 0.027 s | 100.1% m-u | x | x | 318.5 ms +- 2.7 ms | 320.8 ms +- 4.8 ms | 100.7% l-d-r | | | 428.9 ms +- 4.4 ms | 429.5 ms +- 4.0 ms | 100.1% l-d-r | | x | 9.593 s +- 0.081 s | 9.869 s +- 0.043 s | 102.9% l-d-r | x | | 253.2 ms +- 3.6 ms | 254.0 ms +- 2.8 ms | 100.3% l-d-r | x | x | 1.613 s +- 0.009 s | 1.630 s +- 0.017 s | 101.1% Differential Revision: https://phab.mercurial-scm.org/D4587

identify: change {parents} to a list of nodes (BC) This is a part of the name unification. {parents} is a list of nodes in "hg log -Tjson" output. Since {rev} can be computed from (repo, node) pair, we no longer need to put it to provide {rev} to user templates. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

identify: use fm.hexfunc thoroughly This fixes the length of {id} in JSON and template outputs.

formatter: replace contexthint() with demand loading of ctx object And pass in repo instead to resolve ctx from (repo, node) pair.

formatter: populate ctx from repo and node value This will basically replace the fm.contexthint() API. I originally thought this would be too complicated, and I wrote 8399438bc7ef "formatter: provide hint of context keys required by template" because of that. However, I had to add a similar mechanism for fctx templates, and the overall machinery became way simpler than my original patch. The test output slightly changed as {author} is no longer available in the {manifest} context, which isn't the point this test targeted on.

merge with stable

py3: call hgweb.hgweb() with bytes values # skip-blame because just b'' prefixes I believe this should fix some tests. Differential Revision: https://phab.mercurial-scm.org/D4594

py3: use '%d' for integers instead of '%s' Differential Revision: https://phab.mercurial-scm.org/D4593

py3: use "%f" for floats instead of "%s" Differential Revision: https://phab.mercurial-scm.org/D4592

py3: suppress the return value from .write() call Differential Revision: https://phab.mercurial-scm.org/D4591

py3: add b'' prefixes in tests/test-diff-color.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D4590

py3: slice through bytes to prevent getting ascii value I still don't know why python-dev thought it was a nice idea to do this. Differential Revision: https://phab.mercurial-scm.org/D4589

censor: use a reasonable amount of memory Before this change, trying to censor some random revision uses an ever increasing amount of memory (I stopped at 20GB, but it was by no means finished), presumably because these contexts have a lot of information that is kept alive. After this change, the memory usage plateaus quickly. Differential Revision: https://phab.mercurial-scm.org/D4582

help: add internals.wireprotocolrpc to the table

setup: exclude vendored futures package on Python 3 The vendored future can't live on Python 3.

py3: whitelist another passing test Differential Revision: https://phab.mercurial-scm.org/D4562

py3: prevent the win32 ctype _fields_ from being transformed to bytes Otherwise, any hg invocation dies with TypeError: '_fields_' must be a sequence of (name, C type) pairs # skip-blame just a r prefix

cext: fix warnings when building for py3 on Windows MSVC++ 14 now has standard int types that don't need to be redefined (I didn't go back to see when they came along since the build system wants either 2008 or 2015), but doesn't have ssize_t. The FILE pointer in posixfile is only used on python2.

cext: stop preprocessing a partial function call MSVC++ 14 yelled: mercurial/cext/revlog.c(1913): fatal error C1057: unexpected end of file in macro expansion At this point, the C extensions build (with warnings), and it dies in win32.py because the `_fields_` strings in the ctypes classes are being converted to bytes by the source translator.

py3: add b'' to some setup.py strings for Windows These were things found trying to do `make PYTHON="py -3" local`. The following is dumped out, before dying while compiling the C extensions: C:\Program Files\Python37\lib\site-packages\setuptools\dist.py:406: UserWarning: The version specified (b'4.7.1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details. "details." % self.metadata.version running build_py byte-compiling .\mercurial\thirdparty\concurrent\futures\_base.py to _base.cpython-37.pyc File "mercurial\thirdparty\concurrent\futures\_base.py", line 416 raise exception_type, self._exception, self._traceback ^ SyntaxError: invalid syntax # skip-blame since these are just converting to bytes literals

dagop: fix typo spotted while doing unrelated investigation Differential Revision: https://phab.mercurial-scm.org/D4584

hg: don't reuse repo instance after unshare() Unsharing a repository is a pretty invasive procedure and fundamentally changes the behavior of the repository. Currently, hg.unshare() calls into localrepository.__init__ to re-initialize the repository instance. This is a bit hacky. And future commits that refactor how localrepository instances are constructed will make this difficult to support. This commit changes unshare() so it constructs a new repo instance once the unshare I/O has completed. It then poisons the old repo instance so any further use will result in error. Surprisingly, nothing in core appears to access a repo instance after it has been unshared! .. api:: ``hg.unshare()`` now poisons the repo instance so it can't be used. It also returns a new repo instance suitable for interacting with the unshared repository. Differential Revision: https://phab.mercurial-scm.org/D4557

unionrepo: dynamically create repository type from base repository This is basically the same thing we just did for bundlerepo except for union repositories. .. api:: ``unionrepo.unionrepository()`` is no longer usable on its own. To instantiate an instance, call ``unionrepo.instance()`` or ``unionrepo.makeunionrepository()``. Differential Revision: https://phab.mercurial-scm.org/D4556

bundlerepo: dynamically create repository type from base repository Previously, bundlerepository inherited from localrepo.localrepository. You simply instantiated a bundlerepository and its __init__ called localrepo.localrepository.__init__. Things were simple. Unfortunately, this strategy is limiting because it assumes that the base repository is a localrepository instance. And it assumes various properties of localrepository, such as the arguments its __init__ takes. And it prevents us from changing behavior of localrepository.__init__ without also having to change derived classes. Previous and ongoing work to abstract storage revealed these limitations. This commit changes the initialization strategy of bundle repositories to dynamically create a type to represent the repository. Instead of a static type, we instantiate a new local repo instance via localrepo.instance(). We then combine its __class__ with bundlerepository to produce a new type. This ensures that no matter how localrepo.instance() decides to create a repository object, we can derive a bundle repo object from it. i.e. localrepo.instance() could return a type that isn't a localrepository and it would "just work." Well, it would "just work" if bundlerepository's custom implementations only accessed attributes in the documented repository interface. I'm pretty sure it violates the interface contract in a handful of places. But we can worry about that another day. This change gets us closer to doing more clever things around instantiating repository instances without having to worry about teaching bundlerepository about them. .. api:: ``bundlerepo.bundlerepository`` is no longer usable on its own. The class is combined with the class of the base repository it is associated with at run-time. New bundlerepository instances can be obtained by calling ``bundlerepo.instance()`` or ``bundlerepo.makebundlerepository()``. Differential Revision: https://phab.mercurial-scm.org/D4555

bundlerepo: factor out code for instantiating a bundle repository This code will soon become a bit more complicated. So extract to its own function. And change both instantiators of bundlerepository to use it. Differential Revision: https://phab.mercurial-scm.org/D4554

bundlerepo: pass create=True I don't want to know how this came to be. Maybe a holdover from the days before Python had a bool type? Differential Revision: https://phab.mercurial-scm.org/D4553

shelve: use bundlerepo.instance() to construct a repo object The instance() functions are preferred over cls.__init__ for creating repo instances. It doesn't really matter now. But future commits will refactor the bundlerepository class in ways that will cause the old way to break. Differential Revision: https://phab.mercurial-scm.org/D4552

templatekw: add experimental {status} keyword This is another example of fctx-based keywords. I think this is somewhat useful in log templates.

templatekw: add option to include ignored/clean/unknown files in cache They will be necessary to provide {status} of files.