transaction: make names a private attribute This is used to report the transaction name in __repr__. It is very obviously an implementation detail and doesn't need to be exposed as part of the public interface. So mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4633

transaction: make map a private attribute This is used to track which files are modified. It is an implementation detail of current transactions and doesn't need to be exposed to the public interface. So mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4632

transaction: make report a private attribute This is a callable used for logging. It isn't used outside the transaction code. It doesn't need to be part of the public interface. Let's mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4631

transaction: make opener a private attribute The VFS instance is an implementation detail of the transaction and doesn't belong as part of the public interface. So mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4630

transaction: make after a private attribute This is another callable that is passed in at __init__ time. It doesn't need to be part of the public interface. Differential Revision: https://phab.mercurial-scm.org/D4629

transaction: make checkambigfiles a private attribute This holds instance state that is passed in at __init__ time. It doesn't need to be exposed as part of the public interface. Differential Revision: https://phab.mercurial-scm.org/D4628

transaction: make validator a private attribute This is similar to releasefn. It holds state that doesn't need to be exposed as part of the public interface. Differential Revision: https://phab.mercurial-scm.org/D4627

transaction: make releasefn a private attribute This is a handle on a callable that is called when the journal is closed. The value is specified at __init__ time. It doesn't need to be exposed on the public interface. So mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4626

transaction: make file a private attribute This holds a file handle for the journal file. This file handle should not be touched outside the journal class and doesn't belong on the public interface. Differential Revision: https://phab.mercurial-scm.org/D4625

transaction: make journal a private attribute This attribute tracks the name of the journal file. It is an implementation detail of the current transaction and therefore shouldn't be exposed as part of the interface. Let's mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4624

transaction: make undoname a private attribute This attribute tracks the file pattern to use for undo files. It is an implementation detail of the current transaction semantics and doesn't need to be part of the future transaction interface. So mark it as private. Differential Revision: https://phab.mercurial-scm.org/D4623

transaction: make count and usages private attributes I want to formalize the interface for transactions. As part of doing that, let's take the opportunity to make some attributes non-public. "count" and "usages" track how many times the transaction has been opened/nested/closed/released. This is internal state and doesn't need to be part of the public API. Differential Revision: https://phab.mercurial-scm.org/D4622

narrow: don't send the changelog information when widening without ellipses When we widen anon-ellipses narrow copy, the server sends the changelog information of all the changesets. The code was copied from ellipses case and in ellipses cases, it's required to send the new changelog data. But in non-ellipses cases, we don't need to send the changelog data as we will have all the changesets locally. Before this patch, there was a overhead of ~8-10 mins on each widening call because of all the changelog information being pulled and being applied. After this patch, we no more pull the changelog information. So this patch can save ~5 mins on Mozilla repo on each widening and more on repos which have more changesets. When we apply an empty changelog from changegroup, there is a devel-warn. This patch kind of hacks to silence that devel-warn. Differential Revision: https://phab.mercurial-scm.org/D4639

changegroup: add functionality to skip adding changelog data to changegroup In narrow extension, when we have a non-ellipses narrow working copy and we extend it, we pull all the changelog data again and the client tries to reapply all that changelog data. While downloading millions of changeset data is still not very expensive but applying them on the client side is very expensive and takes ~10 minutes. These 10 minutes are added to every `hg tracked --addinclude <>` call and extending a narrow copy becomes very slow. This patch adds a new changelog argument to cgpacker.generate() fn. If the changelog argument is set to False, we won't yield the changelog data. We still have to iterate over the deltas returned by _generatechangelog() because that's a generator and builds the data for clstate variable which is required for calculating manifests and filelogs. Differential Revision: https://phab.mercurial-scm.org/D4638

tests: add debug output in test-narrow-widen-no-ellipsis.t This will help us in understanding the upcoming patches better. Differential Revision: https://phab.mercurial-scm.org/D4637

changegroup: improve the devel-warn to specify changelog was empty Right now, the develwarn says "applied empty changegroup" which is not correct because we can send a changegroup without changelog with just manifest and filelogs and it will still say the same. Let's fix this to say that we are applying empty changelog from changegroup. In future patches I am will be adding functionality to send a changegroup from the server without an empty changelog. Differential Revision: https://phab.mercurial-scm.org/D4636

zsh_completion: add -b/--branch and -B/--bookmark(s) flags properly _hg_branch_bmark_opts used to add these two flags, but had the same descriptions for the flags regardless of what command took them and didn't allow specifying flags more than once (no '*' at the start). Even more importantly, it assumed that -B was always expecting an argument (i.e. --bookmark=foo), but in case of incoming and outgoing it's not so (--bookmarks is self-sufficient). Differential Revision: https://phab.mercurial-scm.org/D4612

narrow: when writing treemanifests, skip inspecting directories outside narrow This provides significant speed benefits when narrow and treemanifests are in use, see the timing numbers below. Note that like previously, differences of <5% are considered noise. The below timing numbers are in the same style as previously (example: ee7ee0c516ca). 'before' is 9db85644, and does not include that example commit's improvements. diff --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.327 s +- 0.051 s | 1.296 s +- 0.009 s | 97.7% m-u | | x | 1.310 s +- 0.020 s | 1.295 s +- 0.015 s | 98.9% m-u | x | | 1.295 s +- 0.018 s | 1.296 s +- 0.007 s | 100.1% m-u | x | x | 83.5 ms +- 0.8 ms | 84.1 ms +- 0.8 ms | 100.7% l-d-r | | | 205.1 ms +- 3.5 ms | 205.0 ms +- 3.8 ms | 100.0% l-d-r | | x | 194.2 ms +- 5.6 ms | 192.3 ms +- 4.3 ms | 99.0% l-d-r | x | | 99.1 ms +- 2.2 ms | 97.8 ms +- 0.9 ms | 98.7% l-d-r | x | x | 66.2 ms +- 1.0 ms | 67.2 ms +- 2.7 ms | 101.5% diff -c . --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 233.9 ms +- 1.9 ms | 235.6 ms +- 5.1 ms | 100.7% m-u | | x | 151.4 ms +- 1.2 ms | 152.2 ms +- 2.0 ms | 100.5% m-u | x | | 234.8 ms +- 2.7 ms | 235.0 ms +- 2.7 ms | 100.1% m-u | x | x | 127.8 ms +- 2.1 ms | 126.0 ms +- 1.1 ms | 98.6% l-d-r | | | 82.5 ms +- 1.6 ms | 82.3 ms +- 2.0 ms | 99.8% l-d-r | | x | 3.742 s +- 0.017 s | 3.819 s +- 0.208 s | 102.1% l-d-r | x | | 84.4 ms +- 1.5 ms | 83.2 ms +- 1.0 ms | 98.6% l-d-r | x | x | 751.2 ms +- 5.0 ms | 755.8 ms +- 12.9 ms | 100.6% rebase -r . --keep -d .^^: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 5.519 s +- 0.038 s | 5.526 s +- 0.057 s | 100.1% m-u | | x | 5.588 s +- 0.048 s | 5.607 s +- 0.061 s | 100.3% m-u | x | | 5.520 s +- 0.044 s | 5.546 s +- 0.059 s | 100.5% m-u | x | x | 586.6 ms +- 12.8 ms | 554.9 ms +- 21.2 ms | 94.6% <-- l-d-r | | | 629.8 ms +- 5.5 ms | 627.4 ms +- 6.6 ms | 99.6% l-d-r | | x | 6.165 s +- 0.058 s | 6.255 s +- 0.303 s | 101.5% l-d-r | x | | 270.2 ms +- 2.3 ms | 271.4 ms +- 2.7 ms | 100.4% l-d-r | x | x | 4.700 s +- 0.025 s | 1.651 s +- 0.016 s | 35.1% <-- status --change . --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 215.4 ms +- 2.3 ms | 216.5 ms +- 4.2 ms | 100.5% m-u | | x | 132.9 ms +- 1.2 ms | 132.0 ms +- 1.4 ms | 99.3% m-u | x | | 217.0 ms +- 1.9 ms | 215.4 ms +- 1.9 ms | 99.3% m-u | x | x | 108.6 ms +- 1.0 ms | 108.2 ms +- 1.5 ms | 99.6% l-d-r | | | 80.0 ms +- 1.3 ms | 80.5 ms +- 1.1 ms | 100.6% l-d-r | | x | 3.916 s +- 0.187 s | 3.966 s +- 0.236 s | 101.3% l-d-r | x | | 84.4 ms +- 3.1 ms | 83.9 ms +- 1.1 ms | 99.4% l-d-r | x | x | 758.0 ms +- 8.2 ms | 753.5 ms +- 5.0 ms | 99.4% status --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.905 s +- 0.025 s | 1.910 s +- 0.044 s | 100.3% m-u | | x | 1.892 s +- 0.009 s | 1.895 s +- 0.012 s | 100.2% m-u | x | | 1.891 s +- 0.012 s | 1.902 s +- 0.018 s | 100.6% m-u | x | x | 93.3 ms +- 0.9 ms | 93.4 ms +- 0.8 ms | 100.1% l-d-r | | | 570.7 ms +- 7.8 ms | 571.9 ms +- 18.5 ms | 100.2% l-d-r | | x | 561.5 ms +- 5.2 ms | 562.9 ms +- 6.1 ms | 100.2% l-d-r | x | | 171.7 ms +- 2.6 ms | 171.9 ms +- 1.2 ms | 100.1% l-d-r | x | x | 142.7 ms +- 2.0 ms | 140.3 ms +- 1.0 ms | 98.3% update $rev^; ~/src/hg/hg{hg}/hg update $rev: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 3.126 s +- 0.016 s | 3.128 s +- 0.015 s | 100.1% m-u | | x | 3.014 s +- 0.068 s | 3.008 s +- 0.031 s | 99.8% m-u | x | | 3.143 s +- 0.037 s | 3.184 s +- 0.086 s | 101.3% m-u | x | x | 308.0 ms +- 1.8 ms | 308.1 ms +- 5.7 ms | 100.0% l-d-r | | | 430.8 ms +- 4.5 ms | 436.4 ms +- 8.7 ms | 101.3% l-d-r | | x | 9.676 s +- 0.127 s | 9.945 s +- 0.272 s | 102.8% l-d-r | x | | 254.2 ms +- 3.3 ms | 255.7 ms +- 3.1 ms | 100.6% l-d-r | x | x | 1.571 s +- 0.030 s | 1.555 s +- 0.014 s | 99.0% Differential Revision: https://phab.mercurial-scm.org/D4606

tests: fix a couple of drawdag.py references Differential Revision: https://phab.mercurial-scm.org/D4635

py3: fix kwargs handling in hgext/fastannotate.py Differential Revision: https://phab.mercurial-scm.org/D4588

narrow: use diffmatcher to send only new filelogs in non-ellipses widening Before this patch, when we widen a non-ellipses narrow clone, we downloads all the filelogs matching the resulting new matcher. This is same as the ellipses case but can be improved because, we don't pull new csets in non-ellipses cases, we can only download the new added files instead of downloading all the files which matches the new matcher. So, we only download files which matches the new matcher but does not matches the old matcher. There exists a match.differencematcher() which is used here. This will lead to significant amount of speedup in extending a non-ellipses narrow copy on large repos because we will download and process only the new required filelogs. The tests changes demonstrate that we are downloading now less files. Thanks to Augie for pointing that functionality of differencematcher exists in core. Differential Revision: https://phab.mercurial-scm.org/D4614

py3: add missing b'' prefixes in couple of test files These were missed in the earlier patch and caught by Yuya. # skip-blame because just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D4613

run-tests: convert the remaining os.system() call to Unicode I wasn't able to hit this path in 543a788eea2d, but I have now when I accidentally left off `--local`.

py3: partially fix pager spawning on Windows Previously, spinning up the pager crashed because the command and environment was in bytes. (See also 543a788eea2d.) Now it aborts with an invalid handle: $ HGMODULEPOLICY=py py -3 ../hg --traceback --config extensions.evolve=! Traceback (most recent call last): File "c:\Users\Matt\projects\hg\mercurial\ui.py", line 967, in _write self.fout.write(''.join(msgs)) File "c:\Users\Matt\projects\hg\mercurial\windows.py", line 173, in write self.fp.write(s[start:end]) OSError: [WinError 6] The handle is invalid During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\Matt\projects\hg\mercurial\scmutil.py", line 164, in callcatch return func() File "c:\Users\Matt\projects\hg\mercurial\dispatch.py", line 350, in _runcatchfunc return _dispatch(req) File "c:\Users\Matt\projects\hg\mercurial\dispatch.py", line 930, in _dispatch return commands.help_(ui, 'shortlist') File "c:\Users\Matt\projects\hg\mercurial\commands.py", line 2930, in help_ ui.write(formatted) File "c:\Users\Matt\projects\hg\mercurial\ui.py", line 948, in write self._writenobuf(*args, **opts) File "c:\Users\Matt\projects\hg\mercurial\ui.py", line 960, in _writenobuf self._write(*msgs, **opts) File "c:\Users\Matt\projects\hg\mercurial\ui.py", line 969, in _write raise error.StdioError(err) mercurial.error.StdioError: [Errno 9] The handle is invalid abort: The handle is invalid The interesting bit here is that the abort message is marked with ANSI color, but the OSError is not.