hook: also use pprint on lists for stable output on py2/3 Differential Revision: https://phab.mercurial-scm.org/D3361

hook: use stringutil.pprint instead of reinventing it Differential Revision: https://phab.mercurial-scm.org/D3360

stringutil: make b prefixes on string output optional I need this to preserve some behavior in hook.py. Differential Revision: https://phab.mercurial-scm.org/D3359

stringutil: teach pprint how to format None Differential Revision: https://phab.mercurial-scm.org/D3358

tests: update inline extensions in test-bundle2-exchange.t to py3 The test doesn't pass for what superficially look like good reasons. We'll need to come back to it later. # skip-blame because it's b prefixes and a couple of b'N' instead of str(N) Differential Revision: https://phab.mercurial-scm.org/D3357

wsgicgi: un-do some prior porting work that is now wrong The Python 3 WSGI behavior is that the environ dict should be full of unicodes. We previously tried Too Hard here, so we unwind that bit of porting. Also add some bytesurl() encodes on status and headers. test-clone-cgi.t now passes. Differential Revision: https://phab.mercurial-scm.org/D3356

hgweb_mod: inform hgweb class about paths actually being bytes Differential Revision: https://phab.mercurial-scm.org/D3355

hgweb: inform hgweb.hgweb() entrypoint that paths should be bytes Differential Revision: https://phab.mercurial-scm.org/D3354

tests: port inline cgi script in test-largefiles.t to python 3 Differential Revision: https://phab.mercurial-scm.org/D3353

lfcommands: use %d on known-int in format string Differential Revision: https://phab.mercurial-scm.org/D3352

py3: fix test-shelve.t on Python 3 We get a better error message out of the abort here, which is fine. Differential Revision: https://phab.mercurial-scm.org/D3351

tests: fix up a couple of minor bytes inconsistencies in run-tests.py Only in the automatic bisection code, so fortunately nothing major is amiss. Fixes test-run-tests.t under Python 3. Differential Revision: https://phab.mercurial-scm.org/D3350

tests: make sure test-run-tests.t actually runs run-tests.py under Python 3 I'm fairly certain it hasn't been until now. Mercifully, there doesn't appear to be any ninja breakage. Differential Revision: https://phab.mercurial-scm.org/D3349

py3: another three passing Differential Revision: https://phab.mercurial-scm.org/D3348

httppeer: work around API differences on urllib Request objects Since this is only a problem in httppeer, I'd rather keep this a local-to-the-module kludge rather than pile more on pycompat. We'll still find it easily to clean up later because it checks pycompat.ispy3. Differential Revision: https://phab.mercurial-scm.org/D3347

httppeer: no matter what Python 3 might think, http headers are bytes Differential Revision: https://phab.mercurial-scm.org/D3346

httppeer: fix debug prints to work on Python 3 Differential Revision: https://phab.mercurial-scm.org/D3345

url: some bytes/str cleanup where we interface with stdlib funcs Differential Revision: https://phab.mercurial-scm.org/D3344

hgweb: these strings should be sysstrs, not bytes Differential Revision: https://phab.mercurial-scm.org/D3343

tests: port inline extensions in test-http.t to Python 3 Differential Revision: https://phab.mercurial-scm.org/D3342

patch: implement a new worddiff algorithm The previous worddiff algorithm has many problems. The major problem is it does a "similarity check" that selects a subset of matched lines to do inline diffs. It is a bad idea because: - The "similarity check" is non-obvious to users. For example, a simple change from "long long x" to "int64_t x" will fail the similarity check and won't be diff-ed as expected. - Selecting "lines" to diff won't work as people expect if there are line wrapping changes. - It has a sad time complexity if lines do not match, could be O(N^2)-ish. There are other problems in implementation details. - Lines can match across distant hunks (if the next hunk does not have "-" lines). - "difflib" is slow. The solution would be removing the "similarity check", and just diff all words in a same hunk. So no content will be missed and everything will be diff-ed as expected. This is similar to what code review tool like Phabricator does. This diff implements the word diff algorithm as described above. It also avoids difflib to be faster. Note about colors: To be consistent, "changed inserted" parts and "purely insertion blocks" should have a same color, since they do not exist in the previous version. Instead of highlighting differences, this patch chooses to dim common parts. This is also more consistent with Phabricator or GitHub webpage. That said, the labels are defined in a way that people can still highlight changed parts and leave purely inserted/deleted hunks use the "non-highlighted" color. As one example, running: hg log -pr df50b87d8f736aff8dc281f816bddcd6f306930c mercurial/commands.py \ --config experimental.worddiff=1 --color=debug --config diff.unified=0 The previous algorithm outputs: [diff.file_a|--- a/mercurial/commands.py Fri Mar 09 15:53:41 2018 +0100] [diff.file_b|+++ b/mercurial/commands.py Sat Mar 10 12:33:19 2018 +0530] [diff.hunk|@@ -2039,1 +2039,4 @@] [diff.deleted|-][diff.deleted.highlight|@command('^forget',][diff.deleted| ][diff.deleted.highlight|walkopts,][diff.deleted| _('[OPTION]... FILE...'), inferrepo=True)] [diff.inserted|+@command(] [diff.inserted|+ '^forget',] [diff.inserted|+ walkopts + dryrunopts,] [diff.inserted|+ ][diff.inserted.highlight| ][diff.inserted| _('[OPTION]... FILE...'), inferrepo=True)] [diff.hunk|@@ -2074,1 +2077,3 @@] [diff.deleted|- rejected = cmdutil.forget(ui, repo, m, prefix="",][diff.deleted.highlight| explicitonly=False)[0]] [diff.inserted|+ dryrun = opts.get(r'dry_run')] [diff.inserted|+ rejected = cmdutil.forget(ui, repo, m, prefix="",] [diff.inserted|+ explicitonly=False, dryrun=dryrun)[0]] The new algorithm outputs: [diff.file_a|--- a/mercurial/commands.py Fri Mar 09 15:53:41 2018 +0100] [diff.file_b|+++ b/mercurial/commands.py Sat Mar 10 12:33:19 2018 +0530] [diff.hunk|@@ -2039,1 +2039,4 @@] [diff.deleted|-][diff.deleted.unchanged|@command(][diff.deleted.unchanged|'^forget',][diff.deleted.unchanged| ][diff.deleted.changed|walkopts][diff.deleted.unchanged|,][diff.deleted.changed| ][diff.deleted.unchanged|_('[OPTION]... FILE...'), inferrepo=True)] [diff.inserted|+][diff.inserted.unchanged|@command(] [diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|'^forget',] [diff.inserted|+][diff.inserted.changed| walkopts][diff.inserted.unchanged| ][diff.inserted.changed|+ dryrunopts][diff.inserted.unchanged|,] [diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|_('[OPTION]... FILE...'), inferrepo=True)] [diff.hunk|@@ -2074,1 +2077,3 @@] [diff.deleted|-][diff.deleted.unchanged| rejected = cmdutil.forget(ui, repo, m, prefix="",][diff.deleted.changed| ][diff.deleted.unchanged|explicitonly=False][diff.deleted.unchanged|)[0]] [diff.inserted|+][diff.inserted.changed| dryrun = opts.get(r'dry_run')] [diff.inserted|+][diff.inserted.unchanged| rejected = cmdutil.forget(ui, repo, m, prefix="",] [diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|explicitonly=False][diff.inserted.changed|, dryrun=dryrun][diff.inserted.unchanged|)[0]] Practically, when diffing a 8k line change, the time spent on worddiff reduces from 4 seconds to 0.14 seconds. Differential Revision: https://phab.mercurial-scm.org/D3212

patch: buffer lines for a same hunk Instead of yielding tokens directly, buffer them if they belong to a same hunk. This makes it easier for the upcoming new worddiff algorithm to only focus on the diff hunk, instead of having to worry about other contents. This breaks how the existing experimental worddiff algorithm works, so the algorithm was removed, and related tests are disabled for now. The next patch will add a new worddiff algorithm. Differential Revision: https://phab.mercurial-scm.org/D3211

patch: move yielding "\n" to the end of loop The original logic makes it harder to reason about - it yields the "\n" character belonging to the last line in the next loop iteration. The new code is in theory a little bit slower. But is more readable. It makes the following changes easier to read. Differential Revision: https://phab.mercurial-scm.org/D3210

context: clarify deprecation warning message I had one developer report that they couldn't find the message. This patch should make it clear where to find it. Differential Revision: https://phab.mercurial-scm.org/D3389