bdiff: avoid pointer arithmetic on void* MSVC 2008 complains: mercurial/cext/bdiff.c(106) : error C2036: 'void *' : unknown size mercurial/cext/bdiff.c(107) : error C2036: 'void *' : unknown size Maybe it's a gcc extension? https://stackoverflow.com/questions/37460579/error-c2036-void-unknown-size

fuzz: add a quick README to try and document how to test new fuzzers Differential Revision: https://phab.mercurial-scm.org/D2633

fuzz: add a fuzzer for xdiff Based entirely on the fuzzer for bdiff. Differential Revision: https://phab.mercurial-scm.org/D2632

tests: add tests about diff quality These show the differences between bdiff and xdiff. Differential Revision: https://phab.mercurial-scm.org/D2604

run-tests: allow #require inside #if Used by the next patch. Differential Revision: https://phab.mercurial-scm.org/D2605

mdiff: add a config option to use xdiff algorithm The `experimental.xdiff` will affect the default diffopts and make mdiff use the xdiff algorithm for better diff quality. Differential Revision: https://phab.mercurial-scm.org/D2603

bdiff: add a xdiffblocks method This is similar to `bdiff.blocks`, but uses xdiff as the backend. The indent heuristic is turned on by default since it has little overhead and improves diff quality significantly. Differential Revision: https://phab.mercurial-scm.org/D2602

xdiff: reduce indent heuristic overhead Adds some threshold to avoid expensive cases, like: ``` #!python open('a', 'w').write(" \n" * 1000000) open('b', 'w').write(" \n" * 1000001) ``` The indent heuristic is O(N * 20) (N = 1000000) for the above case, and makes diff much slower. Before this patch (system git 2.14.2): ``` git diff --no-indent-heuristic a b 0.21s user 0.03s system 100% cpu 0.239 total git diff --indent-heuristic a b 0.77s user 0.02s system 99% cpu 0.785 total ``` After this patch (git 2fc74f41, with xdiffi.c patched): ``` # with the changed xdiffi.c git diff --indent-heuristic a b 0.16s user 0.01s system 90% cpu 0.188 total git diff --no-indent-heuristic a b 0.18s user 0.01s system 99% cpu 0.192 total ``` Now turning on indent-heuristic has no visible impact on performance. Differential Revision: https://phab.mercurial-scm.org/D2601

xdiff: add a bdiff hunk mode xdiff generated hunks for the differences (ex. questionmarks in the `@@ -?,? +?,? @@` part from `diff --git` output). However, bdiff generates matched hunks instead. This patch adds a `XDL_EMIT_BDIFFHUNK` flag used by the output function `xdl_call_hunk_func`. Once set, xdiff will generate bdiff-like hunks instead. That makes it easier to use xdiff as a drop-in replacement of bdiff. Note that since `bdiff('', '')` returns `[(0, 0, 0, 0)]`, the shortcut path `if (xscr)` is removed. I have checked functions called with `xscr` argument (`xdl_mark_ignorable`, `xdl_call_hunk_func`, `xdl_emit_diff`, `xdl_free_script`) work just fine with `xscr = NULL`. Test Plan: Will be tested in a later patch. Differential Revision: https://phab.mercurial-scm.org/D2575

xdiff: remove patience and histogram diff algorithms Patience diff is the normal diff algorithm, plus some greediness that unconditionally matches common common unique lines. That means it is easy to construct cases to let it generate suboptimal result, like: ``` open('a', 'w').write('\n'.join(list('a' + 'x' * 300 + 'u' + 'x' * 700 + 'a\n'))) open('b', 'w').write('\n'.join(list('b' + 'x' * 700 + 'u' + 'x' * 300 + 'b\n'))) ``` Patience diff has been advertised as being able to generate better results for some C code changes. However, the more scientific way to do that is the indention heuristic [1]. Since patience diff could generate suboptimal result more easily and its "better" diff feature could be replaced by the new indention heuristic, let's just remove it and its variant histogram diff to simplify the code. [1]: https://github.com/git/git/commit/433860f3d0beb0c6f205290bd16cda413148f098 Test Plan: `gcc -fPIC *.c --shared -o xdiff.so` still builds. Differential Revision: https://phab.mercurial-scm.org/D2573