index: remove side-effect from failed nt_new() As pointed out by Yuya in the review of D4108, if realloc() fails, we would end up with an invalid nodetree instance (with nt->nodes set to NULL), which means that if it was later accessed again it would likely segfault. It's probably unlikely that much else happens in the process if it ran out memory, but we should of course do our best to handle it. This patch makes it so we don't update the nodetree in this case. Differential Revision: https://phab.mercurial-scm.org/D4154

index: remove side-effect from failed nt_init() As pointed out by Yuya in the review of D4108, if we run into the "overflow in nt_init" case (which I think normally happens only in repos with at least 2^26=64Mi revisions), we would leave the node tree half-initialized. Differential Revision: https://phab.mercurial-scm.org/D4153

index: use PyMem_Free() to free nodeetree instance As Yuya pointed out in the review of D4108, PyMem_Malloc() and PyMem_Free() should be paired. IIUC, PyMem_Malloc() may use a different allocator than malloc(), so using free() with a pointer from PyMem_Malloc() may be very wrong. Differential Revision: https://phab.mercurial-scm.org/D4152

linelog: fix infinite loop vulnerability Checking `len(lines)` is not a great way of detecting infinite loops, as demonstrated in the added test. Therefore check instruction count instead. The original C implementation does not have this problem. There are a few other places where the C implementation enforces more strictly, like `a1 <= a2`, `b1 <= b2`, `rev > 0`. But they are optional. Test Plan: Add a test. The old code forces the test to time out. Differential Revision: https://phab.mercurial-scm.org/D4151