changeset 38938:dcd395dc98d8

index: remove side-effect from failed nt_new()

As pointed out by Yuya in the review of D4108, if realloc() fails, we
would end up with an invalid nodetree instance (with nt->nodes set to
NULL), which means that if it was later accessed again it would likely
segfault. It's probably unlikely that much else happens in the process
if it ran out memory, but we should of course do our best to handle
it. This patch makes it so we don't update the nodetree in this case.

Differential Revision: https://phab.mercurial-scm.org/D4154

--- a/mercurial/cext/revlog.c	Mon Aug 06 22:34:37 2018 -0700
+++ b/mercurial/cext/revlog.c	Mon Aug 06 09:59:51 2018 -0700
@@ -1018,18 +1018,21 @@
 static int nt_new(nodetree *self)
 {
 	if (self->length == self->capacity) {
+		unsigned newcapacity;
+		nodetreenode *newnodes;
 		if (self->capacity >= INT_MAX / (sizeof(nodetreenode) * 2)) {
 			PyErr_SetString(PyExc_MemoryError,
 					"overflow in nt_new");
 			return -1;
 		}
-		self->capacity *= 2;
-		self->nodes = realloc(self->nodes,
-		                      self->capacity * sizeof(nodetreenode));
-		if (self->nodes == NULL) {
+		newcapacity = self->capacity * 2;
+		newnodes = realloc(self->nodes, newcapacity * sizeof(nodetreenode));
+		if (newnodes == NULL) {
 			PyErr_SetString(PyExc_MemoryError, "out of memory");
 			return -1;
 		}
+		self->capacity = newcapacity;
+		self->nodes = newnodes;
 		memset(&self->nodes[self->length], 0,
 		       sizeof(nodetreenode) * (self->capacity - self->length));
 	}