run-tests: test result shows when a failed test could not start a server Failing to start a server happens regularly, at least on windows buildbot. Such a failure often has nothing to do with the test, but with the environment. But half the test output can change because some data is missing. Therefore this is worth an extended error message. Detect the server failure in the diff output because it is most reliable there. Checking the output only does not show if the server failure was expected. Old failure message when server start failed: Failed test-serve.t: output changed New message: Failed test-serve.t: serve failed and output changed