merge revlogng with mpm tip

Fix transaction handling during interleaved file split.

merge 0.8.1 with revlogng

Additional appendfile fixes for interleaved data/index files The appendfile code was not passing default version info to the changelog or manifest classes, and so they were always being created as version 0. revlog.checkinlinesize had to be corrected to seek to the end of the index file when no index file was passed (only clone does this)

Speedup revlog.ancestors for the linear case revlog.ancestors can be expensive on big repos. This cuts down the overall time for hg update by ~19% by short cutting revlog.ancestors when one of the revisions is reachable from another.

Reduce index memory usage by storing the bare string instead of tuples Storing the tuple returned by struct.unpack significantly increases the memory required to store the entire index in ram. This patch uses struct.unpack on demand instead.

New lazy index code for revlogs. This tunes for large repositories. It does not read the whole index file in one big chunk, but tries to buffer reads in more reasonable chunks instead. Search speeds are improved in two ways. When trying to find a specific sha hash, it searches from the end of the file backward. More recent entries are more likely to be relevant, especially the tip. Also, this can load only the mapping of nodes to revlog index number. Loading the map uses less cpu (no struct.unpack) and much less memory than loading both the map and the index. This cuts down the time for hg tip on the 80,000 changeset kernel repo from 1.8s to 3.69s. Most commands the pull a single rev out of a big index get roughly the same benefit. Commands that read the whole index are not slower.

Fill in the uncompressed size during revlog.addgroup This uses code from Matt to calculate the size change that would result from applying a delta to keep an accurate running total of the text size during revlog.addgroup

Fix inlined revlogs to seek to eof after opening "a+"

Create an atomic opener that does not automatically rename on close The revlog.checkinlinesize() uses an atomic opener to replace the index file after converting it from inline to traditional .i and .d files. If this operation is interrupted, the atomic file class can overwrite a valid file with a partially written one. This patch introduces an atomic opener that does not automatically replace the destination file with the tempfile. This way an interrupted checkinlinesize() call turns into a noop.