dirstate-v2: Reuse existing nodes when appending to a data file When writing a dirstate in v2 format by appending to an existing data file, nodes that are still "unparsed" from the previous on-disk representation have been unchanged and can therefore be reused. This makes the new data point to previously-existing data for tree nodes. Differential Revision: https://phab.mercurial-scm.org/D11095

dirstate-v2: Support appending to the same data file For now we’re still writing the entire data every time, so appending is not useful yet. Later we’ll have new nodes pointing to some existing data for nodes and paths that haven’t changed. The decision whether to append is pseudo-random in order to make tests exercise both code paths. This will be replaced by a heuristic based on the amount of unused existing data. Differential Revision: https://phab.mercurial-scm.org/D11094

dirstate-v2: shrink on-disk path lengths to 16-bits Differential Revision: https://phab.mercurial-scm.org/D11091

dirstate-v2: Rename Header to Root, move it to the end of the data file Now that they don’t have to be at the start, a given data file may contain multiple "roots". A docket only points to one of them, and previous ones are left unused to allow allow append-only in-place writing to an existing data file. Differential Revision: https://phab.mercurial-scm.org/D11090

dirstate-v2: Enforce data size read from the docket file The data file may not be shorter than its size given by the docket. It may be longer, but additional data is ignored. Differential Revision: https://phab.mercurial-scm.org/D11089

dirstate-v2: Introduce a docket file .hg/dirstate now only contains some metadata to point to a separate data file named .hg/dirstate.{}.d with a random hexadecimal identifier. For now every update creates a new data file and removes the old one, but later we’ll (usually) append to an existing file. Separating into two files allows doing the "write to a temporary file then atomically rename into destination" dance with only a small docket file, without always rewriting a lot of data. Differential Revision: https://phab.mercurial-scm.org/D11088

dirstate: replace a dead conditional branch with an assert in `update_file` This is a case we never meet, so lets trim it away to simplify the code before more changes. Differential Revision: https://phab.mercurial-scm.org/D11128

dirstate-map: do not use `size` to gate copy dropping during remove_file This get us close to moving the block right above withing the DirstateItem object. Doing so will help us getting rid of magic constant at the dirstatemap level. Differential Revision: https://phab.mercurial-scm.org/D11127

dirstate: drop a duplicated assert This very case is checked in 3 line above that one. Differential Revision: https://phab.mercurial-scm.org/D11126

dirstate: drop duplicated check This is covered by the `@requires_parents_change` decorator that this function use. Differential Revision: https://phab.mercurial-scm.org/D11125