dirstate-v2: Make more APIs fallible, returning Result When parsing becomes lazy, parse error will potentially happen in more places. This propagates such errors to callers. Differential Revision: https://phab.mercurial-scm.org/D10749

dirstate-v2: Add a zero-size error type for dirstate v2 parse errors This error should only happen if Mercurial is buggy or the file is corrupted. It indicates for example that: * A part of the file refers to another part, and the byte offset or item count would cause reading out of bounds, beyond the end of the file. * The byte for an entry state has an invalid value When parsing becomes lazy, many more functions will return a `Result` with this error. Making it zero-size reduces the work that the `?` operator needs to do to pass around the error value. Differential Revision: https://phab.mercurial-scm.org/D10748

dirstate-tree: Add `NodeRef` and `ChildNodesRef` enums They are used instead of `&Node` and `&ChildNodes` respectively. The `ChildNodes` type alias also becomes a similar enum. For now they only have one variant each, to be extended later. Adding enums now forces various use sites go through new methods instead of manipulating the underlying data structure directly. Differential Revision: https://phab.mercurial-scm.org/D10747

rust: Return owned instead of borrowed DirstateEntry in DirstateMap APIs This will enable the tree-based DirstateMap to not always have an actual DirstateEntry in memory for all nodes, but construct it on demand. Differential Revision: https://phab.mercurial-scm.org/D10746

dirstate-tree: Downgrade `&mut Node` to `&Node` in status and serialization Mutable access is not used, and upcoming changes will make it more costly (with copy-on-write nodes that can be read from disk representation) Differential Revision: https://phab.mercurial-scm.org/D10745

dirstate-tree: Remove DirstateMap::iter_node_data_mut In an upcoming changeset we want DirstateMap to be able to work directly with nodes in their "on disk" representation, without always allocating corresponding in-memory data structures. Nodes would have two possible representations: one immutable "on disk" refering to the bytes buffer of the contents of the .hg/dirstate file, and one mutable with HashMap like the curren data structure. These nodes would have copy-on-write semantics: when an immutable node would need to be mutated, instead we allocate new mutable node for it and its ancestors. A mutable iterator of the entire tree would still be possible, but it would become much more expensive since we’d need to allocate mutable nodes for everything. Instead, remove this iterator. It was only used to clear ambiguous mtimes while serializing the `DirstateMap`. Instead clearing and serialization are now two separate passes. Clearing first uses an immutable iterator to collect the paths of nodes that need to be cleared, then accesses only those nodes mutably. Differential Revision: https://phab.mercurial-scm.org/D10744

merge with stable

revlog: close the index file handle after the data one This make sure the data file is flushed before the index. preventing the index to reference unflushed data. Differential Revision: https://phab.mercurial-scm.org/D10776

revlog: simplify the try nesting in the `_writing` context Lets use a single try, with conditional cleanup. This make is easier to add a file handle dedicated to sidedata. Differential Revision: https://phab.mercurial-scm.org/D10775

revlogv2: add a `get_data` helper to grab the next piece of docket This make the processing more compact but abstracting repetitive processing away. Differential Revision: https://phab.mercurial-scm.org/D10774

revlogv2: simplify and clarify the processing of each entry As we add more entries and some of them has non trivial processing it seems useful to make the processing leaner and clearly separated to simplify futures patches. Differential Revision: https://phab.mercurial-scm.org/D10773

revlogv2: use a unique filename for data Having a unique data will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. Differential Revision: https://phab.mercurial-scm.org/D10772

revlogv2: use a unique filename for index Having a unique index will allow for ambiguity less rewriting of revlog content, something useful to clarify handling of some operation like censoring or stripping. Differential Revision: https://phab.mercurial-scm.org/D10771

rhg: A missing .hg/dirstate file is not an error Instead treat it as like an empty file Differential Revision: https://phab.mercurial-scm.org/D10766

rhg: Sort `rhg status` output correctly * The relative order of states is: modified, added, removed, deleted, unknown, ignored, clean * Files in the same state should be sorted by name, regardless of whether or not the were in "unsure" state based on metadata alone. Differential Revision: https://phab.mercurial-scm.org/D10765

dirstate-v2: Add `hg debugupgraderepo` command support This command changes changes the file formats used inside an existing repository to what they would be in a new repository with the current config. For example: hg debugupgraderepo --config format.exp-dirstate-v2=1 --run hg debugupgraderepo --config format.exp-dirstate-v2=0 --run If a repository has a dirstate in v1 format, the first command would upgrade it to dirstate-v2. Conversely, if a repository has a dirstate in v2 format, the second command would downgrade it to v1. (Both may also run some unrelated upgrades.) Since `format.exp-dirstate-v2` is currently disabled by default, not specifying it in `--config` or any configuration file would result in the second command. Differential Revision: https://phab.mercurial-scm.org/D10769

upgrade: Use `improvement` subclasses everywhere, not instances This changes the source definition of optimizations to match that of formats: a subclass with a decorator, instead of an instance passed to a function call. Not having any instance removes the confusion between class attributes and instance attributes, which were used interchangeably. Differential Revision: https://phab.mercurial-scm.org/D10768

tests: monkeypatch `util.get_password()` to avoid deadlocks on Windows This should have been part of 5b3513177f2b. Differential Revision: https://phab.mercurial-scm.org/D10759

rust-status: fix ignore and include not composing (issue6514) While the fix is pretty simple, the overall dispatch logic has become kind of ugly. Thankfully we're currently upstreaming a better algorithm, this code is temporary anyway. Differential Revision: https://phab.mercurial-scm.org/D10639

rust-status: highlight a bug in Rust-augmented status This was reported in issue6514, confirmed with this test reproduction. This will be fixed with the next changeset. Differential Revision: https://phab.mercurial-scm.org/D10638

exewrapper: avoid directly linking against python3X.dll Subsequent code calls `LoadLibrary()` to attempt to load the DLL, but because of this symbol reference, there is an attempt to load the DLL used during the build prior to `_main()` running. This causes the whole process to fail if the DLL isn't in the standard search path. That also means it will never load the DLL for HackableMercurial. (Maybe we should get rid of that for py3, since you can install python for a user without admin rights?) This could also be resolved by calling `GetProcAddress()` on the symbol and dereferencing it, but using the environment variable is consistent with the *.bat file since fc8a5c9ecee0. (The environment variable persists after the interpreter is initialized.) Far more concerning is somehow I've gotten my system into a state where setting the flag causes any output to the pager to be lost (as if it wasn't set at all) in MSYS, cmd.exe, WSL, and PowerShell using py3.9.0, but the environment variable works properly. I'm sure this flag worked on some versions of py3, so I'm not sure what's going on here. This is might be related to init config related changes in 3.8[1], since it works with 3.7.8, but fails with 3.8.1. Somebody who understands encoding issues better than I do should give some thought to if we need to make some changes to our encoding strategy on Windows with py3. With or without the flag/envvar, there is proper output if the command is directly paged by piping to `more.com` (in any environment) or `less` (in MSYS and WSL), or if paging is disabled with `--pager=no`. Legacy mode is required though when Mercurial decides to spin up a pager. [1] https://bugs.python.org/issue41941 Differential Revision: https://phab.mercurial-scm.org/D10756

tests: partially fix test-transaction-rollback-on-revlog-split.t on Windows For whatever reason, the extra space causes different output on Windows: $ printf '% 20d' '1' | xxd - 0000000: 2020 2020 2020 2020 2020 2020 2020 2020 0000010: 2020 2020 31 1 vs WSL: /mnt/c/Users/Matt/hg/tests$ printf '% 20d' '1' | xxd - 00000000: 2020 2020 2020 2020 2020 2020 2020 2020 00000010: 2020 2031 Without the space, they both agree the last word is 2031. There are other failures on Windows that I don't understand that are introduced by 46b828b85eb7 and 18415fc918a1. Differential Revision: https://phab.mercurial-scm.org/D10763

tests: fix test-transaction-safety.t on Windows Mostly this is about `C:\dir\sub` style paths being turned into `C:dirsub` by MSYS, and not being about to execute shell scripts in hooks directly. See also a12819559ccb::9e3979a25bfe. Differential Revision: https://phab.mercurial-scm.org/D10762

docket: move the uid logic in the `revlogutils.docket` module We want to use it for revlog-v2 (& Co), it seems more logical to have the logic lives inside the `docket` file than the `nodemap` file. Differential Revision: https://phab.mercurial-scm.org/D10755

persistent-nodemap: use the intended uuid size I overlooked this in the initial implementation. The format already supporting any uuid size so, nothing horrible should happens. Differential Revision: https://phab.mercurial-scm.org/D10754

persistent-nodemap: add a way to make the picked uid predictable This will make the test less volatile and therefore easier to maintain. I am setting this is for the persistent nodemap docket first, but my target is to use similar logic for the revlogv2 (and co) test. For persisent nodemap, they are too many different in how each test variants (with rust, without rust, etc) needs new uid for this be really useful (right now). However it will be directly useful for the revlogv2 logic. Differential Revision: https://phab.mercurial-scm.org/D10753

run-tests: disable color in 'auto' mode on Windows if stdout is redirected Differential Revision: https://phab.mercurial-scm.org/D10761

run-tests: enable color on Windows In setting up the CI for Windows on heptapod, I noticed it was complaining about color not being enabled because pygments wasn't installed- even though it was. I had initially disabled color on Windows when using Windows 7, because that didn't understand ANSI color codes and made a mess of the output. But now that it's been unsupported for over a year, I don't think we should care about it either. It's admittedly a hack to depend on Mercurial proper to enable color support in the terminal, but I didn't feel like duplicating that code. I'm under the impression that 3rd party stuff is supposed to use this runner in the Mercurial repo instead of using their own copy, so I think it's safe to assume the Mercurial code is available. If it's not, it won't break anything. Differential Revision: https://phab.mercurial-scm.org/D10760

transaction: trivial refactoring Differential Revision: https://phab.mercurial-scm.org/D10714

test: remove some unnecessary dependency on repo format Differential Revision: https://phab.mercurial-scm.org/D10757

hg-cpython: fix new occuring TypeError dd339191f2dc introduced a minor refactoring of error types that highlighted a fragile error creation mechanism that was in place in the Rust `MixedIndex`. `PyErr::from_instance` also accepts a Python class, as long as it's an `Exception` class. Before the aforementioned commit, we never ran into a case where this duck-typing mechanism failed. We rectify this behavior by doing the instantiation ourselves. Differential Revision: https://phab.mercurial-scm.org/D10764

errors: make exit codes class variables instead Kyle pointed out to me that we can simply make the exit codes class variables. Python provides some magic for making them accessible as instance variables. This also makes it easier to let subclasses of existing errors override the exit codes by letting them simply define their own values as class variables. That means that there's no need to pass them into the superclass's constructor arguments, so the superclass doesn't need to expose the them as arguments. (Making a subclass set a different exit code for a subclass of `StorageError` was actually the goal with my recent series.) Differential Revision: https://phab.mercurial-scm.org/D10758