rust-dirstate: call rust dirstatemap from Python Since Rust-backed Python classes cannot be used as baseclasses (for rust-cpython anyway), we use composition rather than inheritance. This also allows us to keep the IO operations in the Python side, removing (for now) the need to rewrite VFS in Rust, which would be a heavy undertaking. Differential Revision: https://phab.mercurial-scm.org/D6634

rust-dirstate: rust-cpython bridge for dirstatemap This change also showcases the limitations of the `py_shared_ref!` macro. See the previous commit 'rust-dirstate: rust implementation of dirstatemap` for an explanation for the TODOs in the code. Differential Revision: https://phab.mercurial-scm.org/D6633

rust-dirstate: rust implementation of dirstatemap The `dirstatemap` is one of the last building blocks needed to get to a `dirstate.walk` Rust implementation. Disclaimer: This change is part of a big (10) series of patches, all of which started as one big changeset that took a long time to write. This `dirstatemap` implementation is a compromise in terms of complexity both for me and for the reviewers. I chose to submit this patch right now because while it is not perfect, it works and is simple enough (IMHO) to be reviewed. The Python implementation uses a lot of lazy propertycaches, breaks encapsulation and is used as an iterator in a lot of places, all of which dictated the somewhat unidiomatic patterns in this change. Like written in the comments, rewriting this struct to use the typestate pattern might be a good idea, but this is a good first step. Differential Revision: https://phab.mercurial-scm.org/D6632

rust-cpython: add macro for sharing references Following an experiment done by Georges Racinet, we now have a working way of sharing references between Python and Rust. This is needed in many points of the codebase, for example every time we need to expose an iterator to a Rust-backed Python class. In a few words, references are (unsafely) marked as `'static` and coupled with manual reference counting; we are doing manual borrow-checking. This changes introduces two declarative macro to help reduce boilerplate. While it is better than not using macros, they are not perfect. They need to: - Integrate with the garbage collector for container types (not needed as of yet), as stated in the docstring - Allow for leaking multiple attributes at the same time - Inject the `py_shared_state` data attribute in `py_class`-generated structs - Automatically namespace the functions and attributes they generate For at least the last two points, we will need to write a procedural macro instead of a declarative one. While this reference-sharing mechanism is being ironed out I thought it best not to implement it yet. Lastly, and implementation detail renders our Rust-backed Python iterators too strict to be proper drop-in replacements, as will be illustrated in a future patch: if the data structure referenced by a non-depleted iterator is mutated, an `AlreadyBorrowed` exception is raised, whereas Python would allow it, only to raise a `RuntimeError` if `next` is called on said iterator. This will have to be addressed at some point. Differential Revision: https://phab.mercurial-scm.org/D6631

rust-docstrings: add missing module docstrings Differential Revision: https://phab.mercurial-scm.org/D6630

rust-dirstate: improve API of `DirsMultiset` - Use opaque `Iterator` type instead of implementation-specific one from `HashMap` - Make `DirsMultiset` behave like a set both in Rust and from Python Differential Revision: https://phab.mercurial-scm.org/D6690

rust-dirstate: use EntryState enum instead of literals This improves code readability quite a bit, while also adding a layer of safety because we're checking the state byte against the enum. Differential Revision: https://phab.mercurial-scm.org/D6629

rust-parsers: switch to parse/pack_dirstate to mutate-on-loop Both `parse_dirstate` and `pack_dirstate` can operate directly on the data they're passed, which prevents the creation of intermediate data structures, simplifies the function signatures and reduces boilerplate. They are exposed directly to the Python for now, but a later patch will make use of them inside `hg-core`. Differential Revision: https://phab.mercurial-scm.org/D6628

rust-parsers: move parser bindings to their own file and Python module This tidies up the Rust side while simplifying the Python side. Differential Revision: https://phab.mercurial-scm.org/D6627

rust-dirstate: create dirstate submodule in hg-cpython This module will soon hold multiple files, this change is to make the review process easier. Differential Revision: https://phab.mercurial-scm.org/D6626