annotate rust/README.rst @ 43271:99394e6c5d12

rust-dirstate-status: add first Rust implementation of `dirstate.status` Note: This patch also added the rayon crate as a Cargo dependency. It will help us immensely in making Rust code parallel and easy to maintain. It is a stable, well-known, and supported crate maintained by people on the Rust team. The current `dirstate.status` method has grown over the years through bug reports and new features to the point where it got too big and too complex. This series does not yet improve the logic, but adds a Rust fast-path to speed up certain cases. Tested on mozilla-try-2019-02-18 with zstd compression: - `hg diff` on an empty working copy: - c: 1.64(+-)0.04s - rust+c before this change: 2.84(+-)0.1s - rust+c: 849(+-)40ms - `hg commit` when creating a file: - c: 5.960s - rust+c before this change: 5.828s - rust+c: 4.668s - `hg commit` when updating a file: - c: 4.866s - rust+c before this change: 4.371s - rust+c: 3.855s - `hg status -mard` - c: 1.82(+-)0.04s - rust+c before this change: 2.64(+-)0.1s - rust+c: 896(+-)30ms The numbers are clear: the current Rust `dirstatemap` implementation is super slow, its performance needs to be addressed. This will be done in a future series, immediately after this one, with the goal of getting Rust to be at least to the speed of the Python + C implementation in all cases before the 5.2 freeze. At worse, we gate dirstatemap to only be used in those cases. Cases where the fast-path is not executed: - for commands that need ignore support (`status`, for example) - if subrepos are found (should not be hard to add, but winter is coming) - any other matcher than an `alwaysmatcher`, like patterns, etc. - with extensions like `sparse` and `fsmonitor` The next step after this is to rethink the logic to be closer to Jane Street's Valentin Gatien-Baron's Rust fast-path which does a lot less work when possible. Differential Revision: https://phab.mercurial-scm.org/D7058
author Raphaël Gomès <rgomes@octobus.net>
date Fri, 11 Oct 2019 13:39:57 +0200
parents 964212780daf
children 8a3b045d9086
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35569
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 ===================
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 Mercurial Rust Code
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 ===================
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 This directory contains various Rust code for the Mercurial project.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7 The top-level ``Cargo.toml`` file defines a workspace containing
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 all primary Mercurial crates.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 Building
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 ========
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 To build the Rust components::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15 $ cargo build
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 If you prefer a non-debug / release configuration::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 $ cargo build --release
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 Features
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 --------
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24 The following Cargo features are available:
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 localdev (default)
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 Produce files that work with an in-source-tree build.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29 In this mode, the build finds and uses a ``python2.7`` binary from
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 ``PATH``. The ``hg`` binary assumes it runs from ``rust/target/<target>hg``
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31 and it finds Mercurial files at ``dirname($0)/../../../``.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 Build Mechanism
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34 ---------------
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 The produced ``hg`` binary is *bound* to a CPython installation. The
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37 binary links against and loads a CPython library that is discovered
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38 at build time (by a ``build.rs`` Cargo build script). The Python
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 standard library defined by this CPython installation is also used.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 Finding the appropriate CPython installation to use is done by
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 the ``python27-sys`` crate's ``build.rs``. Its search order is::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44 1. ``PYTHON_SYS_EXECUTABLE`` environment variable.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 2. ``python`` executable on ``PATH``
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46 3. ``python2`` executable on ``PATH``
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47 4. ``python2.7`` executable on ``PATH``
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 Additional verification of the found Python will be performed by our
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50 ``build.rs`` to ensure it meets Mercurial's requirements.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
51
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52 Details about the build-time configured Python are built into the
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 produced ``hg`` binary. This means that a built ``hg`` binary is only
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
54 suitable for a specific, well-defined role. These roles are controlled
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
55 by Cargo features (see above).
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
56
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
57 Running
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
58 =======
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 The ``hgcli`` crate produces an ``hg`` binary. You can run this binary
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61 via ``cargo run``::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63 $ cargo run --manifest-path hgcli/Cargo.toml
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 Or directly::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
67 $ target/debug/hg
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 $ target/release/hg
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
69
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 You can also run the test harness with this binary::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 $ ./run-tests.py --with-hg ../rust/target/debug/hg
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74 .. note::
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 Integration with the test harness is still preliminary. Remember to
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 ``cargo build`` after changes because the test harness doesn't yet
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78 automatically build Rust code.