Mon, 27 Sep 2021 13:52:49 +0200 dirstate: Remove the Rust abstraction DirstateMapMethods
Simon Sapin <simon.sapin@octobus.net> [Mon, 27 Sep 2021 13:52:49 +0200] rev 48069
dirstate: Remove the Rust abstraction DirstateMapMethods This Rust trait used to exist in order to allow the DirstateMap class exposed to Python to be backed by either of two implementations: one similar to the Python implementation based on a "flat" `HashMap<HgPathBuf, DirstateEntry>`, and the newer one based on a tree of nodes matching the directory structure of tracked files. A boxed trait object was used with dynamic dispatch. With the flat implementation removed and only the tree one remaining, this abstraction is not useful anymore and the concrete type can be stored directly. It remains that the trait was implemented separately for `DirstateMap<'_>` (which takes a lifetime parameter) and `OwningDirstateMap` (whose job is to wrap the former and hide the lifetime parameter), with the latter impl only forwarding calls. This changeset also removes this forwarding. Instead, the methods formerly of the `DirstateMapMethods` trait are now inherent methods implemented for `OwningDirstateMap` (where they will actually be used) but in the module that defines `DirstateMap`. This unusual setup gives access to the private fields of `DirstateMap` from those methods. Differential Revision: https://phab.mercurial-scm.org/D11517
Mon, 27 Sep 2021 12:09:15 +0200 dirstate: Remove the flat Rust DirstateMap implementation
Simon Sapin <simon.sapin@octobus.net> [Mon, 27 Sep 2021 12:09:15 +0200] rev 48068
dirstate: Remove the flat Rust DirstateMap implementation Before this changeset we had two Rust implementations of `DirstateMap`. This removes the "flat" DirstateMap so that the "tree" DirstateMap is always used when Rust enabled. This simplifies the code a lot, and will enable (in the next changeset) further removal of a trait abstraction. This is a performance regression when: * Rust is enabled, and * The repository uses the legacy dirstate-v1 file format, and * For `hg status`, unknown files are not listed (such as with `-mard`) The regression is about 100 milliseconds for `hg status -mard` on a semi-large repository (mozilla-central), from ~320ms to ~420ms. We deem this to be small enough to be worth it. The new dirstate-v2 is still experimental at this point, but we aim to stabilize it (though not yet enable it by default for new repositories) in Mercurial 6.0. Eventually, upgrating repositories to dirsate-v2 will eliminate this regression (and enable other performance improvements). # Background The flat DirstateMap was introduced with the first Rust implementation of the status algorithm. It works similarly to the previous Python + C one, with a single `HashMap` that associates file paths to a `DirstateEntry` (where Python has a dict). We later added the tree DirstateMap where the root of the tree contains nodes for files and directories that are directly at the root of the repository, and nodes for directories can contain child nodes representing the files and directly that *they* contain directly. The shape of this tree mirrors that of the working directory in the filesystem. This enables the status algorithm to traverse this tree in tandem with traversing the filesystem tree, which in turns enables a more efficient algorithm. Furthermore, the new dirstate-v2 file format is also based on a tree of the same shape. The tree DirstateMap can access a dirstate-v2 file without parsing it: binary data in a single large (possibly memory-mapped) bytes buffer is traversed on demand. This allows `DirstateMap` creation to take `O(1)` time. (Mutation works by creating new in-memory nodes with copy-on-write semantics, and serialization is append-mostly.) The tradeoff is that for "legacy" repositories that use the dirstate-v1 file format, parsing that file into a tree DirstateMap takes more time. Profiling shows that this time is dominated by `HashMap`. For a dirstate containing `F` files with an average `D` directory depth, the flat DirstateMap does parsing in `O(F)` number of HashMap operations but the tree DirstateMap in `O(F × D)` operations, since each node has its own HashMap containing its child nodes. This slower costs ~140ms on an old snapshot of mozilla-central, and ~80ms on an old snapshot of the Netbeans repository. The status algorithm is faster, but with `-mard` (when not listing unknown files) it is typically not faster *enough* to compensate the slower parsing. Both Rust implementations are always faster than the Python + C implementation # Benchmark results All benchmarks are run on changeset 98c0408324e6, with repositories that use the dirstate-v1 file format, on a server with 4 CPU cores and 4 CPU threads (no HyperThreading). `hg status` benchmarks show wall clock times of the entire command as the average and standard deviation of serveral runs, collected by https://github.com/sharkdp/hyperfine and reformated. Parsing benchmarks are wall clock time of the Rust function that converts a bytes buffer of the dirstate file into the `DirstateMap` data structure as used by the status algorithm. A single run each, collected by running `hg status` this environment variable: RUST_LOG=hg::dirstate::dirstate_map=trace,hg::dirstate_tree::dirstate_map=trace Benchmark 1: Rust flat DirstateMap → Rust tree DirstateMap hg status mozilla-clean 562.3 ms ± 2.0 ms → 462.5 ms ± 0.6 ms 1.22 ± 0.00 times faster mozilla-dirty 859.6 ms ± 2.2 ms → 719.5 ms ± 3.2 ms 1.19 ± 0.01 times faster mozilla-ignored 558.2 ms ± 3.0 ms → 457.9 ms ± 2.9 ms 1.22 ± 0.01 times faster mozilla-unknowns 859.4 ms ± 5.7 ms → 716.0 ms ± 4.7 ms 1.20 ± 0.01 times faster netbeans-clean 336.5 ms ± 0.9 ms → 339.5 ms ± 0.4 ms 0.99 ± 0.00 times faster netbeans-dirty 491.4 ms ± 1.6 ms → 475.1 ms ± 1.2 ms 1.03 ± 0.00 times faster netbeans-ignored 343.7 ms ± 1.0 ms → 347.8 ms ± 0.4 ms 0.99 ± 0.00 times faster netbeans-unknowns 484.3 ms ± 1.0 ms → 466.0 ms ± 1.2 ms 1.04 ± 0.00 times faster hg status -mard mozilla-clean 317.3 ms ± 0.6 ms → 422.5 ms ± 1.2 ms 0.75 ± 0.00 times faster mozilla-dirty 315.4 ms ± 0.6 ms → 417.7 ms ± 1.1 ms 0.76 ± 0.00 times faster mozilla-ignored 314.6 ms ± 0.6 ms → 417.4 ms ± 1.0 ms 0.75 ± 0.00 times faster mozilla-unknowns 312.9 ms ± 0.9 ms → 417.3 ms ± 1.6 ms 0.75 ± 0.00 times faster netbeans-clean 212.0 ms ± 0.6 ms → 283.6 ms ± 0.8 ms 0.75 ± 0.00 times faster netbeans-dirty 211.4 ms ± 1.0 ms → 283.4 ms ± 1.6 ms 0.75 ± 0.01 times faster netbeans-ignored 211.4 ms ± 0.9 ms → 283.9 ms ± 0.8 ms 0.74 ± 0.01 times faster netbeans-unknowns 211.1 ms ± 0.6 ms → 283.4 ms ± 1.0 ms 0.74 ± 0.00 times faster Parsing mozilla-clean 38.4ms → 177.6ms mozilla-dirty 38.8ms → 177.0ms mozilla-ignored 38.8ms → 178.0ms mozilla-unknowns 38.7ms → 176.9ms netbeans-clean 16.5ms → 97.3ms netbeans-dirty 16.5ms → 98.4ms netbeans-ignored 16.9ms → 97.4ms netbeans-unknowns 16.9ms → 96.3ms Benchmark 2: Python + C dirstatemap → Rust tree DirstateMap hg status mozilla-clean 1261.0 ms ± 3.6 ms → 461.1 ms ± 0.5 ms 2.73 ± 0.00 times faster mozilla-dirty 2293.4 ms ± 9.1 ms → 719.6 ms ± 3.6 ms 3.19 ± 0.01 times faster mozilla-ignored 1240.4 ms ± 2.3 ms → 457.7 ms ± 1.9 ms 2.71 ± 0.00 times faster mozilla-unknowns 2283.3 ms ± 9.0 ms → 719.7 ms ± 3.8 ms 3.17 ± 0.01 times faster netbeans-clean 879.7 ms ± 3.5 ms → 339.9 ms ± 0.5 ms 2.59 ± 0.00 times faster netbeans-dirty 1257.3 ms ± 4.7 ms → 474.6 ms ± 1.6 ms 2.65 ± 0.01 times faster netbeans-ignored 943.9 ms ± 1.9 ms → 347.3 ms ± 1.1 ms 2.72 ± 0.00 times faster netbeans-unknowns 1188.1 ms ± 5.0 ms → 465.2 ms ± 2.3 ms 2.55 ± 0.01 times faster hg status -mard mozilla-clean 903.2 ms ± 3.6 ms → 423.4 ms ± 2.2 ms 2.13 ± 0.01 times faster mozilla-dirty 884.6 ms ± 4.5 ms → 417.3 ms ± 1.4 ms 2.12 ± 0.01 times faster mozilla-ignored 881.9 ms ± 1.3 ms → 417.3 ms ± 0.8 ms 2.11 ± 0.00 times faster mozilla-unknowns 878.5 ms ± 1.9 ms → 416.4 ms ± 0.9 ms 2.11 ± 0.00 times faster netbeans-clean 434.9 ms ± 1.8 ms → 284.0 ms ± 0.8 ms 1.53 ± 0.01 times faster netbeans-dirty 434.1 ms ± 0.8 ms → 283.1 ms ± 0.8 ms 1.53 ± 0.00 times faster netbeans-ignored 431.7 ms ± 1.1 ms → 283.6 ms ± 1.8 ms 1.52 ± 0.01 times faster netbeans-unknowns 433.0 ms ± 1.3 ms → 283.5 ms ± 0.7 ms 1.53 ± 0.00 times faster Differential Revision: https://phab.mercurial-scm.org/D11516
Tue, 28 Sep 2021 20:00:19 +0200 dirstate: drop the from_p2_removed method
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 20:00:19 +0200] rev 48067
dirstate: drop the from_p2_removed method It it no longer in use. Differential Revision: https://phab.mercurial-scm.org/D11515
Tue, 28 Sep 2021 19:29:44 +0200 dirstate: inline the `from_p2_removed` logic
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 19:29:44 +0200] rev 48066
dirstate: inline the `from_p2_removed` logic It is used internally for compatibilty with size used in the `v1` format, but this is the only use. So we can simply inline it. Differential Revision: https://phab.mercurial-scm.org/D11514
Tue, 28 Sep 2021 19:15:46 +0200 dirstate: drop the merged_removed method
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 19:15:46 +0200] rev 48065
dirstate: drop the merged_removed method It it no longer in use. Differential Revision: https://phab.mercurial-scm.org/D11513
Tue, 28 Sep 2021 19:12:44 +0200 dirstate: inline the merged_removed logic
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 19:12:44 +0200] rev 48064
dirstate: inline the merged_removed logic It is used internally for compatibilty with size used in the `v1` format, but this is the only use. So we can simply inline it. Differential Revision: https://phab.mercurial-scm.org/D11512
Tue, 28 Sep 2021 18:57:20 +0200 dirstate: drop some safety assert in largefile
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 18:57:20 +0200] rev 48063
dirstate: drop some safety assert in largefile The code involved in `set_possibly_dirty` is now simpler and safe to use even in the cases that the assert covered. So we can drop this assert. It was the last user of `merged_removed` and `from_p2_removed`. Differential Revision: https://phab.mercurial-scm.org/D11511
Tue, 28 Sep 2021 18:29:57 +0200 dirstate: drop unused condition in `from_p2`
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 18:29:57 +0200] rev 48062
dirstate: drop unused condition in `from_p2` This conditional was added (by me) tentatively because "it seemed more correct", but it is not used anywhere yet, and it is missing from the C and the Rust implementation. So it seems more consistent to drop it for now. This effectively backout f94cc63df859c Differential Revision: https://phab.mercurial-scm.org/D11510
Tue, 28 Sep 2021 20:05:37 +0200 dirstate: drop all logic around the "non-normal" sets
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 28 Sep 2021 20:05:37 +0200] rev 48061
dirstate: drop all logic around the "non-normal" sets The dirstate has a lot of code to compute a set of all "non-normal" and "from_other_parent" entries. This is all used in one, unique, location, when `setparent` is called and moved from a merge to a non merge. At that time, any "merge related" information has to be dropped. This is mostly useful for command like `graft` or `shelve` that move to a single-parent state -before- the commit. Otherwise the commit will already have removed all traces of the merge information in the dirstate (e.g. for a regular merges). The bookkeeping for these sets is quite invasive. And it seems simpler to just drop it and do the full computation in the single location where we actually use it (since we have to do the computation at least once anyway). This simplify the code a lot, and clarify why this kind of computation is needed. The possible drawback compared to the previous code are: - if the operation happens in a loop, we will end up doing it multiple time, - the C code to detect entry of interest have been dropped, for now. It will be re-introduced later, with a processing code directly in C for even faster operation. Differential Revision: https://phab.mercurial-scm.org/D11507
Wed, 22 Sep 2021 17:46:29 +0200 dirstate: use a new `drop_merge_data` in `setparent`
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 17:46:29 +0200] rev 48060
dirstate: use a new `drop_merge_data` in `setparent` What is happening in this `setparent` loop is that we remove all `merge` related information when the dirstate is moved out of a `merge` situation. So instead of shuffling state to get them where we want, we simply add a method on the DirstateItem to do drop the information we want dropped. Differential Revision: https://phab.mercurial-scm.org/D11506
Wed, 22 Sep 2021 15:17:12 +0200 dirstate: move parent state handling in the dirstatemap
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 15:17:12 +0200] rev 48059
dirstate: move parent state handling in the dirstatemap This involves dirstatemap data mostly. Moving this one level down will remove the needs for the dirstatemap to expose some of its internals. This will help us to simplify more code further. Differential Revision: https://phab.mercurial-scm.org/D11505
Wed, 22 Sep 2021 09:46:37 +0200 dirstate: stop checking for path collision when adjusting parents
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 09:46:37 +0200] rev 48058
dirstate: stop checking for path collision when adjusting parents This was already checked at a earlier point when adding the file. Differential Revision: https://phab.mercurial-scm.org/D11504
Wed, 22 Sep 2021 15:08:47 +0200 dirstate: drop the `_updatedfiles` set
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 15:08:47 +0200] rev 48057
dirstate: drop the `_updatedfiles` set This is a lot of book keeping for something that was only used to clear ambiguous time. Since this is no no longer in use, we can drop it. Differential Revision: https://phab.mercurial-scm.org/D11503
Wed, 22 Sep 2021 15:23:03 +0200 dirstate: drop the `clearambiguoustimes` method for the map
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 15:23:03 +0200] rev 48056
dirstate: drop the `clearambiguoustimes` method for the map This is no longer called anywhere. Differential Revision: https://phab.mercurial-scm.org/D11502
Wed, 22 Sep 2021 14:54:42 +0200 dirstate: simplify the ambiguity clearing at write time
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 22 Sep 2021 14:54:42 +0200] rev 48055
dirstate: simplify the ambiguity clearing at write time The serialization function is already doing this, so we don't need to do it manually. We just need to propagate the right definition of "now". Differential Revision: https://phab.mercurial-scm.org/D11501
Tue, 28 Sep 2021 09:32:24 -0700 histedit: use more specific exceptions for more detailed exit codes
Martin von Zweigbergk <martinvonz@google.com> [Tue, 28 Sep 2021 09:32:24 -0700] rev 48054
histedit: use more specific exceptions for more detailed exit codes Differential Revision: https://phab.mercurial-scm.org/D11509
Tue, 28 Sep 2021 09:25:05 -0700 histedit: remove redundant checks for unfinished histedit state
Martin von Zweigbergk <martinvonz@google.com> [Tue, 28 Sep 2021 09:25:05 -0700] rev 48053
histedit: remove redundant checks for unfinished histedit state Both text-based and curses-based histedit already check for unfinished operations (not just unfinished histedit), so there's no need to check specifically for unfinished histedit. Differential Revision: https://phab.mercurial-scm.org/D11508
Tue, 28 Sep 2021 13:43:14 +0200 dirstate: Appease pytype
Simon Sapin <simon.sapin@octobus.net> [Tue, 28 Sep 2021 13:43:14 +0200] rev 48052
dirstate: Appease pytype test-check-pytype.t was failing since 98c0408324e6: File "/home/simon/projects/hg/mercurial/dirstatemap.py", line 572, in addfile: unsupported operand type(s) for &: 'size: None' and 'rangemask: int' [unsupported-operands] No attribute '__and__' on 'size: None' or '__rand__' on 'rangemask: int' File "/home/simon/projects/hg/mercurial/dirstatemap.py", line 573, in addfile: unsupported operand type(s) for &: 'mtime: None' and 'rangemask: int' [unsupported-operands] No attribute '__and__' on 'mtime: None' or '__rand__' on 'rangemask: int' `None` is the default value of the `size` and `mtime` parameters of the `addfile` method. However, the relevant lines are only used in a code path where those defaults are never used. These `size` and `mtime` are passed to `DirstateItem.new_normal` which (in the C implementation) calls `dirstate_item_new_normal` which uses: PyArg_ParseTuple(args, "iii", &mode, &size, &mtime) So `None` values would cause an exception to be raised anyway. The new `assert`s only move that exception earlier, and informs pytype that we expect `None` to never happen in this code path. Differential Revision: https://phab.mercurial-scm.org/D11500
Thu, 23 Sep 2021 18:29:40 +0200 dirstate: Pass the final DirstateItem to _rustmap.addfile()
Simon Sapin <simon.sapin@octobus.net> [Thu, 23 Sep 2021 18:29:40 +0200] rev 48051
dirstate: Pass the final DirstateItem to _rustmap.addfile() Now that the Python DirstateItem class wraps a Rust DirstateEntry value, use that value directly instead of converting through v1 data + 5 booleans. Also remove propogating the return value. None of the callers look at it, and it is always None. Differential Revision: https://phab.mercurial-scm.org/D11494
Thu, 23 Sep 2021 15:36:43 +0200 dirstate: Replace dropfile with drop_item_and_copy_source
Simon Sapin <simon.sapin@octobus.net> [Thu, 23 Sep 2021 15:36:43 +0200] rev 48050
dirstate: Replace dropfile with drop_item_and_copy_source Those removing a DirstateItem and a copy source are always done together Differential Revision: https://phab.mercurial-scm.org/D11493
Thu, 23 Sep 2021 15:29:38 +0200 rust: Remove some obsolete doc-comments
Simon Sapin <simon.sapin@octobus.net> [Thu, 23 Sep 2021 15:29:38 +0200] rev 48049
rust: Remove some obsolete doc-comments About parameters that have been removed or replaced Differential Revision: https://phab.mercurial-scm.org/D11492
Thu, 23 Sep 2021 15:26:33 +0200 dirstate: Remove return boolean from dirstatemap.dropfile
Simon Sapin <simon.sapin@octobus.net> [Thu, 23 Sep 2021 15:26:33 +0200] rev 48048
dirstate: Remove return boolean from dirstatemap.dropfile None of the remaining callers use it. Differential Revision: https://phab.mercurial-scm.org/D11491
Wed, 22 Sep 2021 18:56:58 +0200 dirstate: Propagate dirstate-v2 parse errors from set_dirstate_item
Simon Sapin <simon.sapin@octobus.net> [Wed, 22 Sep 2021 18:56:58 +0200] rev 48047
dirstate: Propagate dirstate-v2 parse errors from set_dirstate_item … so that Python sees a proper ValueError instead of only `SystemError: Rust panic` Differential Revision: https://phab.mercurial-scm.org/D11489
Wed, 22 Sep 2021 18:42:00 +0200 dirstate: Don’t drop unrelated data in DirstateMap::set_entry
Simon Sapin <simon.sapin@octobus.net> [Wed, 22 Sep 2021 18:42:00 +0200] rev 48046
dirstate: Don’t drop unrelated data in DirstateMap::set_entry For example, copy source are handled separately. Removing it goes through the `copy_map_remove` method (exposed to Python as `.copymap.pop()`) Differential Revision: https://phab.mercurial-scm.org/D11488
Wed, 22 Sep 2021 18:21:58 +0200 dirstate: Skip no-op conversion in Rust DirstateMap::set_v1
Simon Sapin <simon.sapin@octobus.net> [Wed, 22 Sep 2021 18:21:58 +0200] rev 48045
dirstate: Skip no-op conversion in Rust DirstateMap::set_v1 Now that the `DirstateItem` python class is implemented in Rust containing a `DirstateEntry` value, use that value directly instead of reconstructing it from v1 data. Also rename from `set_v1` since dirstate-v1 data is not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11487
Wed, 22 Sep 2021 11:33:29 +0200 dirstate: Use the Rust implementation of DirstateItem when Rust is enabled
Simon Sapin <simon.sapin@octobus.net> [Wed, 22 Sep 2021 11:33:29 +0200] rev 48044
dirstate: Use the Rust implementation of DirstateItem when Rust is enabled … instead of the C implementation, with C/Rust conversions at the FFI boundary Differential Revision: https://phab.mercurial-scm.org/D11486
Wed, 22 Sep 2021 11:28:52 +0200 rust: Add Python bindings for DirstateEntry as rustext.dirstate.DirstateItem
Simon Sapin <simon.sapin@octobus.net> [Wed, 22 Sep 2021 11:28:52 +0200] rev 48043
rust: Add Python bindings for DirstateEntry as rustext.dirstate.DirstateItem Differential Revision: https://phab.mercurial-scm.org/D11485
Mon, 20 Sep 2021 19:18:21 +0200 rust: Align DirstateEntry internals with Python/C DirstateItem
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 19:18:21 +0200] rev 48042
rust: Align DirstateEntry internals with Python/C DirstateItem This propagate to this Rust struct the similar change that was made recently to the Python classe and C struct. Namely, instead of storing a four-valued `state` field we now store seven (bit-packed) booleans that give lower-level information. Additionally, the marker values -1 and -2 for mtime and size should not be used internally anymore. They are replaced by some combinations of booleans For now, all uses of of `DirstateEntry` still use the compatibility APIs with `state` and marker values. Later the Rust API for DirstateMap will be increasingly updated to the new style. Also change the expected result of the test_non_normal_other_parent_entries unit test. Only a `DirstateEntry` with `size == -2 && mtime != -1` is affected, but this case never occurs outside of unit tests. `size == -2` was the marker value for "from other parent" entries, where no meaningful mtime is stored. Differential Revision: https://phab.mercurial-scm.org/D11484
Tue, 28 Sep 2021 09:40:57 +0200 branching: merge stable into default
Raphaël Gomès <rgomes@octobus.net> [Tue, 28 Sep 2021 09:40:57 +0200] rev 48041
branching: merge stable into default
Tue, 21 Sep 2021 18:18:56 +0200 manifestlog: also monitor `00manifest.n` when applicable stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Sep 2021 18:18:56 +0200] rev 48040
manifestlog: also monitor `00manifest.n` when applicable This let the locarepo's file cache detect outdated nodemap docket and reload the manifestlog after `localrepo.invalidate` when applicable. The same problem than issue6554 could affect the Manifest too. Differential Revision: https://phab.mercurial-scm.org/D11483
Tue, 21 Sep 2021 18:03:37 +0200 changelog: also monitor `00changelog.n` when applicable (issue6554) stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Sep 2021 18:03:37 +0200] rev 48039
changelog: also monitor `00changelog.n` when applicable (issue6554) This let the locarepo's file cache detect outdated nodemap docket and reload the changelog after `localrepo.invalidate` when applicable. Differential Revision: https://phab.mercurial-scm.org/D11482
Tue, 21 Sep 2021 21:18:50 +0200 persistent-nodemap: introduce a test to highlight possible race stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Sep 2021 21:18:50 +0200] rev 48038
persistent-nodemap: introduce a test to highlight possible race Weakness in the current file caching of the changelog means that a writer can end up using an outdated docket. This might result in "committed" persistent-nodemap data from a previous writer to be overwritten by a later writer. This break the strong "append only" assumption of the persistent nodemap and can result in confused reader. The race windows are quite narrow. See the test documentation for details. The issues is fixed in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D11481
Tue, 21 Sep 2021 21:18:44 +0200 test: enable share-safe in test-persistent-nodemap stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Sep 2021 21:18:44 +0200] rev 48037
test: enable share-safe in test-persistent-nodemap We will use some share in this test and I would rather have them safe. Differential Revision: https://phab.mercurial-scm.org/D11480
Tue, 21 Sep 2021 18:02:07 +0200 filecache: abstract the fetching of the list of tracked file stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 Sep 2021 18:02:07 +0200] rev 48036
filecache: abstract the fetching of the list of tracked file We will need it for a coming fix that will requires to check a variable list of file for the changelog. Differential Revision: https://phab.mercurial-scm.org/D11479
Mon, 20 Sep 2021 18:18:15 +0200 persistent-nodemap: use quiet upgrade in tests stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 Sep 2021 18:18:15 +0200] rev 48035
persistent-nodemap: use quiet upgrade in tests This make them less verbose while keeping the important information in the checked output. Differential Revision: https://phab.mercurial-scm.org/D11478
Mon, 20 Sep 2021 18:08:46 +0200 persistent-nodemap: fix a typo in a test comment stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 Sep 2021 18:08:46 +0200] rev 48034
persistent-nodemap: fix a typo in a test comment oops. Differential Revision: https://phab.mercurial-scm.org/D11477
Wed, 22 Sep 2021 17:14:54 -0400 bookmarks: add an option to make pull mirror remote bookmarks
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Wed, 22 Sep 2021 17:14:54 -0400] rev 48033
bookmarks: add an option to make pull mirror remote bookmarks For backups for instance. Merging bookmarks is not a useful behavior in that case. Differential Revision: https://phab.mercurial-scm.org/D11490
Thu, 23 Sep 2021 09:42:20 -0700 errors: use InputError for bad path arguments to `hg annotate`
Martin von Zweigbergk <martinvonz@google.com> [Thu, 23 Sep 2021 09:42:20 -0700] rev 48032
errors: use InputError for bad path arguments to `hg annotate` Differential Revision: https://phab.mercurial-scm.org/D11498
Thu, 23 Sep 2021 09:40:01 -0700 errors: use InputError for bad --similarity value
Martin von Zweigbergk <martinvonz@google.com> [Thu, 23 Sep 2021 09:40:01 -0700] rev 48031
errors: use InputError for bad --similarity value Differential Revision: https://phab.mercurial-scm.org/D11497
Thu, 23 Sep 2021 09:38:45 -0700 errors: use InputError for some invalid revsets and such
Martin von Zweigbergk <martinvonz@google.com> [Thu, 23 Sep 2021 09:38:45 -0700] rev 48030
errors: use InputError for some invalid revsets and such Differential Revision: https://phab.mercurial-scm.org/D11496
Thu, 23 Sep 2021 17:05:37 -0400 merge: with stable
Augie Fackler <augie@google.com> [Thu, 23 Sep 2021 17:05:37 -0400] rev 48029
merge: with stable
Thu, 23 Sep 2021 16:52:35 +0900 dirstate: fix leak of entry object in dirstate_item_from_v1_data()
Yuya Nishihara <yuya@tcha.org> [Thu, 23 Sep 2021 16:52:35 +0900] rev 48028
dirstate: fix leak of entry object in dirstate_item_from_v1_data()
Thu, 23 Sep 2021 16:44:17 +0900 dirstate: fix parse_dirstate() to error out if NULL entry created
Yuya Nishihara <yuya@tcha.org> [Thu, 23 Sep 2021 16:44:17 +0900] rev 48027
dirstate: fix parse_dirstate() to error out if NULL entry created Since 83f0e93ec34b "dirstate-item: move the C implementation to the same logic", dirstate_item_from_v1_data() gets more likely to return NULL, and the fuzzer crashes because of that.
Mon, 20 Sep 2021 20:55:38 +0200 rust: Remove EntryState::Unknown
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 20:55:38 +0200] rev 48026
rust: Remove EntryState::Unknown This enum variant represented the `state == '?'` case, which was used to represent the absence of a dirstate entry/item (and therefore of that entry’s state). Now that previous refactors have removed this use in the Python/Rust FFI APIs, the remaining uses can be removed by replacing `EntryState` by `Option<EntryState>` where appropriate, using `None` to represent the absence of an entry. Differential Revision: https://phab.mercurial-scm.org/D11465
Mon, 20 Sep 2021 20:21:35 +0200 dirstate: Remove the `state == ' '` special case
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 20:21:35 +0200] rev 48025
dirstate: Remove the `state == ' '` special case Previously this was used to create a `DirstateItem` representing dirstate tree nodes that semantically don’t have an associated `DirtateItem`. This isn’t used anymore now that `dirstatemap.debug_iter` yields plain tuples. Differential Revision: https://phab.mercurial-scm.org/D11464
Mon, 20 Sep 2021 20:20:55 +0200 debugsate: Change debug_iter() to yield tuples instead of DirstateItem
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 20:20:55 +0200] rev 48024
debugsate: Change debug_iter() to yield tuples instead of DirstateItem This removes the need for `DirstateItem` to support the `state == ' '` special case which represents dirstate tree nodes without an item. Differential Revision: https://phab.mercurial-scm.org/D11463
Mon, 20 Sep 2021 19:59:09 +0200 debugstate: Always call dirstatemap.debug_iter()
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 19:59:09 +0200] rev 48023
debugstate: Always call dirstatemap.debug_iter() … passing it a new `all` argument for the `--all` CLI option, instead of conditionally calling `.debug_iter()` or `.items()` This prepares for the next commit. Differential Revision: https://phab.mercurial-scm.org/D11462
Fri, 17 Sep 2021 13:33:45 +0200 rust: Make the fields of DirstateEntry private
Simon Sapin <simon.sapin@octobus.net> [Fri, 17 Sep 2021 13:33:45 +0200] rev 48022
rust: Make the fields of DirstateEntry private This is a first step toward making its internal structure equivalent to Python’s DirstateItem. Differential Revision: https://phab.mercurial-scm.org/D11461
Mon, 20 Sep 2021 13:16:36 +0200 rust: Remove support for passing a dict to the Rust pathutil.dirs()
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 13:16:36 +0200] rev 48021
rust: Remove support for passing a dict to the Rust pathutil.dirs() That is only used by the Python dirstatemap, which not used when Rust is enabled. This allows removing the private `extract_dirstate` function which creates `DirstateEntry` values. This in turn will make easier upcoming changes to `DirstateEntry`. Differential Revision: https://phab.mercurial-scm.org/D11460
Mon, 20 Sep 2021 12:52:32 +0200 rust: Remove the `rustext.parsers` module
Simon Sapin <simon.sapin@octobus.net> [Mon, 20 Sep 2021 12:52:32 +0200] rev 48020
rust: Remove the `rustext.parsers` module It only exported Rust implementations of the parse_dirstate and pack_dirtate functions, which are only used (anymore) when Rust is not enabled. fakedirstatewritetime.py was detecting the presence of `rustext.parsers` but what it really wants to know is whether the Rust implementation of `dirstatemap` is used. This changes it to detect `rustext.dirstate` instead. Differential Revision: https://phab.mercurial-scm.org/D11459
Fri, 17 Sep 2021 14:36:54 +0200 dirstate: remove now-unused .directories() method
Simon Sapin <simon.sapin@octobus.net> [Fri, 17 Sep 2021 14:36:54 +0200] rev 48019
dirstate: remove now-unused .directories() method It calls _rustmap.directories() which was already removed Differential Revision: https://phab.mercurial-scm.org/D11458
Fri, 17 Sep 2021 12:42:24 +0200 rust: Move DirstateEntry to its own module
Simon Sapin <simon.sapin@octobus.net> [Fri, 17 Sep 2021 12:42:24 +0200] rev 48018
rust: Move DirstateEntry to its own module … and RawEntry to the dirstate::parsers module, the only one that uses it. Differential Revision: https://phab.mercurial-scm.org/D11457
Mon, 20 Sep 2021 15:16:10 +0200 patchbomb: fix traceback on outgoing when not specifying revisions stable
Raphaël Gomès <rgomes@octobus.net> [Mon, 20 Sep 2021 15:16:10 +0200] rev 48017
patchbomb: fix traceback on outgoing when not specifying revisions A user reported that `hg email -n -o` raised with the following traceback: ``` ** unknown exception encountered, please report by visiting ** https://mercurial-scm.org/wiki/BugTracker ** Python 3.8.12 (default, Sep 13 2021, 22:12:54) [GCC 7.5.0] ** Mercurial Distributed SCM (version 5.9.1) ** Extensions loaded: patchbomb Traceback (most recent call last): File "/usr/pkg/bin/hg", line 61, in <module> dispatch.run() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 144, in run status = dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 250, in dispatch status = _rundispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 294, in _rundispatch ret = _runcatch(req) or 0 File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 470, in _runcatch return _callcatch(ui, _runcatchfunc) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 480, in _callcatch return scmutil.callcatch(ui, func) File "/usr/pkg/lib/python3.8/site-packages/mercurial/scmutil.py", line 153, in callcatch return func() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 460, in _runcatchfunc return _dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1273, in _dispatch return runcommand( File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 918, in runcommand ret = _runcommand(ui, options, cmd, d) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1285, in _runcommand return cmdfunc() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1271, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) File "/usr/pkg/lib/python3.8/site-packages/mercurial/util.py", line 1886, in check return func(*args, **kwargs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 817, in email revs = _getoutgoing(repo, dest, revs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in _getoutgoing safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in <listcomp> safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/mercurial/utils/urlutil.py", line 501, in get_push_paths msg %= dest TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'NoneType' ``` This is due to patchbomb passing `[None]` in the `dests` parameter of the multi-paths outgoing function, we fix this in this change and add a non-regression test. Differential Revision: https://phab.mercurial-scm.org/D11456
Mon, 20 Sep 2021 14:21:18 -0400 extensions: prevent a crash on py3 with a `minimumhgversion` str value
Matt Harbison <matt_harbison@yahoo.com> [Mon, 20 Sep 2021 14:21:18 -0400] rev 48016
extensions: prevent a crash on py3 with a `minimumhgversion` str value The expectation is that this field is bytes, but unported extensions are a thing and it shouldn't explode on a bad value. We already do this transformation in the version reporting mechanism. Differential Revision: https://phab.mercurial-scm.org/D11476
Mon, 20 Sep 2021 14:16:10 -0400 extensions: prevent a crash on py3 when testing a bad extension minimum
Matt Harbison <matt_harbison@yahoo.com> [Mon, 20 Sep 2021 14:16:10 -0400] rev 48015
extensions: prevent a crash on py3 when testing a bad extension minimum A `None` placeholder is populated for each missing component by `util.versiontuple()`, which could safely be used with `>` on py2, but not py3. I guess there's another hole here where if the string is entirely bogus (i.e no numbers), it will be treated as 0.0, and always load. But that's always been the case. Differential Revision: https://phab.mercurial-scm.org/D11475
Mon, 20 Sep 2021 11:22:27 -0400 typing: drop annotations on `mercurial/logcmdutil.walkopts` attributes
Matt Harbison <matt_harbison@yahoo.com> [Mon, 20 Sep 2021 11:22:27 -0400] rev 48014
typing: drop annotations on `mercurial/logcmdutil.walkopts` attributes Pytype 2021-09-09 must have learned some things about `@attr`, because while these were presumably required to help it before (c1d0f83d62c4::9a26fea2b518 and c11099cc1de4::224c786f4fce), these instances now get flagged as an assignment mismatch: File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 717, in walkopts: Type annotation for pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 718, in walkopts: Type annotation for opts does not match type of assignment [annotation-type-mismatch] Annotation: Dict[bytes, Any] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 722, in walkopts: Type annotation for revspec does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 725, in walkopts: Type annotation for bookmarks does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 726, in walkopts: Type annotation for branches does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 727, in walkopts: Type annotation for date does not match type of assignment [annotation-type-mismatch] Annotation: Optional[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 728, in walkopts: Type annotation for keywords does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 729, in walkopts: Type annotation for no_merges does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 730, in walkopts: Type annotation for only_merges does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 731, in walkopts: Type annotation for prune_ancestors does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 732, in walkopts: Type annotation for users does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 735, in walkopts: Type annotation for include_pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 736, in walkopts: Type annotation for exclude_pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 739, in walkopts: Type annotation for follow does not match type of assignment [annotation-type-mismatch] Annotation: int Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 743, in walkopts: Type annotation for force_changelog_traversal does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 747, in walkopts: Type annotation for filter_revisions_by_pats does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 750, in walkopts: Type annotation for sort_revisions does not match type of assignment [annotation-type-mismatch] Annotation: Optional[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 753, in walkopts: Type annotation for limit does not match type of assignment [annotation-type-mismatch] Annotation: Optional[int] Assignment: mercurial.thirdparty.attr._make._CountingAttr Differential Revision: https://phab.mercurial-scm.org/D11474
(0) -30000 -10000 -3000 -1000 -300 -100 -56 +56 +100 +300 +1000 +3000 tip