rust-index: disabling flagprocessor tests The list of flags supported by the Rust index is not dynamic, hence flagprocessor has no chance to work.

rust-index: support `unionrepo`'s compressed length hack Explanations inline.

rust-index: honour incoming using_general_delta in `deltachain` It looks to be a leftover from some past, but the C index considers only the value passed from Python whereas up to now the Rust index was using the value of its attribute. As a middle ground, we make this argument of `deltachain` optional from the Python side, with the Rust implementation only defaulting to its attribute. This way, we reduce false leads when a difference in results is spotted.

rust-index: use interior mutability in head revs and caches For upcoming changes in `hg-cpython` switching to the `hg-core` index in ancestors iterators, we will need to avoid excessive mutability, restricting the use of mutable references on `hg::index::Index` to methods that actually logically mutate it, whereas the maintenance of caches such as `head_revs` clearly does not. We illustrate that immediately by switching to immutable borrows in the corresponding methods of `hg-cpython::MixedIndex`

rust-index: add Sync bound to all relevant mmap-derived values All readonly mmaps are Sync as far as Rust is concerned. Integrity of the mmap'ed file is a concern separate to Rust's memory model, since it requires out-of-program handling via locks, etc. This will help when we start sharing the Rust Index with Python.

debugindexstats: handle the lack of Rust support better We don't have any stats in the Rust index. Currently it is not known which stats would be interesting to get, so if they end up being important, we can add them later.

rust-python-index: don't panic on a corrupted index when calling from Python This makes `test-verify.t` pass again. In an ideal world, we would find the exact commit where this test breaks and amend part of this change there, but this is a long enough series.

tests: ignore test-storage when using Rust This is only relevant for Python code and the SQLite backend, which is in a half-abandoned state.

rust-index: optimize find_gca_candidates() on less than 8 revisions This is expected to be by far the most common case, given that, e.g., merging involves using it on two revisions. Using a `u8` as support for the bitset obviously divides the amount of RAM needed by 8. To state the obvious, on a repository with 10 million changesets, this spares 70MB. It is also possible that it'd be slightly faster, because it is easier to allocate and provides better cache locality. It is possible that some exhaustive listing of the traits implemented by `u8` and `u64` would avoid the added duplication, but that can be done later and would need a replacement for the `MAX` consts.

rust-index: simplification in find_gca_candidates() `parent_seen` can be made a mutable ref, making this part more obvious, not needing to be commented so much. The micro-optimization of avoiding the union if `parent_seen` and `current_seen` agree is pushed down in the `union()` method of the fast, `u64` based bit set implementation (in case it matters).

rust-index: avoid double negation in find_gca_candidates()

rust-index: avoid some cloning in find_gca_candidates() Instead of keeping the information whether the current revision is poisoned on `current_seen`, we extract it as a boolean. This also allows us to simplify the explanation of `seen[r].is_poisoned()`, as the exceptional case where it is poisoned right after `r` has been determined to be a solution does no longer exist.

rust-index: implement common_ancestors_heads() and ancestors() The only differences betwwen `common_ancestors_heads()` and `find_gca_candidates()` seems to be that: - the former accepts "overlapping" input revisions (meaning with duplicates). - limitation to 24 inputs (in the C code), that we translate to using the arbitrary size bit sets in the Rust code because we cannot bail to Python. Given that the input is expected to be small in most cases, we take the heavy handed approach of going through a HashSet and wait for perfomance assessment In case this is used via `hg-cpython`, we can anyway absorb the overhead by having `commonancestorheads` build a vector of unique values directly, and introduce a thin wrapper over `find_gca_candidates`, to take care of bit set type dispatching only. As far as `ancestors` is concerneed, this is just chaining `common_ancestors_heads()` with `find_deepest_revs`.

rust-index: find_gca_candidates bit sets genericization This allows to use arbitratry size of inputs in `find_gca_candidates()`. We're genericizing so that the common case of up to 63 inputs can be treated with the efficient implementation backed by `u64`. Some complications with the borrow checker came, because arbitrary sized bit sets will not be `Copy`, hence mutating them keeps a mut ref on the `seen` vector. This is solved by some cloning, most of which can be avoided, preferably in a follow-up after proof that this works (hence after exposition to Python layer). As far as performance is concerned, calling `clone()` on a `Copy` object (good case when number of revs is less than 64) should end up just doing a copy, according to this excerpt of the `Clone` trait documentation: Types that are Copy should have a trivial implementation of Clone. More formally: if T: Copy, x: T, and y: &T, then let x = y.clone(); is equivalent to let x = *y;. Manual implementations should be careful to uphold this invariant; however, unsafe code must not rely on it to ensure memory safety. We kept the general structure, hence why there are some double negations. This also could be made nicer in a follow-up. The `NonStaticPoisonableBitSet` is included to ensure that the `PoisonableBitSet` trait is general enough (had to correct `vec_of_empty()` for instance). Moving the genericization one level to encompass the `seen` vector and not its elements would be better for performance, if worth it.

rust-index: core impl for find_gca_candidates and find_deepest This still follows closely the C original and not able to treat more than 63 input revisions (bitset backed by `u64` and one bit reserved for poisoning).

rust-index: add support for `reachableroots2` Exposition in `hg-cpython` done in regular impl block, again for rustfmt support etc.

hg-cpython: rev_pyiter_collect_or_else It will be useful to give callers the control on the generated errors

rust-index: add support for `computephasesmapsets` Exposition in `hg-cpython` done in the regular `impl` block to enjoy rustfmt and clearer compilartion errors.

rust-index: slicechunktodensity returns Rust result Ready for removal of the scaffolding. This time, we allow ourselves a minor optimization: we avoid allocating for each chunk. Instead, we reuse the same vector, and perform at most one allocation per chunk. The `PyList` constructor will copy the buffer anyway.

rust-index: add support for `_slicechunktodensity`

rust-index: headrevsfiltered() returning Rust result

rust-index: add support for `headrevsfiltered` The implementation is merged with that of `headrevs` also to make sure that caches are up to date.

rust-index: implement headrevs

rust-index: variant of assert_py_eq with normalizer expression The example given in doc-comment is the main use case: some methods may require ordering insensitive comparison. This is about to be used for `reachableroots2`

rust-index: add support for delta-chain computation

rust-index: add support for `find_snapshots`

rust-index: add `is_snapshot` method

rust-index: use the Rust index in `partialmatch`

rust-index: add missing special case for null rev This was an oversight, it was never a problem because we didn't use the index much for user-facing things in the past, which is the only real way of getting to this edge case.

rust-index: use the rust index in `shortest`

rust-index: add checks that `__contains__` is synchronized

rust-index: using the Rust index in nodemap updating methods

rust-index: implementation of __getitem__ Although the removed panic tends to prove if the full test suite did pass that the case when the input is a node id does not happen, it is best not to remove it right now. Raising IndexError is crucial for iteration on the index to stop, given the default CPython sequence iterator, see for instance https://github.com/zpoint/CPython-Internals/blobs/master/BasicObject/iter/iter.md This was spotted by `test-rust-ancestors.py`, which does simple interations on indexes (as preflight checks). In `revlog.c`, `index_getitem` defaults to `index_get` when called on revision numbers, which does raise `IndexError` with the same message as the one we are introducing here.

rust-index: optim note for post-scaffolding removal

rust-index: check that the entry bytes are the same in both indexes This is a temporary measure to show that both the Rust and C indexes are kept in sync. Comes with some related documentation precisions. For comparison of error cases, see `index_entry_binary()` in `revlog.c`.

rust-index: return variables systematic naming convention To help knowing at a glance when a method is ready, making us more comofortable when we are close to the final removal of scaffolding, we introduce the systematic variable names `rust_res` and `c_res`. The goal of this series is to always return the formet. We take again the case of `pack_header` as example. Our personal opinion is to usually avoid such poor semantics as `res`, but usually accept it when it close to the actual return, which will be the case in most methods of this series. Also, the name can simply be dropped when we remove the scaffolding. To follow on the example, the body of `pack_header()` should become this in the final version: ``` let index = self.index(py).borrow(); let packed = index.pack_header(args.get_item(py, 0).extract(py)?); Ok(PyBytes::new(py, &packed).into_object()); ``` in these cases it is close to the actual return and will be removed at the end entirely.

rust-index: results comparison helper with details This is a bit simpler to call and has the advantage of systematically log the encountered deviation. To avoid committing dead code, we apply it to the `pack_header` method, that was already returning the Rust result.

rust-index: helper for revision not in index not involving nodemap This is a good match for exceptions raised from the C implementation, when it is not about a nodemap inconsistency.

rust-index: renamed nodemap error function for rev not in index The function name was misleading, as the error wording mentions the nodemap, hence would not be appropriate for missing revisions not related to a nodemap lookup.

rust-index: add `pack_header` support

rust-index: support cache clearing I'm not 100% sure how useful it is outside of perf, but it's still worth implementing.

rust-index: check rindex and cindex return the same get_rev This is a temporary safeguard while we synchronize both indexes.

rust-index: synchronize remove to Rust index Future steps will bring the two indexes further together until we can rip the C index entirely when running Rust code.

rust-index: remove `__setitem__` method from the mixed index This is not defined on the Python or C one, and isn't used anywhere.

rust-index: check equality between rust and cindex for `__len__`

rust-index: synchronize append method We now append to the Rust index just as we do to the C index. Future steps will bring the two indexes further together until we can rip the C index entirely when running Rust code.

rust-revlog: teach the revlog opening code to read the repo options This will become necessary as we start writing revlog data from Rust.

rust-index: pass data down to the Rust index This will allow us to start keeping the Rust index synchronized with the cindex as we gradually implement more and more methods in Rust. This will eventually be removed.

rust-index: add append method This is the first time the Rust index has any notion of mutability. This will be used in a future patch from Python, to start synchronizing the Rust index and the C index.

rust-index: add an abstraction to support bytes added at runtimes In order to support appending data to the Rust index, we need to abstract data access away from the immutable (on-disk) bytes, to seemlessly fetch either from the preexisting data or from the newly added data.

rust-mixed-index: move the mmap keepalive into a function The same code will be used for keeping the new index mmap around.

rust-mixed-index: rename variable to make the next change clearer We're going to add another mmap reference holder, so let's rename this one first.

rust: fix cargo doc for hg-cpython

branching: merge with default We merge with the current children of the bad merge (37b52b938579)

branching: merge with stable This recreates `37b52b938579` right as a `hg branch --rev 5b186ba40001` screwed up the content.

changelog: disallow delayed write on inline changesets Since this will never happens, we can make the situation invalid and to stop to handling the associated the case. This simplify the random access file reading too.

changelog: never inline changelog The test suite mostly use small repositories, that implies that most changelog in the tests are inlined. As a result, non-inlined changelog are quite poorly tested. Since non-inline changelog are most common case for serious repositories, this lack of testing is a significant problem that results in high profile issue like the one recently fixed by 66417f55ea33 and 849745d7da89. Inlining the changelog does not bring much to the table, the number of total file saved is negligible, and the changelog will be read by most operation anyway. So this changeset is make it so we never inline the changelog, and de-inline the one that are still inlined whenever we touch them. By doing that, we remove the "dual code path" situation for writing new entry to the changelog and move to a "single code path" situation. Having a single code path simplify the code and make sure it is covered by test (if test cover that situation obviously) This impact all tests that care about the number of file and the exchange size, but there is nothing too complicated in them just a lot of churn. The churn is made "worse" by the fact rust will use the persistent nodemap on any changelog now. Which is overall a win as it means testing the persistent nodemap more and having less special cases. In short, having inline changelog is mostly useless and an endless source of pain. We get rid of it.

test-transaction-safety: glog out irrelevant flag The test is focussing on the inline flag, so we glob out the other to highlight that fact and prevent noise in the future.

test-transaction-safety: perform the test on a filelog This test previously checked the transaction safety of splitting the changelog. The changelog is a special case, with delayed/diverted writes and we will stop inlining it soon. So we keep testing that transaction is safe around inline on another revlog type : a filelog. Minor comestic adjustement will be done in the next changesets.

test: clarify test-parseindex offsets We will make this revlog non-inline, so we clarify the code to make sure it is simple to adjust the test later.

test: use more globing for perf timing Not sure why we kept the number here.

branching: merge with stable

statprof: handle `lineno == None` in more cases This continues the work from 972f3e5c94b8. We saw a crash on line 956 but I updated lots of other places as well.

branching: merge stable into default

add: don't attempt to add back removed files unless explicitly listed This fixes the bug demonstrated by the previous patch.

tests: show failure to `hg add -I` a dir->symlink transition

setup: try a non-pure version of the local Mercurial if the pure fails Things like `zstd` can make the pure version fails.

setup: make debug simpler by adding a `__repr__` to `hgcommand` This help when trying to debug this logic.

rhg: support rhg status --rev --rev

rust: add a utility function to merge ordered fallible iterators Adding a function merge_join_results_by, a version of itertools::merge_join_by that works on "fallible" iterators (iterators that can produce errors)

rhg: refactor hg status, make the display code usable for non-dirstate status

debugformat: speedup the "plain-cl-delta" check Checking every single revision is too slow, let's use a sampling approach.

templatekw: fix inconsistency of diffstat with diff.merge Previously, `-T'{diffstat}'` was giving stats from the diff against p1, regardless of whether `--config diff.merge=yes` is set. This was inconsistent with `log --patch` which is aware of that option.

util: move diff_parent from logcmdutil to diffutil This function will be used outside of the log command (in templatekw, used by hgweb, for which logcmdutil is not available). Let's move this function together with the rest of the diff-related utils instead.

logcmdutil: return structured diffstat data for json

cleanup: drop the `bytes` compatibility for attribute related function We can rely on the builtins directly now.

cleanup: turn `wrapfunction` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: turn `wrappedfunction` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: turn `pathsuboption` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: drop deprecated config attribute on the revlog class This code was marked for deletion in the next cycle. We are now in the next cycle.

cleanup: drop `path.pushloc` deprecated since 6.5 This was scheduled for removal, let us remove it.

cleanup: drop `dirstate.is_changing_parent` deprecated since 6.5 This was scheduled for removal, let us remove it.

cleanup: remove some code scheduled to be removed after 5.9 looks like we missed this when cleaning up previous code.

branching: merge stable into default

py3: pass unicode strings to hasattr() throughout I guess we missed these instances because they're not covered by tests.

perf: support --template on perf::phases

debugformat: fix formatting for compression level `bytes(<int>)` gives a very different result as `str(<int>)` and the display of `hg debugformat` have been broken for a while as a result.

tests: tweak chg test to make it fail less often the test apparently sometimes prints the word "start" as a part of profile, so let's no longer match "start": CHGHG=/*/install/bin/hg (glob) + \x1b[90m | 50.0% 0.01s profiling.py: __enter__ line 196: self.start()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s profiling.py: start line 261: self._profiler.__enter__()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s profiling.py: statprofile line 125: statprof.start(mechanism=b'...\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s statprof.py: start line 356: state.thread.start()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s threading.py: start line 852: self._started.wait()\x1b[0m (esc)

cext: fix potential memory leaks of list items appended with PyList_Append Also reduce the duplication in the tricky code that uses PyList_Append by extracting it into a function `pylist_append_owned`.

Added signature for changeset 3fd1efb3ad12

Added tag 6.6.3 for changeset 3fd1efb3ad12

relnotes: add 6.6.3

tests: fix nondeterministic test failure in test-contrib-perf.t It turns out (not too shockingly!) the kernel sometimes has some work to do, perhaps at the very least context-switching, so asserting the system time is 0.000000 doesn't work.

grep: restore usage of --include/--exclude options The refactor in 4a73df6eb67d accidentally forgot to transform the opts argument for walkopts into a byteskwargs. This resulted in its options being ignored. In particular, the -X/-I pair of options was missing. A simple fix restores its usage. Tests included, of course.

rust-changelog: don't panic on empty file lists

tests: use sha256line.py instead of /dev/random in test-censor.t (issue6858) Sometimes the systems that run our test suite don't have enough entropy and they cannot produce target file of the expected size using /dev/random, which results in test failures. Switching to /dev/urandom would give us way more available data at the cost of it being less "random", but we don't really need to use entropy for this task at all, since we only care if the file size after compression is big enough to not be stored inline in the revlog. So let's use something that we already have used to generate this kind of data in other tests.