rust-index: check rindex and cindex return the same get_rev This is a temporary safeguard while we synchronize both indexes.

rust-index: synchronize remove to Rust index Future steps will bring the two indexes further together until we can rip the C index entirely when running Rust code.

rust-index: remove `__setitem__` method from the mixed index This is not defined on the Python or C one, and isn't used anywhere.

rust-index: check equality between rust and cindex for `__len__`

rust-index: synchronize append method We now append to the Rust index just as we do to the C index. Future steps will bring the two indexes further together until we can rip the C index entirely when running Rust code.

rust-revlog: teach the revlog opening code to read the repo options This will become necessary as we start writing revlog data from Rust.

rust-index: pass data down to the Rust index This will allow us to start keeping the Rust index synchronized with the cindex as we gradually implement more and more methods in Rust. This will eventually be removed.

rust-index: add append method This is the first time the Rust index has any notion of mutability. This will be used in a future patch from Python, to start synchronizing the Rust index and the C index.

rust-index: add an abstraction to support bytes added at runtimes In order to support appending data to the Rust index, we need to abstract data access away from the immutable (on-disk) bytes, to seemlessly fetch either from the preexisting data or from the newly added data.

rust-mixed-index: move the mmap keepalive into a function The same code will be used for keeping the new index mmap around.

rust-mixed-index: rename variable to make the next change clearer We're going to add another mmap reference holder, so let's rename this one first.

rust: fix cargo doc for hg-cpython

branching: merge with default We merge with the current children of the bad merge (37b52b938579)

branching: merge with stable This recreates `37b52b938579` right as a `hg branch --rev 5b186ba40001` screwed up the content.

changelog: disallow delayed write on inline changesets Since this will never happens, we can make the situation invalid and to stop to handling the associated the case. This simplify the random access file reading too.

changelog: never inline changelog The test suite mostly use small repositories, that implies that most changelog in the tests are inlined. As a result, non-inlined changelog are quite poorly tested. Since non-inline changelog are most common case for serious repositories, this lack of testing is a significant problem that results in high profile issue like the one recently fixed by 66417f55ea33 and 849745d7da89. Inlining the changelog does not bring much to the table, the number of total file saved is negligible, and the changelog will be read by most operation anyway. So this changeset is make it so we never inline the changelog, and de-inline the one that are still inlined whenever we touch them. By doing that, we remove the "dual code path" situation for writing new entry to the changelog and move to a "single code path" situation. Having a single code path simplify the code and make sure it is covered by test (if test cover that situation obviously) This impact all tests that care about the number of file and the exchange size, but there is nothing too complicated in them just a lot of churn. The churn is made "worse" by the fact rust will use the persistent nodemap on any changelog now. Which is overall a win as it means testing the persistent nodemap more and having less special cases. In short, having inline changelog is mostly useless and an endless source of pain. We get rid of it.

test-transaction-safety: glog out irrelevant flag The test is focussing on the inline flag, so we glob out the other to highlight that fact and prevent noise in the future.

test-transaction-safety: perform the test on a filelog This test previously checked the transaction safety of splitting the changelog. The changelog is a special case, with delayed/diverted writes and we will stop inlining it soon. So we keep testing that transaction is safe around inline on another revlog type : a filelog. Minor comestic adjustement will be done in the next changesets.

test: clarify test-parseindex offsets We will make this revlog non-inline, so we clarify the code to make sure it is simple to adjust the test later.

test: use more globing for perf timing Not sure why we kept the number here.

branching: merge with stable

statprof: handle `lineno == None` in more cases This continues the work from 972f3e5c94b8. We saw a crash on line 956 but I updated lots of other places as well.

branching: merge stable into default

add: don't attempt to add back removed files unless explicitly listed This fixes the bug demonstrated by the previous patch.

tests: show failure to `hg add -I` a dir->symlink transition

setup: try a non-pure version of the local Mercurial if the pure fails Things like `zstd` can make the pure version fails.

setup: make debug simpler by adding a `__repr__` to `hgcommand` This help when trying to debug this logic.

rhg: support rhg status --rev --rev

rust: add a utility function to merge ordered fallible iterators Adding a function merge_join_results_by, a version of itertools::merge_join_by that works on "fallible" iterators (iterators that can produce errors)

rhg: refactor hg status, make the display code usable for non-dirstate status

debugformat: speedup the "plain-cl-delta" check Checking every single revision is too slow, let's use a sampling approach.

templatekw: fix inconsistency of diffstat with diff.merge Previously, `-T'{diffstat}'` was giving stats from the diff against p1, regardless of whether `--config diff.merge=yes` is set. This was inconsistent with `log --patch` which is aware of that option.

util: move diff_parent from logcmdutil to diffutil This function will be used outside of the log command (in templatekw, used by hgweb, for which logcmdutil is not available). Let's move this function together with the rest of the diff-related utils instead.

logcmdutil: return structured diffstat data for json

cleanup: drop the `bytes` compatibility for attribute related function We can rely on the builtins directly now.

cleanup: turn `wrapfunction` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: turn `wrappedfunction` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: turn `pathsuboption` deprecation warning into an error We could simply drop the check, but lets raise explicit error instead of suffering strange error in case of misuse.

cleanup: drop deprecated config attribute on the revlog class This code was marked for deletion in the next cycle. We are now in the next cycle.

cleanup: drop `path.pushloc` deprecated since 6.5 This was scheduled for removal, let us remove it.

cleanup: drop `dirstate.is_changing_parent` deprecated since 6.5 This was scheduled for removal, let us remove it.

cleanup: remove some code scheduled to be removed after 5.9 looks like we missed this when cleaning up previous code.

branching: merge stable into default

py3: pass unicode strings to hasattr() throughout I guess we missed these instances because they're not covered by tests.

perf: support --template on perf::phases

debugformat: fix formatting for compression level `bytes(<int>)` gives a very different result as `str(<int>)` and the display of `hg debugformat` have been broken for a while as a result.

tests: tweak chg test to make it fail less often the test apparently sometimes prints the word "start" as a part of profile, so let's no longer match "start": CHGHG=/*/install/bin/hg (glob) + \x1b[90m | 50.0% 0.01s profiling.py: __enter__ line 196: self.start()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s profiling.py: start line 261: self._profiler.__enter__()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s profiling.py: statprofile line 125: statprof.start(mechanism=b'...\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s statprof.py: start line 356: state.thread.start()\x1b[0m (esc) + \x1b[90m | 50.0% 0.01s threading.py: start line 852: self._started.wait()\x1b[0m (esc)

cext: fix potential memory leaks of list items appended with PyList_Append Also reduce the duplication in the tricky code that uses PyList_Append by extracting it into a function `pylist_append_owned`.

Added signature for changeset 3fd1efb3ad12

Added tag 6.6.3 for changeset 3fd1efb3ad12

relnotes: add 6.6.3

tests: fix nondeterministic test failure in test-contrib-perf.t It turns out (not too shockingly!) the kernel sometimes has some work to do, perhaps at the very least context-switching, so asserting the system time is 0.000000 doesn't work.

grep: restore usage of --include/--exclude options The refactor in 4a73df6eb67d accidentally forgot to transform the opts argument for walkopts into a byteskwargs. This resulted in its options being ignored. In particular, the -X/-I pair of options was missing. A simple fix restores its usage. Tests included, of course.

rust-changelog: don't panic on empty file lists

tests: use sha256line.py instead of /dev/random in test-censor.t (issue6858) Sometimes the systems that run our test suite don't have enough entropy and they cannot produce target file of the expected size using /dev/random, which results in test failures. Switching to /dev/urandom would give us way more available data at the cost of it being less "random", but we don't really need to use entropy for this task at all, since we only care if the file size after compression is big enough to not be stored inline in the revlog. So let's use something that we already have used to generate this kind of data in other tests.

tests: make sha256line.py available for all tests This was previously only used in test-revlog-delta-find.t, but it will be useful (and used) in other tests that might need to generate poorly-compressible files.

tests: don't use "status" operand of dd in test-censor.t (issue6858) Some implementations don't have this operand, let's just direct stderr into /dev/null, that's pretty cross-platform. Also specify bs=512 (the default for me), because the default might be different on different systems. Other uses of dd in the tests do specify it, so this is more consistent.

Added signature for changeset 136902b3a95d

Added tag 6.6.2 for changeset 136902b3a95d

relnotes: add 6.6.2

pycompat: fix bytestr(bytes) in Python 3.11 In Python 3.10, the `bytes` type itself does not have a `__bytes__` attribute, but it does in 3.11. Yet `bytes(bytes)` does not give the wished output, so we have to add an exceptional case. The added case in the doctest reproduces the problem with Python 3.11. Impact: error treatment in expressions such as `repo[b'invalid']` gets broken.

narrow: prevent removal of ACL-defined excludes

narrow: add test demonstrating bug in acl exclusion enforcement

bundle: do not detect --base argument that match nothing as lack of argument With the previous version of the code, if --base did not match anything, it will be handled as if no --base was provided and will fallback to using discovery with the default path. This has two issues : - The resulting bundle won't match what the user requested, - if not default path is configured, it will crash. We now properly distinct between the two cases and if the --base query does not find any changeset, we will assume that everything under --rev needs to be sent.

bundle: highlight misbehavior when --base does not match any revision See next changeset for fix and details.

generate-churning-bundle: fix script for python3 This script has apparently not run for a long time.

persistent-nodemap: respect the mmap setting when refreshing data After writing updated data, we reload the in-memory data. However, that logic was… wrong. We were doing file read when mmap was requested and when the configuration was requesting to not use mmap… we were using it. This should now be fine.

persistent-nodemap: avoid writing nodemap for empty revlog The format cannot encode the lack of tip_rev. There is currently nothing known to write such empty nodemap right now, but the change we are preparing on default reveal this issue. So I had rather fix it on stable.

histedit: remove superfluous echo() and endwin() calls (issue6859) ncurses patchlevel 20231111 started returning an error from endwin() if called twice without a intervening screen update. Per Sven Joachim in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1058041#17: "AFAICS, invoking curses.echo() and curses.endwin() is superfluous because curses.wrapper already does that for you, and calling curses.endwin() twice throws an error with the newer ncurses. Removing those two lines should fix the problem."

Added signature for changeset 71bd09bebbe3

Added tag 6.6.1 for changeset 71bd09bebbe3

relnotes: add 6.6.1

procutil: move stdin assignment outside of try-finally block There is an stdin variable in the global scope of this module. And in the `finally` block of this try-finally statement we're checking `if stdin is not None`. Let's make sure we don't confuse code check tools into thinking we want to use global stdin by moving this line of code outside of `try`. This was caught by pytype 2023.11.21 on Python 3.11.2.

zeroconf: give inet_aton() str instead of bytes All other uses of this function in this extension are already fixed (i.e. use strings instead of bytes). This was caught by pytype 2023.11.21 on Python 3.11.2.

revlog: avoid wrongly updating the data file location on "divert" If we are in the inline case, we need to align the location of the "data" file with the temporary location of the file (i.e. "00changelog.i.a"). However we should not do that for non-inline case… and before this changeset we had been doing it. In addition `index_file` is already a property taking care of updating the "segment file" filename when needed. So we can simply remove all that code. As a result, code trying to read the diverted data before they were committed ended deeply confused as the "00changelog.i.a" file is nothing like the "00changelog.d" file. However nothing corrupted data as all writing where properly handled outside of the "segment file". In "best" cases this small in-memory corruption of the filename when unnoticed until the transaction was committed or rolled back and in the worse case, some data reading was failing during the transaction and resulted in the transaction to be rolled back. However wrong data never reached the disk, so this bug should be have corrupted any repository. This is not catch by tests because most test use a small repository and therefor an inline revlog. In addition the bug only triggers when a changelog read is done in the following "rare" situation: - after some delayed write - after that data have been written in a "divert" file (i.e. `00.changelog.i.a`) - before transaction commit - outside of a "writing" context The issue was introduced in d83d788590a8

revlog: avoid exposing delayed index entry too widely in non-inline revlog Before this change, the index entry would be seen as "appended" to the data file. It did not hurt too much as there are never accessed for reading, but this was odd. So lets stop doing so.

revlog: add one more assert about state of thing when splitting This assert is currently happy, but it does not hurt to adds it to clarify expected state and catch potential error in the future.

censor: fix things around inlining The temporary revlog cannot go through the inline → split process as this would break at transaction commit. (that might be fixable, but lets keep things simple for now). We introduce a cleaner way to enforce this as the previous one was broken in 6.6 On the way we remove multiple weird, fragile and broken overwrite of revlog attributes and we focus on passing the configuration across. We also had to update the test to actually create a non-inline revlog.

revlog: add a `may_inline` argument to revlog This allow for a clean skipping of the inline feature when needed, for example by censor.

revlog: allow explicit passing of config to revlog This will be useful to fix censor in a later changeset.

censor: show that the `not-inline` → `inline` test is broken The source revlog should not be inlined and it is…

perf-tags: fix clear_cache_fnodes to actually clear that cache The function was not doing it what it advertise for a long time. So we fix it and we add a way for the perf extensions to detect broken version.

perf-tags: fix the --clear-fnode-cache-rev code It seems like this code never run?

tests: do not fail tests in a state with uncommitted .py file removal The problem is that [hg locate] lists removed files too. We use [hg files] instead because that does not list removed files.

tests: fill in the Windows pattern for `$EADDRNOTAVAIL$` matching This fixes test-https.t on Windows. It looks like the real error translation is "Cannot assign requested address.", and the message here is the start of a longer description, so I'm not sure why this part is emitted. But it's not worth digging into, as it's evidently the same failure.

tests: avoid a cascading failure on Windows The `identify --debug` command here on Windows emits skip updating dirstate: identity mismatch because of the debug switch, which got captured and added to `.hgtags`, and then hijinx ensued. The point of `--debug` seemed to be to get the long hash, so just do that with templating. I have not idea if the message is indicating a problem- there seems to be many more of them in other tests that are not present on Linux.

phabricator: stringify the argument to `getattr()` This fixes a "TypeError: getattr(): attribute name must be string" crash and subsequent test spew. This likely broke in 18c8c18993f0 and was missed in CI because `pytest-vcr` is missing.

py3: pass unicode strings to hasattr() throughout I guess we missed these instances because they're not covered by tests. This is a graft of 1625fe807c04a490f9516bc8e14140e570c06146 that landed on default.

relnotes: add 6.6 There are no differences between the rc and the actual release.

Added signature for changeset 26c57e7a0890

Added tag 6.6 for changeset 26c57e7a0890

Added signature for changeset 27055614b685

Added tag 6.6rc0 for changeset 27055614b685

relnotes: add 6.6rc0

branching: merge default into stable for 6.6rc0

unstable: do not consider internal phases when computing unstable The revisions that are not part of the "working" set by other means should not be considered for the evolution related computation. This impact the test introduced in 5f9af8422b31 as this is actually a more semantic fix of the issue.

unstable: use the `_mutablerevs` function when computing content divergent This is a useful function to get the revision relevant to these computation, lets make sure all code use it, so that we can improve that `_mutablerevs` function in a later changeset.

unstable: use the `_mutablerevs` function when computing phase divergent This is a useful function to get the revision relevant to these computation, lets make sure all code use it, so that we can improve that `_mutablerevs` function in a later changeset.

rust: add explicit resolver field to top-level cargo manifest file Since Rust 1.51.0, Cargo has support for a better dependency resolver. Workspace packages (like ours) need to explicitely set the field, so let's. https://doc.rust-lang.org/edition-guide/rust-2021/default-cargo-resolver.html

rust: run a clippy pass with the latest stable version Our current version of clippy is older than the latest stable. The newest version has new lints that are moslty good advice, so let's apply them ahead of time. This has the added benefit of reducing the noise for developpers like myself that use clippy as an IDE helper, as well as being more prepared for a future clippy upgrade.

rust-clippy: ignore clippy's recommendation for "useless" cast See explanation inline.

branching: merge stable into default

smartset: don't ignore hidden revs when intersecting This fixes the bug I demonstrated in the previous commit, but I'm not sure at all if it's the right way of doing it.

tests: demonstrate crash in `unstable()` with internal-phase orphans

rust-matchers: fix quadratic complexity in `FileMatcher` Concretely, this command: ``` $ echo hg up -r <nodeid>; time hg revert dir1 dir2 -r <othernode> --debug hg up -r <nodeid> real 0m14.690s user 0m14.766s sys 0m5.430s ``` was much slower despite using 16 cores before this change. The approach taken here is the same one used in match.py, in exactmatcher. This changeset was originally written by Valentin Gatien-Baron in a private repository. I have redacted the commit message and did a minor clean up of the code.

revlog: add a small cache of unfiltered chunk This can provides a massive boost to the reading of multiple revision and the computation of a valid delta chain. This greatly help operation like `hg log --patch`, delta computation (helping pull/unbundle), linkrev adjustment (helping copy tracing). A first round of benchmark for `hg log --patch --limit 1000` shows improvement in the 10-20% range on "small" repository like pypy or mercurial and large improvements (about 33%) for more complex ones like netbeans and mozilla's. These speeds up are consistent with the improvement to `hg pull` (from a server sending poor deltas) I saw benchmarking this last year. Further benchmark will be run during the freeze. I added some configuration in the experimental space to be able to further test the effect of various tuning for now. This feature should fit well in the "usage/resource profile" configuration that we should land next cycle. When it does not provides a benefit the overhead of the cache seem to be around 2%, a small price for the big improvement. In addition I believe we could shave most of this overhead with a more efficent lru implementation.

revlog: minor refactor in the chunk gather process We will introduce some caching in this method in the next changeset, we make some of the most "disruptive" change first as touching this could break (and maybe did during the development process).

changelog-delay: move the delay/divert logic inside the (inner) revlog Instead of hacking throught the vfs/opener, we implement the delay/divert logic inside the `_InnerRevlog` and `randomaccessfile` object. This will allow to an alternative implementation of the `_InnerRevlog` that does not need to use Python details. As a result, the new implementation can use the transaction less agressively and avoid some extra output since no data had been written yet. That seems like a good side effect.

revlog: add a `canonical_index_file` attribute on inner revlog This is currently the same and the index_file but it will become more complex when we start doing delayed write.

changelog-delay: move "delayed" check to a more official API To avoid reaching inside the inner object in the future, we needs some official API. We put one of such API early to reduce the size of the final diff.

changelog-delay: move the appender class next to randomaccessfile We want to move the delay/divert logic at the revlog level (to have all IO related logic in the _InnerRevlog) we start with small piece that are easy to move on their own.

changelog-delay: adds some check around delaying and diverting write Theses assert shows we never call delay or divert if the revlog hold file handle on the revlog's file.

revlog: consolidate cache invalidation within the inner objet The invalidation of data that belong to the inner object can and should be delegated to the inner object.

shelve: drop some weird manually stripping before transaction abort I cannot find a case where this is actually useful, so lets triggering strip in the middle of uncommitted transaction.

revlog: move entry writing in the inner object This is the second big piece of API we want to live on the inner object. With this the inner object have freedom it needs to implement things internally. (except for the delayed write feature of the changelog, that will be migrated in coming changesets)

revlog: move `sidedata` in the inner object Same object as rawtext, but for sidedata.

revlog: move the `rawtext` method on the inner object This is one of the current core goal of the inner object: having a "simple" API for some of the core revlog action, like getting a raw text, with the inner object able to do that part internally, on its own.

revlog: move the_revisioncache on the inner object The goal for this inner object is to compute that things we cache, it make more sense to have the inner object handle it directly.

revlog: move the `deltachain` method on the inner object This is a necessary step before being able to move more logic around restoring a revision content there. For now, we do a simple patch for the perf extension logic, when the implementation of the inner object changes, we will likely need some evolution of the API. However this is true of many things in the perf extension. So we will see this later.

revlog: move the `_chunks` method on the inner object This is a necessary step before being able to move more logic around restoring a revision content there. For now, we do a simple patch for the perf extension logic, when the implementation of the inner object changes, we will likely need some evolution of the API. However this is true of many things in the perf extension. So we will see this later.