filelog: fix backwards comment for 'backrevref'

filelog: remove trailing "form feed" character

filelog: remove unused variable 'lkr' It's used further down, but it's overwritten before, so it's technically a dead assignment, but unnecessary nevertheless.

branchmap: add seek() to end of file before calling tell() on append open() This is similar to 48c232873a54, which was subsequently modified in 19f5dec2d61f for 2.4. Unexpected test changes on Windows occurred without this.

tests: fix test-casefolding.t output for branchcache This belongs with 7d63398fbfd1. I assume that the failure to read is OK, because there is similar output in test-convert-svn-encoding.t.

setdiscovery: remove '_setupsample' function It is now unused.

setdiscovery: document '_takequicksample'

setdiscovery: drop '_setupsample' usage in '_takequicksample' For '_takefullsample' we can just retrieve the list of head directly and ignore the rest of the complex return values. This was the last call to the infamous '_updatesample' function.

setdiscovery: drop the 'always' argument to '_updatesample' This argument exists because of the complex code flow in '_takequicksample'. It first gets the list of heads and then calls '_updatesample' on an empty initial sample and a size limit matching the differences between the number of heads and the target sample size. Finally the heads and the sample from '_updatesample' were added. To ensure this addition result had the exact target length, the code had to ensure no elements from the heads were added to the '_updatesample' content and therefore was passing this "always included set of heads". Instead we can just update the initial heads sample directly and use the final target size as target size for the update. This removes the need for this 'always' parameter to the '_updatesample' function The test are affected because different set building order results in different random sampling.

setdiscovery: always add exponential sample to the heads As explained in a previous changeset, prioritizing heads too much behaves pathologically when there are more heads than the sample size. To counter this, we always inject exponential samples before reducing to the sample size limit. This already show some benefit in the test themselves, but on a real-world example this moves my discovery for push to pathologically headed repo from 45 rounds to 17 of them. We should maybe ensure that at least 25% of the result sample is heads, but I think the random sampling will be fine in practice.

setdiscovery: directly run '_updatesample' The heads and exponential sample are going to end up in the same set before any extra processing happens. We simplify the code by directly updating a set with heads. Changes in the order the set is built lead to small changes in the random sampling output. But after double checking, I can confirm the input data to the random sampling is consistent.

setdiscovery: stop using '_setupsample' in '_takefullsample' Very few of the return values of '_setupsample' remain in use, so we directly retrieve the value we care about and drop the '_setupsample' call.

setdiscovery: randomly pick between heads and sample when taking full sample Before this changeset, the discovery protocol was too heads-centric. Heads of the undiscovered set were always sent for discovery and any room remaining in the sample were filled with exponential samples (and random ones if any room remained). This behaved extremely poorly when the number of heads exceeded the sample size, because we keep just asking about the existence of heads, then their direct parent and so on. As a result, the 'O(log(len(repo)))' discovery turns into a 'O(len(repo))' one. As a solution we take a random sample of the heads plus exponential samples. This way we ensure some exponential sampling is achieved, bringing back some logarithmic convergence of the discovery again. This patch only applies this principle in one place. More places will be updated in future patches. One test is impacted because the random sample happen to be different. By chance, it helps a bit in this case.

setdiscovery: document the '_updatesample' function This function is central in the sample building process, having it documented help code readability a lot.

setdiscovery: avoid calling any sample building if the undecided set is small If the length of undecided is smaller than the sample size, we can just request information for all of them. This conditional was previously handled by '_setupsample'. But '_setupsample' is in my opinion a problematic function with blurry semantics. Having this conditional explicitly earlier makes the code more explicit and moves us closer to removing this '_setupsample' function.

setdiscovery: delay sample building calls to gather them in a single place Some of the logic around sample building is duplicated in the sample builders, it would clean up thing to extract it in the top function, but this requires all codes to be in the same place. This changeset mostly exists to make the next one more clear.

setdiscovery: drop unused 'initial' argument for '_takequicksample' There is a single call site, and it is always using 'initial=True'. So we just drop the argument and the associated condition.

readmarkers: add a SHA256 fixme note

readmarkers: fast-path single successors and parents This gives about a 5% performance bump.

readmarkers: promote global constants to locals for performance

readmarkers: drop a temporary

readmarkers: read node reading into node length conditional This removes some conditional assignments

readmarkers: drop a temporary Two other temporaries are renamed to fit line-length.

readmarkers: hoist subtraction out of loop comparison

readmarkers: streamline offset tracking This minimizes the number of assignments and operations needed to use offsets.

readmarkers: use unpacker for fixed header

readmarkers: drop metadata temporary

readmarkers: drop date temporary

readmarkers: drop another conditional

readmarkers: drop a conditional