posix: always seek to EOF when opening a file in append mode Python 3 already does this, so skip it there. Consider the program: #include <stdio.h> int main() { FILE *f = fopen("narf", "w"); fprintf(f, "narf\n"); fclose(f); f = fopen("narf", "a"); printf("%ld\n", ftell(f)); fprintf(f, "troz\n"); printf("%ld\n", ftell(f)); return 0; } on macOS, FreeBSD, and Linux with glibc, this program prints 5 10 but on musl libc (Alpine Linux and probably others) this prints 0 10 By my reading of https://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html this is technically correct, specifically: > Opening a file with append mode (a as the first character in the > mode argument) shall cause all subsequent writes to the file to be > forced to the then current end-of-file, regardless of intervening > calls to fseek(). in other words, the file position doesn't really matter in append-mode files, and we can't depend on it being at all meaningful unless we perform a seek() before tell() after open(..., 'a'). Experimentally after a .write() we can do a .tell() and it'll always be reasonable, but I'm unclear from reading the specification if that's a smart thing to rely on. This matches what we do on Windows and what Python 3 does for free, so let's just be consistent. Thanks to Yuya for the idea.

move: --force flag forcibly moves, not copies

copy: correct synopsis by making SOURCE a required argument

debugrevlog: fix average size computation for empty data (issue6167) If the file has no full snapshot (eg: was always empty), `hg debugrevlog` would fails when trying to compute their average size.

bookmarks: backout the attempt to fix the delete race This backs out 044045dce23a because it broke a bunch of tests on Windows. Yuya's theory is that we still rely on in-memory changelog data to be flushed out of the transaction.

help: add a missing blank line to unhide `revlog-compression` The help was output, but it was elided with "Enabled by default" from the previous item.

bookmarks: actual fix for race condition deleting bookmark This is a simple but efficient fix to prevent the issue tested in `test-bookmarks-corner-case.t`. It might be worth pursuing a more generic approach where filecache learn to depend on each other, but that would not be suitable for stable. The issue is complicated enough that I documented the race and its current solution as inline comment. See this comment for details on the fix.

localrepo: introduce a `_refreshchangelog` method See next changeset for usage and documentation for details.

bookmarks: actually trigger the race deleting bookmark in the test The previous committed version of the test did not triggered the race, but this was hidden by a strange behavior from the test runner. So we are moving the test to a slightly more complex that actually trigger the issue.

test: add some assert in the bookrace extension This cannot hurt to have a bit more security in the test extension.

test: factor out the "wait" logic in bookrace The test is currently not testing the race it is supposed to test. The synchronisation is still valid, but needs to run at a different point. We start with extracting the synchronisation logic for clarity.

test: remove dead code in the bookrace extension This code is the remain of a previous version of the code. It is never ran, so we can remove it.

run-tests: stop matching line for missing feature Before this change, the following unified test input would silently pass $ echo foo foo (false !) After this change, the "foo" output is properly detected as unexpected. The output of an handful of test had to be updated from broken conditional (that ended up working by chance).

cborutil: fix streamencode() to handle subtypes Otherwise the template filter 'cbor' could crash because of bytes subclass: ValueError: do not know how to encode <class 'mercurial.encoding.safelocalstr'>

bookmarks: use correct store for "ambiguity check" I still don't quite know what the check does, but I clearly got it wrong in 526750cdd02d (bookmarks: keep bookmarks in .hg/store if new config set, 2019-05-15). Just compare with the strings we use in @repofilecache and @storecache. These bugs were then copied to the stable branch in c2b83c957621 (localrepo: grab mixedrepostorecache class from 526750cdd02d, 2019-05-20) and 2338bdea4474 (bookmark: also make bookmark cache depends of the changelog, 2019-05-20). As a result, test-wireproto-exchangev2.t is flaky on both branches. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D6469

Added signature for changeset c3484ddbdb96

Added tag 5.0.1 for changeset c3484ddbdb96

manifest: add some documentation to _lazymanifest python code It was not particularly easy figuring out the design of this class and keeping track of how the pieces work. So might as well write some of it down for the next person.

manifest: avoid corruption by dropping removed files with pure (issue5801) Previously, removed files would simply be marked by overwriting the first byte with NUL and dropping their entry in `self.position`. But no effort was made to ignore them when compacting the dictionary into text form. This allowed them to slip into the manifest revision, since the code seems to be trying to minimize the string operations by copying as large a chunk as possible. As part of this, compact() walks the existing text based on entries in the `positions` list, and consumed everything up to the next position entry. This typically resulted in a ValueError complaining about unsorted manifest entries. Sometimes it seems that files do get dropped in large repos- it seems to correspond to there being a new entry that would take the same slot. A much more trivial problem is that if the only changes were removals, `_compact()` didn't even run because `__delitem__` doesn't add anything to `self.extradata`. Now there's an explicit variable to flag this, both to allow `_compact()` to run, and to avoid searching the manifest in cases where there are no removals. In practice, this behavior was mostly obscured by the check in fastdelta() which takes a different path that explicitly drops removed files if there are fewer than 1000 changes. However, timeless has a repo where after rebasing tens of commits, a totally different path[1] is taken that bypasses the change count check and hits this problem. [1] https://www.mercurial-scm.org/repo/hg/file/2338bdea4474/mercurial/manifest.py#l1511

tests: demonstrate broken manifest generation with the pure module This will be fixed next. But I don't fully understand how 'b.txt' is actually removed properly in the second test, given what's broken. Also, I'm not sure why 'bb.txt' is flagged as not being in the manifest, when it clearly appears to be.

bookmark: also make bookmark cache depends of the changelog Since the changelog is also used during the parsing of bookmark data, it should be listed as a file cache dependency. This fix the race condition we just introduced a test for. This is a simple fix that might lead bookmark data to be invalidated more often than necessary. We could have more complicated code to deal with this race in a more "optimal" way. I feel it would be unsuitable for stable. In addition, the performance impact of this is probably minimal and I don't foresee the more advanced fix to actually be necessary.

localrepo: grab mixedrepostorecache class from 526750cdd02d On default, Martin von Zweigbergk <martinvonz@google.com> introduced a more advance filecache decorator. I need this decorator to fix a bug on stable. So I am grafting the relevant part of 526750cdd02d.

bookmark: add a test for a race condition on push Bookmark pointing to unknown nodes are ignored. Later these ignored bookmarks are dropped when writing the file back on disk. On paper, this behavior should be fine, but with the current implementation, it can lead to unexpected bookmark deletions. In theory, to make sure writer as a consistent view, taking the lock also invalidate bookmark data we already loaded into memory. However this invalidation is incomplete. The data are stored in a `filecache` that preserve them if the bookmark related file are untouched. In practice, the bookmark data in memory also depends of the changelog content, because of the step checking if the bookmarks refers to a node known to the changelog. So if the bookmark data were loaded from an up to date bookmark file but filtered with an outdated changelog file this go undetected. This condition is fairly specific, but can occurs very often in practice. We introduce a test recreating the situation. The test comes in an independant changeset to show it actually reproduce the situation. The fix will come soon after. A large share of the initial investigation of this race condition was made by Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>.

test: properly gate a zstd section This part of the test can't run if zstd is not available. This was caught by --pure test (who don't support zstd).

test: update test for expected test output In 1fac9b931d46 as new test session was introduced. It did not take in account some part that only ran for pure. The test is now fixed.

sslutil: fsencode path returned by certifi (issue6132) By inspection, this is the only codepath that could be returning a string instead of a bytes on Python 3.

commit: allow --interactive to work again when naming a directory (issue6131)

parser: fix crash by parsing "()" in keyword argument position A tree node can be either None or a tuple because x=("group", None) is reduced to x[1].

Added signature for changeset 07e479ef7c96

Added tag 5.0 for changeset 07e479ef7c96