Yuya Nishihara <yuya@tcha.org> [Sun, 16 Jun 2019 12:31:07 +0900] rev 42176
cborutil: fix streamencode() to handle subtypes
Otherwise the template filter 'cbor' could crash because of bytes subclass:
ValueError: do not know how to encode
<class 'mercurial.encoding.safelocalstr'>
Martin von Zweigbergk <martinvonz@google.com> [Fri, 31 May 2019 22:37:14 -0700] rev 42175
bookmarks: use correct store for "ambiguity check"
I still don't quite know what the check does, but I clearly got it
wrong in 526750cdd02d (bookmarks: keep bookmarks in .hg/store if new
config set, 2019-05-15). Just compare with the strings we use in
@repofilecache and @storecache. These bugs were then copied to the
stable branch in c2b83c957621 (localrepo: grab mixedrepostorecache
class from 526750cdd02d, 2019-05-20) and 2338bdea4474 (bookmark: also
make bookmark cache depends of the changelog, 2019-05-20). As a
result, test-wireproto-exchangev2.t is flaky on both branches. This
patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6469
Augie Fackler <raf@durin42.com> [Wed, 05 Jun 2019 10:14:19 -0400] rev 42174
Added signature for changeset c3484ddbdb96
Augie Fackler <raf@durin42.com> [Wed, 05 Jun 2019 10:14:18 -0400] rev 42173
Added tag 5.0.1 for changeset c3484ddbdb96
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 22:50:11 -0400] rev 42172
manifest: add some documentation to _lazymanifest python code
It was not particularly easy figuring out the design of this class and keeping
track of how the pieces work. So might as well write some of it down for the
next person.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:54:24 -0400] rev 42171
manifest: avoid corruption by dropping removed files with pure (issue5801)
Previously, removed files would simply be marked by overwriting the first byte
with NUL and dropping their entry in `self.position`. But no effort was made to
ignore them when compacting the dictionary into text form. This allowed them to
slip into the manifest revision, since the code seems to be trying to minimize
the string operations by copying as large a chunk as possible. As part of this,
compact() walks the existing text based on entries in the `positions` list, and
consumed everything up to the next position entry. This typically resulted in
a ValueError complaining about unsorted manifest entries.
Sometimes it seems that files do get dropped in large repos- it seems to
correspond to there being a new entry that would take the same slot. A much
more trivial problem is that if the only changes were removals, `_compact()`
didn't even run because `__delitem__` doesn't add anything to `self.extradata`.
Now there's an explicit variable to flag this, both to allow `_compact()` to
run, and to avoid searching the manifest in cases where there are no removals.
In practice, this behavior was mostly obscured by the check in fastdelta() which
takes a different path that explicitly drops removed files if there are fewer
than 1000 changes. However, timeless has a repo where after rebasing tens of
commits, a totally different path[1] is taken that bypasses the change count
check and hits this problem.
[1] https://www.mercurial-scm.org/repo/hg/file/2338bdea4474/mercurial/manifest.py#l1511
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:39:19 -0400] rev 42170
tests: demonstrate broken manifest generation with the pure module
This will be fixed next. But I don't fully understand how 'b.txt' is actually
removed properly in the second test, given what's broken. Also, I'm not sure
why 'bb.txt' is flagged as not being in the manifest, when it clearly appears
to be.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:28 +0200] rev 42169
bookmark: also make bookmark cache depends of the changelog
Since the changelog is also used during the parsing of bookmark data, it should
be listed as a file cache dependency. This fix the race condition we just
introduced a test for.
This is a simple fix that might lead bookmark data to be invalidated more often
than necessary. We could have more complicated code to deal with this race in a
more "optimal" way. I feel it would be unsuitable for stable.
In addition, the performance impact of this is probably minimal and I don't
foresee the more advanced fix to actually be necessary.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:17 +0200] rev 42168
localrepo: grab mixedrepostorecache class from 526750cdd02d
On default, Martin von Zweigbergk <martinvonz@google.com> introduced a more
advance filecache decorator. I need this decorator to fix a bug on stable. So I
am grafting the relevant part of 526750cdd02d.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:06:53 +0200] rev 42167
bookmark: add a test for a race condition on push
Bookmark pointing to unknown nodes are ignored. Later these ignored bookmarks
are dropped when writing the file back on disk. On paper, this behavior should
be fine, but with the current implementation, it can lead to unexpected
bookmark deletions.
In theory, to make sure writer as a consistent view, taking the lock also
invalidate bookmark data we already loaded into memory. However this
invalidation is incomplete. The data are stored in a `filecache` that preserve
them if the bookmark related file are untouched. In practice, the bookmark data
in memory also depends of the changelog content, because of the step checking
if the bookmarks refers to a node known to the changelog. So if the bookmark
data were loaded from an up to date bookmark file but filtered with an outdated
changelog file this go undetected.
This condition is fairly specific, but can occurs very often in practice. We
introduce a test recreating the situation. The test comes in an independant
changeset to show it actually reproduce the situation. The fix will come soon
after.
A large share of the initial investigation of this race condition was made by
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>.