subsetmaker: rework the antichain generation to be usable Before this, antichain computation can run for 10s of hours without completion in sight. We use a more direct approach in the computation to keep the computation in complexity in check. With good result. We can now have a full antichain computation on mozilla-try in about one minute. Which is usable. Differential Revision: https://phab.mercurial-scm.org/D12396

subsetmaker: use SortedSet for the scratch variant This provides a massive speedup on wide repository with many heads. For example on mozilla-try, this move from un-usable slow to fairly instant. Differential Revision: https://phab.mercurial-scm.org/D12395

subsetmaker: stabilize the computation of `scratch` subset `heads` is set, order of the element are not deterministic and we need to stabilize that if we want to get reproducible results. Differential Revision: https://phab.mercurial-scm.org/D12394

revlog: recommit 49fd21f32695 with a fix for issue6528 `filelog.size` currently special cases two forms of metadata encoding: - copy data via the parent order as flag bit - censor data by peaking into the raw delta All other forms of metadata encoding including the empty metadata block are mishandled. In `basefilectx.cmp` the empty metadata block is explicitly checked to compensate for this. Restore 49fd21f32695, but disable it for filelog, so that the original flag bit use contines to work. Document all this mess for now in preparation of a proper rework. Differential Revision: https://phab.mercurial-scm.org/D11203

merge-lists: make it possible to specify pattern to match The `merge-lists` tool doesn't know anything about Python other than its regex that attempts to match import lines. Let's make it possible to pass in a custom regex so it's easy to use the tool for e.g. C/C++ `#include` lines or Rust `use` lines (given the limited). Differential Revision: https://phab.mercurial-scm.org/D12392

contrib: add a partial-merge tool for sorted lists (such as Python imports) This is a pretty naive tool that uses a regular expression for matching lines. It is based on a Google-internal tool that worked in a similar way. For now, the regular expression is hard-coded to attempt to match single-line Python imports. The only commit I've found in the hg core repo where the tool helped was commit 9cd6292abfdf. I think that's because we often use multiple imports per import statement. I think this tool is still a decent first step (especially once the regex is made configurable in the next patch). The merging should ideally use a proper Python parser and do the merge at the AST (or CST?) level, but that's significantly harder, especially if you want to preserve comments and whitespace. It's also less generic. Differential Revision: https://phab.mercurial-scm.org/D12380

branching: merge stable into default

Added signature for changeset 5bd6bcd31dd1

Added tag 6.1.1 for changeset 5bd6bcd31dd1

relnotes: add notes for 6.1.1 This also fixes the header for 6.1 from 6.1rc0

merge: stable into default

rust-hgpath: add `repr(transparent)` to `HgPath` It's been stabilized a long time ago, so let's not rely on an implementation detail now. Differential Revision: https://phab.mercurial-scm.org/D12433

rust-dirstatemap: correctly decrement the copies counter This was caught when writing unit tests for the `DirstateMap`. We were always setting `had_copy_source` to `false` since we erased the value just before. Differential Revision: https://phab.mercurial-scm.org/D12432

rust-dirstatemap: properly decrement counter for tracked descendants I found this bug when writing unit tests after the fact for the `DirstateMap`. We never decremented the tracked descendants counter since we were always resetting the node data before reading it. This also drops the use of `state`, in favor of the new API to get that information. Differential Revision: https://phab.mercurial-scm.org/D12431

rust-dirstate: panic if the DirstateMap counters go below 0 When modifying the API I hit some... interesting errors (trying to allocate 178GB of RAM, for example) because I failed to keep the counters correctly updated. This counter underflow is likely to happen when code is changed around and can have up to eat-your-dirstate level of consequences, which is not nice. The very small runtime cost of checking these counters should really not be an issue and will help us uncover bugs when/if they do appear in the future. Differential Revision: https://phab.mercurial-scm.org/D12430

rust: fix unsound `OwningDirstateMap` As per the previous patch, `OwningDirstateMap` is unsound. Self-referential structs are difficult to implement correctly in Rust since the compiler is free to move structs around as much as it wants to. They are also very rarely needed in practice, so the state-of-the-art on how they should be done within the Rust rules is still a bit new. The crate `ouroboros` is an attempt at providing a safe way (in the Rust sense) of declaring self-referential structs. It is getting a lot attention and was improved very quickly when soundness issues were found in the past: rather than relying on our own (limited) review circle, we might as well use the de-facto common crate to fix this problem. This will give us a much better chance of finding issues should any new ones be discovered as well as the benefit of fewer `unsafe` APIs of our own. I was starting to think about how I would present a safe API to the old struct but soon realized that the callback-based approach was already done in `ouroboros`, along with a lot more care towards refusing incorrect structs. In short: we don't return a mutable reference to the `DirstateMap` anymore, we expect users of its API to pass a `FnOnce` that takes the map as an argument. This allows our `OwningDirstateMap` to control the input and output lifetimes of the code that modifies it to prevent such issues. Changing to `ouroboros` meant changing every API with it, but it is relatively low churn in the end. It correctly identified the example buggy modification of `copy_map_insert` outlined in the previous patch as violating the borrow rules. Differential Revision: https://phab.mercurial-scm.org/D12429

rust: explain why the current `OwningDirstateMap` is unsound See inline comments. Differential Revision: https://phab.mercurial-scm.org/D12428

dispatch: fix silly blackbox entries when hg is interrupted When hg is interrupted, it creates ui.log like this: 1970/01/01 00:00:00 user @0000000000000000000000000000000000000000 (62488)> killed! exited 255 after 1.78 seconds This is due to a scoping problem: two different uses of the name "msg" collide. So rename one of them. Differential Revision: https://phab.mercurial-scm.org/D12427

tests: stop excluding the pycompat module from pyflakes I assume this was skipped because of all of the py2 stuff causing a lot of spew. The "unused" imports are left in place in case any 3rd party stuff is using it. I don't care about most of it, but TortoiseHg uses `io` and `queue`, so minimally I'd like to keep those. Differential Revision: https://phab.mercurial-scm.org/D12423

tests: drop some py2 specific pyflake failures Differential Revision: https://phab.mercurial-scm.org/D12422

util: drop a duplicate import This was already imported several lines above. Differential Revision: https://phab.mercurial-scm.org/D12421

pycompat: drop the pickle import I suspect this is what df56e6bd37f6 meant to eliminate. Differential Revision: https://phab.mercurial-scm.org/D12420

util: restore the util.pickle symbol This was accidently dropped in df56e6bd37f6, which started importing pickle directly. That commit explicitly says it will retain it for compatibility with external stuff though. The unused import in pycompat isn't flagged because that module is skipped. Just importing with a comment seemed cleaner than `import X as Y` and then assigning to a `pickle` variable, just to avoid the pyflakes warning. Differential Revision: https://phab.mercurial-scm.org/D12419

merge: stable into default