view relnotes/5.5 @ 49000:dd6b67d5c256 stable

rust: fix unsound `OwningDirstateMap`

As per the previous patch, `OwningDirstateMap` is unsound. Self-referential
structs are difficult to implement correctly in Rust since the compiler is
free to move structs around as much as it wants to. They are also very rarely
needed in practice, so the state-of-the-art on how they should be done within
the Rust rules is still a bit new.

The crate `ouroboros` is an attempt at providing a safe way (in the Rust sense)
of declaring self-referential structs. It is getting a lot attention and was
improved very quickly when soundness issues were found in the past: rather than
relying on our own (limited) review circle, we might as well use the de-facto
common crate to fix this problem. This will give us a much better chance of
finding issues should any new ones be discovered as well as the benefit of
fewer `unsafe` APIs of our own.

I was starting to think about how I would present a safe API to the old struct
but soon realized that the callback-based approach was already done in
`ouroboros`, along with a lot more care towards refusing incorrect structs.

In short: we don't return a mutable reference to the `DirstateMap` anymore, we
expect users of its API to pass a `FnOnce` that takes the map as an argument.
This allows our `OwningDirstateMap` to control the input and output lifetimes
of the code that modifies it to prevent such issues.

Changing to `ouroboros` meant changing every API with it, but it is relatively
low churn in the end. It correctly identified the example buggy modification of
`copy_map_insert` outlined in the previous patch as violating the borrow rules.

Differential Revision: https://phab.mercurial-scm.org/D12429

== New Features ==

 * clonebundles can be annotated with the expected memory requirements
   using the `REQUIREDRAM` option. This allows clients to skip
   bundles created with large zstd windows and fallback to larger, but
   less demanding bundles.

 * The `phabricator` extension now provides more functionality of the
   arcanist CLI like changing the status of a differential.

 * Phases processing is much faster, especially for repositories with
   old non-public changesets.


== New Experimental Features ==

 * The core of some hg operations have been (and are being)
   implemented in rust, for speed. `hg status` on a repository with
   300k tracked files goes from 1.8s to 0.6s for instance.
   This has currently been tested only on linux, and does not build on
   windows. See rust/README.rst in the mercurial repository for
   instructions to opt into this.

 * An experimental config `rewrite.empty-successor` was introduced to control
   what happens when rewrite operations result in empty changesets.


== Bug Fixes ==

 * For the case when connected to a TTY, stdout was fixed to be line-buffered
   on Python 3 (where it was block-buffered before, causing the process to seem
   hanging) and Windows on Python 2 (where it was unbuffered before).

 * Subversion sources of the convert extension were fixed to work on Python 3.

 * Subversion sources of the convert extension now interpret the encoding of
   URLs like Subversion. Previously, there were situations where the convert
   extension recognized a repository as present but Subversion did not, and
   vice versa.

 * The empty changeset check of in-memory rebases was fixed to match that of
   normal rebases (and that of the commit command).

 * The push command now checks the correct set of outgoing changesets for
   obsolete and unstable changesets. Previously, it could happen that the check
   prevented pushing changesets which were already on the server.


== Backwards Compatibility Changes ==

 * Mercurial now requires at least Python 2.7.9 or a Python version that
   backported modern SSL/TLS features (as defined in PEP 466), and that Python
   was compiled against a OpenSSL version supporting TLS 1.1 or TLS 1.2
   (likely this requires the OpenSSL version to be at least 1.0.1).
 
 * The `hg perfwrite` command from contrib/perf.py was made more flexible and
   changed its default behavior. To get the previous behavior, run `hg perfwrite
   --nlines=100000 --nitems=1 --item='Testing write performance' --batch-line`.

 * The absorb extension now preserves changesets with no file changes that can
   be created by the commit command (those which change the branch name
   compared to the parent and those closing a branch head).


== Internal API Changes ==

 * logcmdutil.diffordiffstat() now takes contexts instead of nodes.

 * The `mergestate` class along with some related methods and constants have
   moved from `mercurial.merge` to a new `mercurial.mergestate` module.

 * The `phasecache` class now uses sparse dictionaries for the phase data.
   New accessors are provided to detect if any non-public changeset exists
   (`hasnonpublicphases`) and get the correponsponding root set
   (`nonpublicphaseroots`).

 * The `stdin`, `stdout` and `stderr` attributes of the `mercurial.pycompat`
   module were removed. Instead, the attributes of same name from the
   `mercurial.utils.procutil` module should be used, which provide more
   consistent behavior across Python versions and platforms.