view relnotes/5.9 @ 49000:dd6b67d5c256 stable

rust: fix unsound `OwningDirstateMap`

As per the previous patch, `OwningDirstateMap` is unsound. Self-referential
structs are difficult to implement correctly in Rust since the compiler is
free to move structs around as much as it wants to. They are also very rarely
needed in practice, so the state-of-the-art on how they should be done within
the Rust rules is still a bit new.

The crate `ouroboros` is an attempt at providing a safe way (in the Rust sense)
of declaring self-referential structs. It is getting a lot attention and was
improved very quickly when soundness issues were found in the past: rather than
relying on our own (limited) review circle, we might as well use the de-facto
common crate to fix this problem. This will give us a much better chance of
finding issues should any new ones be discovered as well as the benefit of
fewer `unsafe` APIs of our own.

I was starting to think about how I would present a safe API to the old struct
but soon realized that the callback-based approach was already done in
`ouroboros`, along with a lot more care towards refusing incorrect structs.

In short: we don't return a mutable reference to the `DirstateMap` anymore, we
expect users of its API to pass a `FnOnce` that takes the map as an argument.
This allows our `OwningDirstateMap` to control the input and output lifetimes
of the code that modifies it to prevent such issues.

Changing to `ouroboros` meant changing every API with it, but it is relatively
low churn in the end. It correctly identified the example buggy modification of
`copy_map_insert` outlined in the previous patch as violating the borrow rules.

Differential Revision: https://phab.mercurial-scm.org/D12429

== New Features ==

 * `hg config` now has a `--source` option to show where each
   configuration value comes from.
 * Introduced a command (debug-repair-issue6528) to repair repositories affected
   by issue6528 where certain files would show up as modified even if they were
   clean due to an issue in the copy-tracing code.

== Default Format Change ==

These changes affect newly created repositories (or new clone) done with
Mercurial 5.9.


== New Experimental Features ==

 * A `changelogv2` format has been introduced. It is not ready for use yet, but
   will be used later to address some of the weaknesses of the current revlog
   format.
 * Initial experiment and support for `dirstatev2`, a new dirstate format that
   addresses some of the weaknesses of the current dirstate format. Python + C
   and Rust support are being implemented, but the Rust solution is the one
   currently getting the attention for performance.
 * Initial support for `rhg status`. `rhg` is the Rust wrapper executable for hg
   that shortcuts some commands for faster execution speed.

== Bug Fixes ==

 * Fixed committing empty files with `narrow`
 * Allow overriding `pip`'s pep517 compliance to build C or Rust extensions
 * Fixed regression on outgoing email when not specifying revisions
 * Fixed a regression causing bookmarks to disappear when using Rust persistent nodemap
 * Fixed a regression (in 5.9.1) introduced in 5.9 when cloning repos with
   deep filenames
 * Fixed detection of directories becoming symlinks, but only when using the
   Rust extensions.
 * Fixed ignore and include not composing in the Rust status
 * `hg commit --interactive` now handles deselecting edits of a rename
 * Fixed a case where `hg evolve` gives different results when interrupted
 * Fixed a memory leak in phases computation
 * `histedit` and `shelve` don't swallow errors when updating the working copy
   anymore
 * Improve error message when detecting content-divergence with a hidden
   common predecessor
 * No longer re-order parents in filelog, see issue6533
 * Fix revisions affected by issue6533 on the fly during exchange
 * Many Windows fixes for stability and py3 compatibility improvements
 * Many other miscellaneous fixes

== Backwards Compatibility Changes ==


== Internal API Changes ==

The Dirstate API have been updated as the previous function leaked some
internal details and did not distinguish between two important cases: "We are
changing parent and need to adjust the dirstate" and "some command is changing
which file is tracked". To clarify the situation:

* the following functions have been deprecated,

  - `dirstate.add`,
  - `dirstate.normal`,
  - `dirstate.normallookup`,
  - `dirstate.merge`,
  - `dirstate.otherparent`,
  - `dirstate.remove`,
  - `dirstate.drop`,
  - `dirstateitem.__getitem__`,

* these new functions are added for the "adjusting parents" use-case:

  - `dirstate.update_file`,
  - `dirstate.update_file_p1`,

* these new function are added for the "adjusting wc file" use-case":

  - `dirstate.set_tracked`,
  - `dirstate.set_untracked`,
  - `dirstate.set_clean`,
  - `dirstate.set_possibly_dirty`,

See inline documentation of the new functions for details.

* Additionally, the following have been deprecated:
  - `urlutil.getpath` function
  - `localrepository.updatecaches`' `full` argument

* The following have been removed:
  - `revlog.revlogio` has been removed