dirstate: fix restoration of "merged" state after a remove Before this change, "merged" file that get removed and re-added later were recorded as "from_p2" instead. This came from 8fe74328f700, a 2014 changeset that start explicitly doing so for reason I have not been able to fully grasp. The graft test mentioned in the description are still happy after this changeset. So this changeset restore what seems to be the intended behavior. Restoring information as it was before the removal. Differential Revision: https://phab.mercurial-scm.org/D11429

dirstate: inline the last two `_drop` usage The function is small and having the associated code directly inline help use to cleanup the dirstate API. Differential Revision: https://phab.mercurial-scm.org/D11428

dirstate: make a conditionnal easier to read in `setparents` By grabing `nullid` in the local space, we get the conditionnal to fit in one line. Differential Revision: https://phab.mercurial-scm.org/D11427

dirstate: removed the now unused `_add` method Differential Revision: https://phab.mercurial-scm.org/D11426

dirstate: replace `_add` call in `setparent` Same logic as the previous changeset, it get us closer to removed older API on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D11425

dirstate: directly call the right function in `set_tracked` This get use closer to removing some older API on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D11424

dirstate: same logic as what we did for `_drop` This is part of the dirstatemap so let the dirstatemap deal with it. Differential Revision: https://phab.mercurial-scm.org/D11423

dirstate: use `tracked` property in `_addpath` It is semantically better. Differential Revision: https://phab.mercurial-scm.org/D11422

rust: Update the memmap2 crate to version 0.4.0 This version: * Optionally implements the `StableDeref` trait, removing the need for a wrapper type doing that * Adds support for mapping empty files, which can routinely happen for dirstate-v2 data files. This was a cause of some failing tests when running with `run-tests.py --rhg --extra-config-opt rhg.status=1` Differential Revision: https://phab.mercurial-scm.org/D11446

tests: make removeemptydirs more portable The behavior of pwd change from one system (i.e. default shell/sh) from one system to another so we use fuzzy matching for the output we do not care and we use narrower command call to highlight what we are trying to test here. Differential Revision: https://phab.mercurial-scm.org/D11449

check-code: make it possible to ignore the PWD check in some situation The check-code script does not motivate the banning of $PWD so I am not sure what it is about. And I will needs to use the env variable in the next tests. So I am adding a way to make an exception. Differential Revision: https://phab.mercurial-scm.org/D11450

relnotes: update next Differential Revision: https://phab.mercurial-scm.org/D11448

relnotes: add release notes for 5.9 Differential Revision: https://phab.mercurial-scm.org/D11447

dirstate: fix compilation warnings in `dirstate_item_set_possibly_dirty()` Since https://phab.mercurial-scm.org/D11387 (i.e. the same patch as mentioned in my previous patch), Clang has also started warning about `dirstate_item_set_possibly_dirty()` missing an explicit return, and about its use of the result of an assignment as a condition without using parentheses. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D11445

dirstate: make dirstate flags char be unsigned Since https://phab.mercurial-scm.org/D11387, `CC='clang -Werror' make local` has started failing like this: ``` mercurial/cext/util.h:41:50: error: implicit conversion from 'int' to 'char' changes value from 128 to -128 [-Werror,-Wconstant-conversion] static const char dirstate_flag_rust_special = 1 << 7; ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~^~~~ ``` This patch fixes that by making the flags be an unsigned char. That also matches the `bool` typedef we have in `util.h`, which seems good since many of the `dirstate_item_c_*()` functions return a `bool`. Differential Revision: https://phab.mercurial-scm.org/D11444

dirstate: introduce a `set_clean` method on dirstate's map and items This method is the "reverse" of "set possibly dirty", and can be used to more accurately other call that the dirstate was making. It is currently heavily influenced by its origin. Differential Revision: https://phab.mercurial-scm.org/D11421

dirstate: extract the logic to check file/dirname collision when adding a file Differential Revision: https://phab.mercurial-scm.org/D11420

dirstate: make dirstatemap.set_untracked deal with added file This merge dropfile in set_untracked. Differential Revision: https://phab.mercurial-scm.org/D11419

dirstate: remove some usage of `_drop` This is a step toward being able to remove the `_drop` method on `dirstate`. Differential Revision: https://phab.mercurial-scm.org/D11418

dirstate: move the copymap drop inside dropfile Since the copymap is part of the dirstatemap it make more sense for the dirstatemap to manage it directly. This is part of a generic effort to move unified logic at lower level and to clean up higher level API. Differential Revision: https://phab.mercurial-scm.org/D11417

rust: Rename get_node methods to data_for_node, get_rev to data_for_rev These are respective methods of Changelog, Manifestlog, and Filelog; three Rust structs that that wrap a Revlog struct. This rename clarifies that node IDs or revision numbers are parameters, not return values. Also reword doc-comments in Manifestlog and Filelog to separate node IDs and revision numbers that are local to a given (non-changelog) revlog from those of a changeset. Differential Revision: https://phab.mercurial-scm.org/D11416

rust: Rename the `Revlog::get_node_rev` method to `rev_from_node` This better describes the input and outputs of this method. Also rewrite the doc-comment, which seemed to have been left from copy-paste of another method. Differential Revision: https://phab.mercurial-scm.org/D11415

rust: Make private the `index` field of the `Revlog` struct To replace the previous use of this field from another module, add a `node_from_rev` method. This is the same method that already existed on `Changelog`. Differential Revision: https://phab.mercurial-scm.org/D11414

dirstate-v2: Remove the `.d` suffix in data file names It could cause confusion since `.d` is already used for revlogs. This suffix is not necessary since there is already a `dirstate.` prefix. Differential Revision: https://phab.mercurial-scm.org/D11413

rhg: Don’t compare ambiguous files one byte at a time Even though the use of `BufReader` reduces the number of syscalls to read the file from disk, `.bytes()` yields a separate `Result` for every byte. Creating those results and dispatching on them is most likely costly. Instead, this commit opts for simplicity by reading the entire file into memory and comparing a single pair of byte strings. Note that memory already needs to contain the entire previous contents of the file, as read from the filelog. So with an extremely large file this doubles memory use but does not make it grow by orders of magnitude. At first I wrote code that still avoids reading the entire file into memory and compares one buffer at a time with `BufReader`. Find this code below for posterity. However its correctness is subtle. I ended up preferring the simplicity of the obviously-correct single comparison. ```rust let mut reader = BufReader::new(fobj); let mut expected = &contents_in_p1[..]; loop { let buf = reader.fill_buf().when_reading_file(&fs_path)?; if buf.is_empty() { // Found EOF return Ok(expected.is_empty()); } else if let Some(rest) = expected.drop_prefix(buf) { // What we read so far matches the expected content, continue reading let buf_len = buf.len(); reader.consume(buf_len); expected = rest } else { // Found different content return Ok(false); } } ``` Differential Revision: https://phab.mercurial-scm.org/D11412

rhg: Reuse manifest when checking status of multiple ambiguous files When `rhg status` cannot determine whether a file is clean based on mtime and size alone, it needs to compare its contents with those found in the parent commit. Previously, rhg would find the (same) manifest of that commit again for every such file. This is lifted out of the loop and reused. Differential Revision: https://phab.mercurial-scm.org/D11411

rust: Return HgError instead of RevlogError in revlog constructors This leaves fewer cases for callers to handle, as RevlogError is more general Differential Revision: https://phab.mercurial-scm.org/D11410

rhg: Align with Python on some revset parsing corner cases In particular: * A string of ASCII digits can be either an integer on a hex prefix * The NULL node ID should convert to the NULL revision number Differential Revision: https://phab.mercurial-scm.org/D11409

rust: Add a Filelog struct that wraps Revlog Some filelog-specific logic is moved from code `rhg cat` into this struct where it can better be reused. Additionally, a missing end delimiter for metadata causes an error to be returned instead of being silently ignored. Differential Revision: https://phab.mercurial-scm.org/D11408

rust: Add Repo::manifest(revision) This deduplicates some common code. Differential Revision: https://phab.mercurial-scm.org/D11407

rust: Keep lazily-initialized Changelog and Manifest log on the Repo object That way if one of them is accessed multiple times it won’t be reopened from the filesystem. Differential Revision: https://phab.mercurial-scm.org/D11406

rust: Move lazy initialization of `Repo::dirstate_map` into a generic struct More components of `Repo` will be added following the same pattern. Differential Revision: https://phab.mercurial-scm.org/D11405

rust: Rename Manifest to Manifestlog, ManifestEntry to Manifest This appears to match the terminology used in Python code and on https://www.mercurial-scm.org/wiki/Manifest Differential Revision: https://phab.mercurial-scm.org/D11404

rust: Add Repo::dirstate_map and use it in `rhg status` This moves low-level dirstate wrangling out of the status command and into a more reusable location. The open dirstate map is lazily initialized and kept on the Repo object, for reuse by sub-sequent calls. Differential Revision: https://phab.mercurial-scm.org/D11398

rust: Switch to the memmap2-rs crate https://github.com/RazrFalcon/memmap2-rs This is a fork of the original memmap crate which appears to be unmaintained: https://github.com/danburkert/memmap-rs/issues/90 This fork is the most popular according to https://crates.io/keywords/mmap Differential Revision: https://phab.mercurial-scm.org/D11397

rust: Make OwningDirstateMap generic and move it into hg-core This will enable using it in rhg too. The `OwningDirstateMap::new_empty` constructor is generic and accepts a value of any type that gives acces to a bytes buffer. That buffer must stay valid as long as the value hasn’t been dropped, and must keep its memory address even if the value is moved. The `StableDeref` marker trait encodes those constraints. Previously no trait was needed because the value was always of type `PyBytes` which we know satisfies those constraints. The buffer type is ereased in the struct itself through boxing and dynamic dispatch, in order to simplify other signatures that mention `OwningDirstateMap`. Differential Revision: https://phab.mercurial-scm.org/D11396

rust: Move PyBytesWithData out of copy-tracing code So we can use it in other places to. Replace its `.data()` method with the `Deref<Target = [u8]>` trait, allowing this type to be used in generic contexts. Rename the type accordingly. Differential Revision: https://phab.mercurial-scm.org/D11395

rust: Move VFS code to its own module It was previously in the hg::repo module, but both repo code and vfs will likely grow in the future. Differential Revision: https://phab.mercurial-scm.org/D11394

rhg: Switch rhg.ignored-extensions config to Python-compatible list syntax This includes the ability to have an extension name/path that contains a comma, by double-quoting it. Differential Revision: https://phab.mercurial-scm.org/D11390

rhg: Port Python’s `ui.configlist` as `Config::get_list` This new method is not used yet outside of its own unit tests, so this changeset should make no observable change. The Rust parser implementation attempts to exactly replicate the behavior of the Python one, even in edge cases where that behavior is… surprising. New unit tests capture some of these edge cases. This started as a line-by-line port. The main changes are: * Pass around a parser mode enum instead of parser functions * Inline the whole parser into one function * Use `[u8]::get` which returns an `Option`, instead of indexing after explicitly checking the length. Differential Revision: https://phab.mercurial-scm.org/D11389

rust: Generalize the `trim_end_newlines` utility of byte strings … into `trim_end_matches` that takes a callack. Also add `trim_start_matches`. Differential Revision: https://phab.mercurial-scm.org/D11388

dirstate-item: move the C implementation to the same logic Now that we know were we are going, we can update the C implementation. Differential Revision: https://phab.mercurial-scm.org/D11387

dirstate-item: introduce low level C function Since a lot of logic is about to change, we need to isolate it into bits that are easy reusable in the rest of the C code. Especially we need to split the part that do the pure logic from the part that do the Python wrapping. Differential Revision: https://phab.mercurial-scm.org/D11386

dirstate-item: factor some code in the C implementation We have both `dirstate_item_from_v1_meth`, a class method, and `dirstate_item_from_v1_data`, a function taking low level C argument. Lets implement the former with the later. Differential Revision: https://phab.mercurial-scm.org/D11385

dirstate-item: `dirstate_item_from_v1_data` replaces make_dirstate_item For different format, we will need multiple capsule anyways. Differential Revision: https://phab.mercurial-scm.org/D11384

pathutil: replace the `skip` argument of `dirs` with a boolean It is ever only used for `r` file. So we make it a boolean this will give use more versatility later as we will stop storing the state explicitly. Differential Revision: https://phab.mercurial-scm.org/D11383

dirstate: clarify the message in nonnormal checking The previous message was quite verbose and hard to understand. The new one should be friendlier. Differential Revision: https://phab.mercurial-scm.org/D11382

merge: with stable

narrow: fix commits of empty files The problem is that when committing a new file with empty contents (or in general empty file with filelog p1 = -1), hg commit with narrow doesn't create a filelog revision at all, which causes failures in further commands. The problem seems to be that: - hg thinks that instead of creating a new filelog revision, it can use the filelog's p1 (the nullrev) - because it thinks the file contents is the same in that revision and in p1 - because `narrowfilelog.cmp(nullrev, b'')` is True (unlike with `filelog.cmp`) It's not clear to me which `cmp` behaves better. But I think it makes sense to change the commit code to not to "reuse" the null rev when adding an empty file with filelog p1 == filelog p2 == -1. This is consistent with never writing the null rev in the manifest, which `hg verify` claims is an invariant: ``` inside/c@4: manifest refers to unknown revision 000000000000 ``` Differential Revision: https://phab.mercurial-scm.org/D11400

narrow: show repo corruption when commiting empty files Differential Revision: https://phab.mercurial-scm.org/D11399

filemerge: be more strict when detecting conflict markers, add `|` markers I received a user complaint about detecting a line that contained 78 `=` followed by `*/` as a conflict marker. We'll never generate that, we generate 7 identical characters and either the end of the line, or a space. Let's be explicit about detecting exactly what we produce to reduce the chances of a false positive. While we're here, add `|||||||` as a detected conflict marker (generated with the `keep-merge3` style conflicts). Differential Revision: https://phab.mercurial-scm.org/D11391

dirstate-item: implement `v1_mtime` with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11375

dirstate-item: implement `v1_size` with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11374

dirstate-item: implement v1_state with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11373

dirstate-item: add more logic to `from_p2` This seem semantically better even if this tracking case seems to always comes with clean_p2 for now. Differential Revision: https://phab.mercurial-scm.org/D11372

dirstate-item: implement `merged_removed` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11371