hg: convert an exception to bytes in the repo creation exception handler Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/hg.py", line 77, in _local: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: Union[TypeError, ValueError]) Differential Revision: https://phab.mercurial-scm.org/D10176

typing: add an assertion instead of blacklisting mercurial/extensions.py Differential Revision: https://phab.mercurial-scm.org/D10175

debug: convert a few exceptions to bytes before wrapping in another error Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/debugcommands.py", line 2118, in debugmanifestfulltextcache: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: mercurial.error.LookupError, ...) File "/mnt/c/Users/Matt/hg/mercurial/debugcommands.py", line 2453, in debugobsolete: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: ValueError) The following methods aren't implemented on ValueError: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10174

grep: convert an exception to bytes for a warning message Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/commands.py", line 3457, in grep: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: re.error) The following methods aren't implemented on re.error: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10173

morestatus: convert a UI message about merge conflicts to bytes Differential Revision: https://phab.mercurial-scm.org/D10172

changegroup: convert a warning message to bytes Differential Revision: https://phab.mercurial-scm.org/D10171

branchmap: force Exception to bytes before logging Here was an instance where `black` mangled the formatting so that `pytype` didn't recognize the suppression directive. But it seems that the error was correct, and the code should follow other recent changes around exception conversion. File "/mnt/c/Users/Matt/hg/mercurial/branchmap.py", line 303, in fromfile: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: Exception) The following methods aren't implemented on Exception: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10170

nodemap: convert error message to bytes Differential Revision: https://phab.mercurial-scm.org/D10169

mail: convert SMTPException to bytes before passing to error.Abort() Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/mail.py", line 168, in _smtp: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: smtplib.SMTPException) Differential Revision: https://phab.mercurial-scm.org/D10168

typing: switch an argument type to the generic form This fixes the following pytype complaint: File "/mnt/c/Users/Matt/hg/mercurial/commands.py", line 4672, in log: Function mercurial.logcmdutil.parseopts was called with the wrong arguments [wrong-arg-types] Expected: (ui, pats: List[Union[bytearray, bytes, memoryview]], ...) Actually passed: (ui, pats: tuple, ...) Differential Revision: https://phab.mercurial-scm.org/D10167

typing: ensure that error.Abort is given bytes There's a bunch more typing to be done here, but the list of things to fix is already long, and I know there are instances where this is being used incorrectly. Differential Revision: https://phab.mercurial-scm.org/D10166

typing: fix a suppression directive that was mangled by black formatting It looks like black is moving comments needed by pytype out of position, and causing some things that should be disabled to be enforced anyway. Differential Revision: https://phab.mercurial-scm.org/D10165

commit: reorder if-else conditional to give mergestate info priority Looking at the code now, I was unable to find a good reason as why we only rely on mergestate extras info after checking whether a filelog parent is ancestor of other or not. I mean if we have stored in mergestate that `other` was chosed, we should blindly pick that one. This cleanup will also help introduce more cases when both `fparent1` and `fparent2` are non-null but using info from mergestate, we can fastpath. The test change actually demonstrates the point of the patch. During merge we were getting the other side of the file but on commit we were marking that as merged. Differential Revision: https://phab.mercurial-scm.org/D10147

enclink: check contents of symlinks not just size in case of fcrypt Check content of symlinks because st_size may indicate the size of encrypted data which does not match actual link value. Differential Revision: https://phab.mercurial-scm.org/D10146

rebase: inline simple function for finding obsolete subset of commits `_filterobsoleterevs()` is just one line long. It was introduced in 2d294dada4f8 (rebase: small refactoring to allow better extensibility from extensions, 2016-01-14), for use by the "inhibit" extension. That extension was removed from the evolve repo in 87e87881059d (compat: drop the inhibit hacky extension, 2017-10-24). Differential Revision: https://phab.mercurial-scm.org/D10198

ci: hook network-io tests into the pipeline This runs the "pip install" tests once for Python 2 and 3 each. Differential Revision: https://phab.mercurial-scm.org/D10075

merge: with stable

test: update expected output in test-http.t The output was introduced in a4c19a162615 and is wrong and unstable. So we glob it as other part of these tests already do. Differential Revision: https://phab.mercurial-scm.org/D10153

remotefilelog: remove unused import This is no longer used since a4c19a162615. Differential Revision: https://phab.mercurial-scm.org/D10152

wireprotov1peer: don't raise internal errors in some cases Specifically, when the peer is closed in the middle of a batch of rpcs. Differential Revision: https://phab.mercurial-scm.org/D10074

sshpeer: don't fail forwarding output from closed connections The test still shows an internal error, but one that happens further along. Differential Revision: https://phab.mercurial-scm.org/D10073

test: show internal exception with batchable rpcs over ssh Differential Revision: https://phab.mercurial-scm.org/D10072

rhg: Ignore trailing newlines in .hg/sharedpath Differential Revision: https://phab.mercurial-scm.org/D10132

tests: Add `--rhg` and `--with-rhg=<path>` options for `run-tests.py` They are mostly equivalent to the corresponding `chg` options. For now, many tests are still failing in this configuration. It is *not* run on CI. Differential Revision: https://phab.mercurial-scm.org/D10095

rhg: Add support for automatic fallback to Python `rhg` is a command-line application that can do a small subset of what `hg` can. It is written entirely in Rust, which avoids the cost of starting a Python interpreter and importing many Python modules. In a script that runs many `hg` commands, this cost can add up. However making users decide when to use `rhg` instead of `hg` is not practical as we want the subset of supported functionality to grow over time. Instead we introduce "fallback" behavior where, when `rhg` encounters something (a sub-command, a repository format, …) that is not implemented in Rust-only, it does nothing but silently start a subprocess of Python-based `hg` running the same command. That way `rhg` becomes a drop-in replacement for `hg` that sometimes goes faster. Whether Python is used should be an implementation detail not apparent to users (other than through speed). A new `fallback` value is added to the previously introduced `rhg.on-unsupported` configuration key. When in this mode, the new `rhg.fallback-executable` config is determine what command to use to run a Python-based `hg`. The previous `rhg.on-unsupported = abort-silent` configuration was designed to let a wrapper script call `rhg` and then fall back to `hg` based on the exit code. This is still available, but having fallback behavior built-in in rhg might be easier for users instead of leaving that script "as an exercise for the reader". Using a subprocess like this is not idea, especially when `rhg` is to be installed in `$PATH` as `hg`, since the other `hg.py` executable needs to still be available… somewhere. Eventually this could be replaced by using PyOxidizer to a have a single executable that embeds a Python interpreter, but only starts it when needed. Differential Revision: https://phab.mercurial-scm.org/D10093

rhg: Add a `rhg.on-unsupported` configuration key For now the two values are: * `abort-silent`: silently exit with code 252, the previous default behavior * `abort`: print an error message about what feature is not supported, then exit with code 252. Now the default. Differential Revision: https://phab.mercurial-scm.org/D10091

rhg: Make configuration available as early as possible in main() Differential Revision: https://phab.mercurial-scm.org/D10090

rust: Upgrade Cargo.lock to the newer format As discussed in https://phab.mercurial-scm.org/D10085#153099 See https://github.com/rust-lang/cargo/pull/7070 and https://blog.rust-lang.org/2020/01/30/Rust-1.41.0.html#less-conflict-prone-cargolock-format Differential Revision: https://phab.mercurial-scm.org/D10089

sshpeer: enable+fix warning about sshpeers not being closed explicitly I recommend looking at this with a diff that ignores indentation. The test changes are because localrepo.close() updates some cache, which appears happens earlier now on rollbacks or strips or something. The http changes are because httppeer.close() prints stats with --verbose. Differential Revision: https://phab.mercurial-scm.org/D9999

sshpeer: add a develwarning if an sshpeer is not closed explicitly The warning is disabled until the next commit, because fixing it results in a noisy diff due to indentation changes. Differential Revision: https://phab.mercurial-scm.org/D9998

remotefilelog: rework workaround for sshpeer deadlocks The wrapping of `sshpeer.cleanup` silently broke when `cleanup` was renamed to `_cleanup`, a couple of years ago. I don't know what `orig.im_self` is, but regardless, the intention of the wrapping seems pretty clear: close stderr before sshpeer._cleanuppipes blocks on it. So do that. Differential Revision: https://phab.mercurial-scm.org/D9997

sshpeer: add a method to check if a doublepipe is closed So we can tell in a next commit if we're trying to close an already closed connection or not (in which case, we may warn). Differential Revision: https://phab.mercurial-scm.org/D9996

sshpeer: make sshpeer.close() close the underlying connection So the connection can be closed eagerly in future commits, instead of relying on __del__. Differential Revision: https://phab.mercurial-scm.org/D9995

copies-rust: add a macro-based unit-testing framework `compare_values`, `merge_copies_dict`, and `CombineChangesetCopies` are APIs whose signatures involve non-trivial types. Calling them directly in unit tests would involve a lot of verbose setup code that obscures the meaningful parts of a given test case. This adds a macro-based test-harness with pseudo-syntax to tersely create arguments and expected return values in the correct types. For now there is only one (not particularly meaningful) test case per tested function, just to exercize the macros. Differential Revision: https://phab.mercurial-scm.org/D10071

copies-rust: rewrite ChangedFiles binary parsing by using the new from-bytes-safe crate and a custom struct that encodes the expected data structure. Differential Revision: https://phab.mercurial-scm.org/D10068

tags: validate nodes in _getfnodes() and update cache in case of unknown nodes `hgtagsfnodescache` can contain unknown nodes due to cache corruption and this lead to a traceback on operations like `hg tags` as we don't validate nodes. This patch validates that all filenodes returned after `hgtagsfnodescache` are known to the repository. If there exists any unknown filenode, we force recompute it and update the cache. The test change demonstrates the fix. Differential Revision: https://phab.mercurial-scm.org/D10083

debugtagscache: verify that filenode is correct Previous patch from Matt demonstrates that `debugtagscache` does not warn about filenode being unknown which can be caused by a corrupted cache. We start by showing that it's an unknown node. Differential Revision: https://phab.mercurial-scm.org/D10015

tests: demonstrate a case where a corrupt tag cache causes an abort I happened to hit this trying to cover other cases around valid vs missing entries. I have no idea if this is something that could occur more naturally (similar to how a missing file node in `hgtagsfnodes1` can occur after a strip). There is a test just above this added in f5a7cf0adb12 mentioning it "overwrites the junk", though that tests truncation instead of actual garbage. But since this is just a cache, it probably shouldn't abort with a cryptic message like this. The two options I see both have downsides- either rebuild the cache (and potentially take a long time), or hint to the user to run a debug command. Differential Revision: https://phab.mercurial-scm.org/D9812

debugcommands: prevent using `is False` I was touching this code in a future patch and marmoute warned about usage of `is False` here. Quoting marmoute: ``` "is False" is going to check if the object you have the very same object in memory than the one Python allocated for False (in practice 0) This will "mostly work" on cpython because of implementation details, but is semantically wrong and can start breaking unexpectedly ``` Differential Revision: https://phab.mercurial-scm.org/D10014

hgtagsfnodes: refactor code to compute fnode into separate fn I plan to use this code at one more place while fixing a bug caused by an invalid fnode present in cache. Differential Revision: https://phab.mercurial-scm.org/D10013

error: remove shortening of node in error message This strips the complete 20 bytes node which was not found. Having the the full node in error message is important as it makes debugging easier. If a short node is to be displayed, that should be done by callers. Differential Revision: https://phab.mercurial-scm.org/D9994

copies: filter out copies grafted from another branch Consider this simple history: ``` @ 3 modify y | o 2 copy x to y, modify x | | o 1 copy x to y, modify x |/ o 0 add x ``` If we now rebase commit 3 onto 1, Mercurial will look for copies between commit 2 and commit 1. It does that by going backwards from 2 to 0 and then forwards from 0 to 1. It will find that x was copied to y, since that was what happened on the path between them (namely in commit 1). That leads Mercurial to do a 3-way merge between y@3 and y@1 with x@2 as base. We want to use y@2 as base instead. That's also what happened until commit 1d6d1a15. This patch fixes the regression by adding another filtering step when chaining copies via a diffbase. The new filtering step removes copies that were the same between the two branches (same source and destination, but not necessarily the same contents). Differential Revision: https://phab.mercurial-scm.org/D10120

copies: inline _backwardrenames() in pathcopies() I'll add another filtering step in `patchcopies()` next. I need access to the forward copies for that. Differential Revision: https://phab.mercurial-scm.org/D10119

copies: extract function _backwardcopies() for reversing renames I'll add another callers in the next patch. Differential Revision: https://phab.mercurial-scm.org/D10118

tests: demonstrate how grafted copies are counted when tracing across branches This test demonstrates a regression from 1d6d1a15. Differential Revision: https://phab.mercurial-scm.org/D10117

typing: add some type annotations to mercurial/pathutil.py Differential Revision: https://phab.mercurial-scm.org/D10128

typing: add some type annotations to mercurial/util.py Differential Revision: https://phab.mercurial-scm.org/D10127

typing: add some type annotations to mercurial/phases.py Some of these were helpful in typing other modules, and then I typed the easy-ish ones. Black forces the long `Phasedefaults` definition to be wrapped, which pytype seems OK with (as shown with `reveal_type()`), but it does seem to confuse PyCharm a bit. Differential Revision: https://phab.mercurial-scm.org/D10126

typing: add type annotations to the public methods of mercurial/subrepoutil.py Differential Revision: https://phab.mercurial-scm.org/D10125

typing: add type annotations to mercurial/i18n.py I'm a little unsure of this because `gettext()` clearly allows for passing unicode. But the comments seem to indicate that this is related to tests, and this was useful for catching unicode being passed to `_()` in the keyring extension. I'm also not sure why `_(None)` would make any sense, so maybe the argument shouldn't be optional? I didn't add it to the lambda in plain mode because that spilled beyond 80 characters and so black mangled it. Black and pytype disagree on where the comment to disable a check needs to go, so this has to disable and then enable the checking. Differential Revision: https://phab.mercurial-scm.org/D10124

typing: add type annotations to mercurial/utils/dateutil.py For now, I'm just typing around the edges to help find issues with TortoiseHg. If the custom `hgdate` type is useful elsewhere as I go, I'll move it to a file dedicated to custom types. I'm not loving the ban on camelcase type names here that test-check-code.t flagged, but I'm not sure how to disable that even if everyone agreed that it's a bad idea to go against the normal convention for types. While here, fix an issue that pytype found in `parsedate` when an invalid date tuple is passed by raising a ProgrammingError instead of crashing. (Tuple doesn't have a `strip` attribute.) Differential Revision: https://phab.mercurial-scm.org/D10123

shelve: fix conversion of exceptions to strings flagged by pytype I've seen this done several ways and don't know what's correct. But pytype was unhappy about the previous way: FAILED: /mnt/c/Users/Matt/hg/tests/.pytype/pyi/mercurial/shelve.pyi /usr/bin/python3.6 -m pytype.single --imports_info /mnt/c/Users/Matt/hg/tests/.pytype/imports/mercurial.shelve.imports --module-name mercurial.shelve -V 3.6 -o /mnt/c/Users/Matt/hg/tests/.pytype/pyi/mercurial/shelve.pyi --analyze-annotated --nofail --quick /mnt/c/Users/Matt/hg/mercurial/shelve.py File "/mnt/c/Users/Matt/hg/mercurial/shelve.py", line 244, in _verifyandtransform: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: Union[KeyError, TypeError, ValueError]) File "/mnt/c/Users/Matt/hg/mercurial/shelve.py", line 253, in _getversion: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: ValueError) The following methods aren't implemented on ValueError: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10122

releasenotes: use the right API to access the 'sections' Preventing direct access to the underlying dict fix a breakage introduced by the refactoring in d3df397e7a59. This changeset is similar to 271dfcb98544, 5272542196cc and f7621fa14b84. The breackage of `releasenotes.py` stayed under my radar as the CI did not have fuzzywuzzy installed. (Something that is about to be fixed). Differential Revision: https://phab.mercurial-scm.org/D10121

requirements: also add a fncache constant Continue the cleanup to the remaining requirements Differential Revision: https://phab.mercurial-scm.org/D10109

requirements: also add a store constant Continue the cleanup to the remaining requirements Differential Revision: https://phab.mercurial-scm.org/D10108

requirements: also add a dotencode constant Continue the cleanup to the remaining requirements Differential Revision: https://phab.mercurial-scm.org/D10107

pure-parsers: document index class constants This also adds the big endian prefix `>` to make the constants truly platform-independent, even if no issue with this has been reported in the wild. Differential Revision: https://phab.mercurial-scm.org/D10104

copies: choose target directory based on longest match If one side of a merge renames `dir1/` to `dir2/` and the subdirectory `dir1/subdir1/` to `dir2/subdir2/`, and the other side of the merge adds a file in `dir1/subdir1/`, we should clearly move that into `dir2/subdir2/`. We already detect the directories correctly before this patch, but we iterate over them in arbitrary order. That results in the new file sometimes ending up in `dir2/subdir1/` instead. This patch fixes it by iterating over the source directories by visiting subdirectories first. That's achieved by simply iterating over them in reverse lexicographical order. Without the fix, the test case still passes on Python 2 but fails on Python 3. It depends on the iteration order of the dict. I did not look into how it's built up and why it behaved differently before the fix. I could probably have gotten it to fail on Python 2 as well by choosing different directory names. Differential Revision: https://phab.mercurial-scm.org/D10115

exchange: stop advertising rev-branch-cache bundle capability Since Mercurial 5.7, the corresponding bundle part is ignored as redundant. Stop advertising it so that peers don't have to spend time creating or transfering it. Differential Revision: https://phab.mercurial-scm.org/D10114

errors: use more specific errors in rebase extension Differential Revision: https://phab.mercurial-scm.org/D9914

changelog: rename parameters to reflect semantics `read` and `readfiles` can be used with a revision just as well, so follow the naming convention in revlog to reflect this. Differential Revision: https://phab.mercurial-scm.org/D10081

rhg: Don’t attempt to read persistent nodemap without .hg/requires opt-in Differential Revision: https://phab.mercurial-scm.org/D10077

debian: support a "chg-first" installation mechanism (hg is actually chg) This mechanism builds chg such that it looks for `hg` to be available at /usr/lib/mercurial/hg instead of in the $PATH as `hg`, and makes the `hg` in /usr/bin be a symlink to `chg`. It's important to note that the hg binary must continue to be named `hg`. If we wanted to instead place it at /usr/bin/pyhg or something similar, we would need to modify Mercurial to allow that basename. Failure to do so would break Mercurial's shell aliases that use `hg`, `chg`, or `$HG`. I don't know if we should ever have a setup like this be the default setup, but I'm willing to get more information on our experience with it for making such a determination. Actually making it the default might be rather involved, as we don't maintain the official debian packaging rules. Differential Revision: https://phab.mercurial-scm.org/D10020

relnotes: document a number of node->revision type changes Differential Revision: https://phab.mercurial-scm.org/D10103

requirements: also add a generaldelta constant Continue the cleanup to the remaining requirements Differential Revision: https://phab.mercurial-scm.org/D10106

requirements: add constant for revlog v1 requirement Since this series is adding a new requirement, we might as well clean up while we're here. Differential Revision: https://phab.mercurial-scm.org/D10105

error: add `hint` attribute to `SidedataHashError` This prevents an exception within an exception because `hint` does not exist. Differential Revision: https://phab.mercurial-scm.org/D10024

changegroup: use the local variable instead of reaching through self Gratuitous cleanup while I was here. Differential Revision: https://phab.mercurial-scm.org/D10023

config: use level to properly deal with value priority A higher priority alias will now take precedence over lower priority ones. This was a requirements step before using alias more widely, especially to rename existing and established config option. Differential Revision: https://phab.mercurial-scm.org/D9927

config: track the "level" of a value Config value now remember the "level" of the config that loaded it. This will be used to ensure good priority management for alias. Differential Revision: https://phab.mercurial-scm.org/D9926

config: track "source" along side value Currently the source is stored in a entirely different way than the data. This is impractical. Especially if we are about to add more of such metadata. So lets put them back together. Differential Revision: https://phab.mercurial-scm.org/D9925

config: use a new `alter` method in `fixconfig` The `set` function is doing various work related to metadata (eg: the source, later the level). However the `fixconfig` call only updating some values according to standard processing, not changing any of the related metadata. So we introduce a new method and use it there. Differential Revision: https://phab.mercurial-scm.org/D9924

copies-rust: remove an unnecessary format!() inside assert!() The `assert!()` macro supports formatting. Rust 2021 no longer supports an unnecessary `format!()` inside it. I noticed because I recently update my Rust toolchain and `test-check-cargo-lock.t` started failing. Differential Revision: https://phab.mercurial-scm.org/D10085

copies: tests and fix parallel computation of changed file information The code was mixing variable name and misbehaving in some case. This changeset fix it and also add a tests to validate it does not regress. Without the fix, the parallel-upgrade misbehavior in random ways. Differential Revision: https://phab.mercurial-scm.org/D10084

config: test priority involving alias and cli Differential Revision: https://phab.mercurial-scm.org/D9920

config: test priority involving alias and include Differential Revision: https://phab.mercurial-scm.org/D9919

config: test priority involving alias Differential Revision: https://phab.mercurial-scm.org/D9918

rhg: Check .hg/requires for absence of required features Some old repository layouts are not supported. Differential Revision: https://phab.mercurial-scm.org/D10076

rhg: Bug fix: with share-safe, always read store requirements That is, the `store/requires` file, regardless of whether the repository is a shared. Differential Revision: https://phab.mercurial-scm.org/D10078

copies-rust: pass closures and iterators instead of `&ChangedFiles` … to some functions that only use one method. This will makes it easier to unit-test them. Differential Revision: https://phab.mercurial-scm.org/D10070

copies-rust: pass `PathToken` around by value It’s just a `usize`. Differential Revision: https://phab.mercurial-scm.org/D10069

tests: correct a commit description in test-copies-chain-merge.t Differential Revision: https://phab.mercurial-scm.org/D10065

fuzz: if the caller of our makefile sets CC and CXX, trust them This should fix the broken fuzzing build, because we've been explicitly using clang++ but are now being given a CXX=afl++, which does extra stuff. Differential Revision: https://phab.mercurial-scm.org/D10066

rhg: Use clap’s support for global CLI arguments By default, clap only accepts app-level arguments (as opposed to sub-command level) to be specified before a sub-command: `rhg -R ./foo log`. Specifying them after would be rejected: `rhg log -R ./foo`. Previously we worked around that by registering global arguments both at the app level and on each sub-command, but that required looking for their value in two places. It turns out that Clap has built-in support for what we want to do, so let’s use it. Also, Clap "settings" turn out to be either global or not too. Let’s make `AllowInvalidUtf8` apply to sub-commands too. Differential Revision: https://phab.mercurial-scm.org/D10080

revlog: add a mechanism to verify expected file position before appending If someone uses `hg debuglocks`, or some non-hg process writes to the .hg directory without respecting the locks, or if the repo's on a networked filesystem, it's possible for the revlog code to write out corrupted data. The form of this corruption can vary depending on what data was written and how that happened. We are in the "networked filesystem" case (though I've had users also do this to themselves with the "`hg debuglocks`" scenario), and most often see this with the changelog. What ends up happening is we produce two items (let's call them rev1 and rev2) in the .i file that have the same linkrev, baserev, and offset into the .d file, while the data in the .d file is appended properly. rev2's compressed_size is accurate for rev2, but when we go to decompress the data in the .d file, we use the offset that's recorded in the index file, which is the same as rev1, and attempt to decompress rev2.compressed_size bytes of rev1's data. This usually does not succeed. :) When using inline data, this also fails, though I haven't investigated why too closely. This shows up as a "patch decode" error. I believe what's happening there is that we're basically ignoring the offset field, getting the data properly, but since baserev != rev, it thinks this is a delta based on rev (instead of a full text) and can't actually apply it as such. For now, I'm going to make this an optional component and default it to entirely off. I may increase the default severity of this in the future, once I've enabled it for my users and we gain more experience with it. Luckily, most of my users have a versioned filesystem and can roll back to before the corruption has been written, it's just a hassle to do so and not everyone knows how (so it's a support burden). Users on other filesystems will not have that luxury, and this can cause them to have a corrupted repository that they are unlikely to know how to resolve, and they'll see this as a data-loss event. Refusing to create the corruption is a much better user experience. This mechanism is not perfect. There may be false-negatives (racy writes that are not detected). There should not be any false-positives (non-racy writes that are detected as such). This is not a mechanism that makes putting a repo on a networked filesystem "safe" or "supported", just *less* likely to cause corruption. Differential Revision: https://phab.mercurial-scm.org/D9952

narrow: remove assertion about working copy being clean The user can always modify the working copy, including while they're running `hg tracked --remove-include <path>`. Nothing really bad happens when they do that, and we already have code for printing a nice warning, so we can safely remove the assertion we had. Differential Revision: https://phab.mercurial-scm.org/D10063

tests: demonstrate assertion error when modifying working copy while narrowing Differential Revision: https://phab.mercurial-scm.org/D10062

ci: test real dependency installation for pip In the past, the pip smoke test inhibited actual dependency installation, but that fails in different environments for setuptools itself. Since it isn't what we actually want to test (which is pip install), allow this to call home, if HGTESTS_ALLOW_NETIO=1 is set in the environment. Differential Revision: https://phab.mercurial-scm.org/D9950

rust: Add some unit tests for parse_byte_size in config Differential Revision: https://phab.mercurial-scm.org/D10022

rust: Move config value parsing functions to a new module Differential Revision: https://phab.mercurial-scm.org/D10021

rhg: Add support for the blackbox extension Only `command` and `commandfinish` events are logged. The `dirty`, `logsource`, `track` and `ignore` configuration items are not supported yet. To indicate commands executed without Python, a `(rust) ` prefix is added in corresponding log messages. Differential Revision: https://phab.mercurial-scm.org/D10012

blackbox: Remove misleading quotes in config example This example previously looked like quotes were part of configuration file syntax, and the parsed `date-format` value was the part inside of them. This is not the case: config syntax only parses quotes in list values. Instead using that config would result in literal quotes being written to `.hg/blackbox.log` as part of the date format. This changes the example to what was probably intended. Differential Revision: https://phab.mercurial-scm.org/D10011

rust: Add a log file rotation utility This is ported to Rust from `mercurial/loggingutil.py`. The "builder" pattern is used to make it visible at call sites what the two numeric parameters mean. In Python they might simply by keyword arguments. Differential Revision: https://phab.mercurial-scm.org/D10010

rust: Add a `ConfigValueParseError` variant to common errors Configuration files are parsed into sections of key/value pairs when they are read, but at that point values are still arbitrary bytes. Only when a value is accessed by various parts of the code do we know its expected type and syntax, so values are parsed at that point. Let’s make a new error type for this latter kind of parsing error, and add a variant to the common `HgError` so that most code can propagate it without much boilerplate. Differential Revision: https://phab.mercurial-scm.org/D10009

rust: Add config parsing support for more value types * Rust `str` (ASCII or UTF-8) * Integer * Byte quantities Differential Revision: https://phab.mercurial-scm.org/D10008

rust: Introduce a get_bytes_from_os_str utility function It does the same as get_bytes_from_path but takes an `OsStr` instead of a `Path`. The implementation is the same so using either ends up correct but the function name suggests it’s not. Differential Revision: https://phab.mercurial-scm.org/D10007