fuzz: clean up production of seed corpora This was getting out of hand. Differential Revision: https://phab.mercurial-scm.org/D7599

status: add template/json data about whether a file has unresolved conflicts Differential Revision: https://phab.mercurial-scm.org/D7594

status: split morestatus data loading from display This is a small refactoring in preparation for adding more morestatus functionality (notably for templated/JSON output) - the goal is to use the data inside the status display loop, as well as output the overall state in a templatable/structured way. Differential Revision: https://phab.mercurial-scm.org/D7593

phabricator: add a "phabstatus" template keyword We add a "phabstatus" template keyword, returning an object with "url" and "status" keys. This is quite similar to "phabreview" template keyword, but it queries phabricator for each specified revision so it's going to be slow (as compared to the "phabstatus" show view from previous changeset). Differential Revision: https://phab.mercurial-scm.org/D7507

phabricator: add a "phabstatus" show view We add a "phabstatus" show view (called as "hg show phabstatus") which renders a dag with underway revisions associated with a differential revision and displays their status. The revisions shown is a subset of that shown by "work" view, only including revisions with known by Phabricator. Differential Revision: https://phab.mercurial-scm.org/D7506

dirs: fix out-of-bounds access in Py3 The hack for mutating Python's variable-length integers that was ported to py3 in cb3048746dae (dirs: port PyInt code to work on Python 3, 2016-10-08) was reading from ob_digit[1] instead of ob_digit[0] for some reason. Space for ob_digit[1] would only be allocated for integers larger than 30 bits, so we ended up writing to unallocated memory. Also, we would write an integer that's 2^30 times too large, so we would never free these integers. Found by AddressSanitizer. Differential Revision: https://phab.mercurial-scm.org/D7597

manifestcache: add some test involving shares Some issue around shares have been reported. I am adding tests to better cover the share case. The test show that the code behave as expected so far. Differential Revision: https://phab.mercurial-scm.org/D7602

fuzz: fix test-fuzz-targets.t to run with python3 Differential Revision: https://phab.mercurial-scm.org/D7601

py3: fix a bytes vs str issue in remotefilelog extension Differential Revision: https://phab.mercurial-scm.org/D7598

rust-matchers: add doctests for `AlwaysMatcher` Differential Revision: https://phab.mercurial-scm.org/D7527

rust-hg-path: add method to get part of a path relative to a prefix This will be used in the next patch in this series. Differential Revision: https://phab.mercurial-scm.org/D7526

rust-matchers: improve `Matcher` trait ergonomics `VisitChildrenSet` has no need to own the set, this will save allocations. The `file_set` return type change is motivated by both ergonomics and... being able to compile code. The `AlwaysMatcher` does not store a `file_set`, which requires it to return an owned `HashSet`, which in turn would change our return type to `Cow<&HgPath>` (lifetimes omitted). This is both un-ergonomic and troublesome for more complex lifetime issues (especially with the upcoming `FileMatcher` in the following patch). Differential Revision: https://phab.mercurial-scm.org/D7525

rust-dirs-multiset: use `AsRef` instead of concrete types when possible I also renamed `vec` to `dirstate`, because it was not a great name. Differential Revision: https://phab.mercurial-scm.org/D7524

logcmdutil: call _exthook() in changesettemplater Class changesetprinter has an _exthook() method that is called in _show() before the patch is displayed. Call the method as well in changesettemplater._show(). Differential Revision: https://phab.mercurial-scm.org/D7505

phabricator: fix processing of tags/desc in getoldnodedrevmap() It seems that the previous logic was wrong (it essentially comes from changeset 3ab0d5767b54 where the result got accumulated instead of early returned). First of all, the "continue" in first "if m:" is useless because we're at the end of the loop. Then, the algorithm seems weird because we will process all predecessors of a node and possibly override `toconfirm[node]` for each of these having a tag (maybe this doesn't happen, but still). Finally, we would also override `toconfirm[node]` when the "Differential Revision: " is found in changeset description. Maybe this is not a big deal when there is no mix of local tag and changeset description update? The logic is changed so that the loop on predecessors stops upon first match of a tag and so that the changeset description is only checked if no tag was found. Therefore, `toconfirm[node]` is only set once. Differential Revision: https://phab.mercurial-scm.org/D7513

tests: replace [[]] bashism with portable [] invocation In this case nothing fancy is required. Differential Revision: https://phab.mercurial-scm.org/D7596

rust-hg-path: implement `Display` for `HgPath` and `HgPathBuf` This is useful when debugging, to get a human readable output instead of an array of `u8`. Differential Revision: https://phab.mercurial-scm.org/D7523

rust-performance: introduce FastHashMap type alias for HashMap Rust's default hashing is slow, because it is meant for preventing collision attacks. For all of the current Rust code, we don't care about those attacks, because if an person with bad intentions has write access to your repo, you have other issues. I've chosen to use the TwoXHash crate because it was made by a reputable member of the Rust community and has very good benchmarks. For now it does not seem to improve performance by much for the current code, but it's something else to not worry about when benchmarking code: in a previous experiment with copytracing in Rust, it accounted for more than 10% of the time of the entire script. Differential Revision: https://phab.mercurial-scm.org/D7116

mail: use procutil.shellsplit instead of bytes.split to parse command Differential Revision: https://phab.mercurial-scm.org/D7541

annotate: describe --skip as taking a revset It's obvious to me, but probably wouldn't be obvious to a novice user. Differential Revision: https://phab.mercurial-scm.org/D7557

automation: use latest Windows AMI as base It looks like the previous base image no longer exists. I guess Amazon expires them or something. Let's switch to the newest version of the equivalent image. Differential Revision: https://phab.mercurial-scm.org/D7571

tests: stabilize test-extdiff.t on Windows This goes with 765a9c299c44. I'm not sure if we care about the missing file being spelled `nul`, so I removed it from the glob to be explicit about it. The line needed to be special cased anyway because of the quoting on Windows. Differential Revision: https://phab.mercurial-scm.org/D7572

exchange: fix an attempt to format a list into bytes This goes with 27c6d6f53d46. Differential Revision: https://phab.mercurial-scm.org/D7583

status: remove pointless filtering by alwaysmatcher in morestatus The code has been like this since it was imported from FB's hg-experimental repo. It has been like that even since it was added in that repo. So I don't know why it looks that way. Perhaps the idea was to one day filter the unresolved paths by any patterns provided by the user. We can add a matcher back if we ever decide to do that. Differential Revision: https://phab.mercurial-scm.org/D7591

tests: add test for Rust formatting We enforce formatting for Python and C. It makes sense to do it for Rust as well. Since our rustfmt.toml relies on unstable rustfmt features, we need to use a Nightly rustfmt with --unstable-features in order for it to work. This is a bit hacky and I would prefer we remove this requirement. But for now, this commit assumes this is the way things must be and we go out of our way to detect and use the rustfmt from the "nightly" toolchain, as installed via rustup. We had to add some environment variables to the tests to make the Rust binaries happy. Otherwise when running rustfmt we get an error about no default toolchain being installed. Differential Revision: https://phab.mercurial-scm.org/D7579

rust: run rustfmt # skip-blame automated reformatting Differential Revision: https://phab.mercurial-scm.org/D7578

fuzz: clean up some repetition on building parsers.so fuzzers There was a lot of repetition here that I realized could be cleared out. I think there's some more work we could do here, but this is enough for now. Differential Revision: https://phab.mercurial-scm.org/D7568

fuzz: remove legacy setup for running fuzzers outside oss-fuzz We don't need this anymore. Differential Revision: https://phab.mercurial-scm.org/D7567

tests: finally fix up test-fuzz-targets.t It's been failing on my workstation for a while, since I have a new enough LLVM that I had the fuzzer goo, but not so new that I actually had FuzzedDataProvider. This is a better solution all around in my opinion. I _believe_ this should let us run these tests on most systems, even those using GCC instead of clang. That said, my one attempt to test this on my macOS laptop failed miserably, and I don't feel like doing more work on this right now. Differential Revision: https://phab.mercurial-scm.org/D7566

fuzz: make standalone_fuzz_target_runner call LLVMFuzzerInitialize Otherwise some of our fuzzers crash when they try and use Python. Differential Revision: https://phab.mercurial-scm.org/D7565

fuzz: use a more standard approach to allow local builds of fuzzers This is taken from the (improved since we started fuzzing) guide on ideal integrations. Rather than have our own wonky targets for building outside the fuzzer universe, we have a driver program we carry along and use when we're not using LibFuzzer. This will let us jettison a fair amount of goo. contrib/fuzz/standalone_fuzz_target_runner.cc is https://github.com/google/oss-fuzz/ file projects/example/my-api-repo/standalone from git revision c4579d9358a73ea5dbcc99cb985de1f2bf76dcf7, reformatted with out clang-format settings and a no-check-code comment added. It allows running a single test input through a fuzzer, rather than performing ongoing fuzzing as libfuzzer would. contrib/fuzz/FuzzedDataProvider.h is https://github.com/llvm/llvm-project/ file /compiler-rt/include/fuzzer/FuzzedDataProvider.h from git revision a44ef027ebca1598892ea9b104d6189aeb3bc2f0, reformatted with our clang-format settings and a no-check-code comment added. We can discard this if we instead want to add an hghave check for a new enough llvm that includes FuzzedDataProvder.h in the fuzzer headers. Differential Revision: https://phab.mercurial-scm.org/D7564

fuzz: use a variable to allow specifying python-config to use Eventually we should probably default this to just `python-config` and have the oss-fuzz build.sh script specify the sanpy python-config, but for now this lets us make progress. Differential Revision: https://phab.mercurial-scm.org/D7563

fuzz: suppress deprecated-register warnings in our compile These come from the Python.h headers still using the `register` keyword and our use of C++17. I think this will go away when we're using Python 3 for our fuzzing, but that can come later. Differential Revision: https://phab.mercurial-scm.org/D7562

fuzz: follow modern guidelines and use LIB_FUZZING_ENGINE This will make our lives easier in an upcoming change, but it's also how we're supposed to set things up anyway. Differential Revision: https://phab.mercurial-scm.org/D7561

fuzz: always define LLVMFuzzerInitialize() even if we don't need it This will make it easier to test our fuzzers outside oss-fuzz. Differential Revision: https://phab.mercurial-scm.org/D7560

fuzz: remove debug prints from revlog_corpus.py Differential Revision: https://phab.mercurial-scm.org/D7559

fuzz: fix an unused result on getcwd() in pyutil clang was rightly complaining about this, so let's just fix it. Differential Revision: https://phab.mercurial-scm.org/D7558

upgraderepo: add a config option for parallel computation The option is put to use to compute new copy tracing side data in parallel. It use the multiprocessing module as it had the appropriate primitive for what we needed. Gregory Szorc had concerned on windows so we disabled it there. See inline comment for details on the parallel implementation.

upgrade-repo: colorize some of the output Having clear color for requirement added and removed is useful.

tests: remove hardcoded errno values Fixes test failures on hurd-i386 Ref: https://bugs.debian.org/946178 Differential Revision: https://phab.mercurial-scm.org/D7556

tests: expect return status 255 on exception for test-blackbox.t with chg Differential Revision: https://phab.mercurial-scm.org/D7554

tests: fix command name in test-blackbox.t to be bytes This command is only used when tests are run with --with-chg, so this was missed before. Differential Revision: https://phab.mercurial-scm.org/D7553

tests: fix deprecation warning about regex flags not at beginning of expr This may only show up when running the tests under python3.6+. Currently the only test that does this is test-patchbomb-tls.t, and it only uses (?i), so that's all that's handled at the moment. Differential Revision: https://phab.mercurial-scm.org/D7552

tests: fix test-chg.t to work with py3 (no setprocname) Differential Revision: https://phab.mercurial-scm.org/D7551

amend: check for file modifications when updating dirstate (issue6233) Previously, we called dirstate.normal(f), which would put information into the dirstate claiming that the file on disk is what it "should be" for the current checkout, and it would have the size and timestamp of the most recent modification to the file (which is not necessarily the one we just committed). If the file was modified while the commit message editor was open, we would put incorrect information into the dirstate. Differential Revision: https://phab.mercurial-scm.org/D7521

graft: never set both parents equal in the dirstate (issue6098) merge.graft() can set both parents equal in the dirstate when keepparent=True. We don't seem to set that in core, but the evolve extension does use it. So I couldn't figure out a way to add a test for this patch in core. Differential Revision: https://phab.mercurial-scm.org/D7549

graft: extract `repo['.']` to local variable It's used in two places and I'm about to use it more. Differential Revision: https://phab.mercurial-scm.org/D7548

graft: rename `pctx` argument to `base` since that's what it is The new name better matches the docstring. It also frees up `pctx` to be used for something else (next patch). Differential Revision: https://phab.mercurial-scm.org/D7547

exchange: replace a "not x in ys" by more Pythonic "x not in ys" Found by one of our (Google-)internal tools. Differential Revision: https://phab.mercurial-scm.org/D7546

packaging: include defaultrc/*.rc instead of default.d/*.rc Differential Revision: https://phab.mercurial-scm.org/D7555

merge with stable

py3: make a pycompat.osdevnull, use it in extdiff Differential Revision: https://phab.mercurial-scm.org/D7545

subrepo: use pycompat.open directly instead of importing open from pycompat Differential Revision: https://phab.mercurial-scm.org/D7544

rust-dirs: address failing tests for `dirs` impl with a temporary fix https://phab.mercurial-scm.org/D7252 (5d40317d42b7083b49467502549e25f144888cb3) introduced a regression in Rust tests. This is a temporary fix that replicates the behavior of the C and Python impl, pending the resolution of the discussion (in the phabricator link) about how we actually want to solve this problem. Differential Revision: https://phab.mercurial-scm.org/D7503

cleanup: fix docstring formatting This is just removing the b'' prefix (except demandimportpy2), and making sure it is triple quoted. I skipped the mapping.py module in zope because that's 3rd party code. Differential Revision: https://phab.mercurial-scm.org/D7539

copies: split the combination of the copies mapping in its own function In some case, this part take up to 95% of the copy tracing that take about a hundred second. This poor performance comes from the fact we keep duplciating and merging dictionary that are mostly similar. I want to experiment with smarter native code to do this, so I need to isolate the function first.

copies: do not initialize the dictionary with root in changeset copies We don't strictly need the dict to be prefiled with `{}`. Not doing so will make the next changeset simpler. This part of a preparation to use native code for this part of copy tracing.

copies: expand `_chain` variable name to make the function easier to read This came up while explaining what the function is about. I find the function easier to follow that way.

copies: show we can read data from the "first" mapping In practice this make no difference since `t` is a copy of `a`. Having this change before hand will make the next cleanup changeset clearer. This is work on the road to using some native code for some performance critical part of copy tracing.

copies: return consistent type from revinfo The return is expected to be a list, so we should return a list. This make the use of this function from type-checked language (eg: rust) much simpler.