rust: fix non-utf8 char in requirements.rs Apparently Phabricator detect `rust/hg-core/src/requirements.rs` file as non utf8 ‽, and mark it as binary. During application it ended up being non-utf8 and this made Rust (and as a result heptapod) very angry:: error: couldn't read hg-core/src/requirements.rs: stream did not contain valid UTF-8 --> hg-core/src/lib.rs:11:9 | 11 | pub mod requirements; | ^^^^^^^^^^^^ error: aborting due to previous error error: could not compile `hg-core`. The after "fixing", the file content has no character matching the following regexp: [^0-9-a-zA-Z /(|).,{}!\[\]:"&=>?_*-;<`'#] So we should be fine, unless Phabricator does something funny again. Differential Revision: https://phab.mercurial-scm.org/D9444

run-tests: use a context manager when looking for available ports Differential Revision: https://phab.mercurial-scm.org/D9441 Differential Revision: https://phab.mercurial-scm.org/D9452

dispatch: print the version of each extension in the bug report, if available Sometimes the wrong extensions is blamed, so we might as well print the version info for all of them. Additionally, since the internal extensions are never blamed, this is a good way to make the pygit2 version available in a bug report. Differential Revision: https://phab.mercurial-scm.org/D9440

dispatch: sort the loaded extension names in the bug report This makes a long list of extensions easier to read. On very rare occasion I've seen issues where the load order mattered, however that info should still be obtainable with `hg config extensions`. Differential Revision: https://phab.mercurial-scm.org/D9439

dispatch: quote the extension when printing the bug report I think this reads better in the wall of text. Differential Revision: https://phab.mercurial-scm.org/D9438

dispatch: print the version of the extension being blamed in a bug report I don't know of a lot of extensions using this, but it seems like useful info in a bug report. Differential Revision: https://phab.mercurial-scm.org/D9437

git: show the version of `pygit2` with verbose version output This seems like useful info to have when debugging. I followed the precedent of hg-git, which prints something like: hggit external 0.9.0a1 (dulwich 0.19.15) We don't have a version number assigned (because it's internal), so it's just the parenthetical. Differential Revision: https://phab.mercurial-scm.org/D9436

git: add the standard `testedwith` attribute Otherwise this shows up as an external extension. Differential Revision: https://phab.mercurial-scm.org/D9435

tests: add a comment that we're purposely testing py2 extension attributes Avoid someone unknowingly removing test coverage. There are tests for a properly byteified `testedwith` a few lines down. I don't see similar for `buglink`, but it's a trivial conversion to bytes, so I'm not concerned about testing the expected/wanted extension state. Differential Revision: https://phab.mercurial-scm.org/D9434

extensions: avoid a crash when the version isn't properly byteified on py3 We already force bytestr on the `testedwith` and `buglink` attributes in dispatch.py when generating a bug report with a similar comment about not every extension being ported to py3. We should do the same here, so the function can be properly typed. Differential Revision: https://phab.mercurial-scm.org/D9433

formatting: drop a few extra double quotes in docstrings These were made obvious by the reformatting in D9430. Differential Revision: https://phab.mercurial-scm.org/D9432

copies: introduce the hg-cpython wrapper for `combine_changeset_copies` This patch focus on the `hg-cpython` part of this work. Bridging the python code with the new rust code in `hg-core`. The next patch will actually plug this in the python code. The rust code use multiple Python callback, python related error within this callback are not expected unless they are a programming error or a data corruption. In addition, these callback will slowly be replaced by native Rust code. For these reasons, we use will deal with unexpected error within this callback using rust Panic and let the `rust-cpython` layer deal with raising a Python exception. The code dealing with the ChangedFile instance is repeating itself a lot. I did not factor these duplication out because that whole code will get replaced by entirely different one in a handful of changesets. Differential Revision: https://phab.mercurial-scm.org/D9298

copies: introduce a basic Rust function for `combine_changeset_copies` This new function mirror the python code. This first implementation does a lot of data copies and is therefore quite slow. However my goal here is to create a simple "frame" from where to start adding optimization. This patch focus on the `hg-core` part of this work. Coming patches will do the necessary `hg-cpython` work to be able to use this from Python. Differential Revision: https://phab.mercurial-scm.org/D9297

hghave: adjust the detection of `pylint` for modern versions I have pylint 2.6.0 installed locally, and it only has a single space. I have no idea when this changed. Differential Revision: https://phab.mercurial-scm.org/D9431

formating: upgrade to black 20.8b1 This required a couple of small tweaks to un-confuse black, but now it works. Big formatting changes come from: * Dramatically improved collection-splitting logic upstream * Black having a strong (correct IMO) opinion that """ is better than ''' Differential Revision: https://phab.mercurial-scm.org/D9430

osutil: reformat triple-quoted string so black doesn't confuse itself Differential Revision: https://phab.mercurial-scm.org/D9429

merge: remove spurious ' and trailing whitespace from triple-quoted string Differential Revision: https://phab.mercurial-scm.org/D9428

cleanup: fix a few recent black formatting violations Differential Revision: https://phab.mercurial-scm.org/D9427

rhg: check that .hg/requires is ASCII Differential Revision: https://phab.mercurial-scm.org/D9400

rhg: exit with relevant code for unsupported requirements Differential Revision: https://phab.mercurial-scm.org/D9399

revlog: store new index entries as binary For a pure-Python unbundle of the current NetBSD test repository, this results in a 10% peak RSS reduction. Using the C revlog index, it shows 25% peak RSS reduction. This is a direct result of avoiding at least 8 objects per new changeset or 200 Bytes+ on AMD64. Differential Revision: https://phab.mercurial-scm.org/D9162

packaging: enable rust extensions on centos Test on CentOS 7, repository with ~170000 tracked files, no untracked files: 10 runs with this enabled: -- Run #0 time: 0.6519973278045654 -- Run #1 time: 0.6933724880218506 -- Run #2 time: 0.7512078285217285 -- Run #3 time: 0.7517638206481934 -- Run #4 time: 0.5966529846191406 -- Run #5 time: 0.5960886478424072 -- Run #6 time: 0.5940573215484619 -- Run #7 time: 0.5963726043701172 -- Run #8 time: 0.6048009395599365 -- Run #9 time: 0.603604793548584 10 runs without this enabled: -- Run #0 time: 2.127584457397461 -- Run #1 time: 2.066192865371704 -- Run #2 time: 2.0831892490386963 -- Run #3 time: 2.077716588973999 -- Run #4 time: 2.07608962059021 -- Run #5 time: 2.072899341583252 -- Run #6 time: 2.094369888305664 -- Run #7 time: 2.067504644393921 -- Run #8 time: 2.069610834121704 -- Run #9 time: 2.0567898750305176 Differential Revision: https://phab.mercurial-scm.org/D9294

tests: use `testrepohg` in one more place in test-check-code.t This is already used elsewhere in the test to access the current hg repo, and avoids an error about the unknown `revlog-compression-zstd` when C extensions aren't built. The only other such error is in test-check-interfaces.py, but I don't see a way to avoid it other than to create an empty scratch repo. Differential Revision: https://phab.mercurial-scm.org/D9364

setup: copy pythonXY.dll next to the hg.exe wrapper when building This avoids the problem of having the newly built binary complaining that it can't find the DLL. There is an option in the python.org installer to add the python install to PATH (which defaulted to "on" with py2, and therefore was not an issue up to this point), but that makes switching between python versions harder. This shouldn't be an issue with the PyOxidizer binary, but that current has issues running some of the tests, and took noticeably longer to build last time I tried it. Differential Revision: https://phab.mercurial-scm.org/D9362

exthelper: update the examples to be python3 complaint Differential Revision: https://phab.mercurial-scm.org/D9368

helptext: byteify extensions examples Make it easier on the copy/paste crowd. I haven't looked at the other help text to see if there are other instances; I was just looking to confirm `buglink` is meant to be bytes and this popped up along with the code. Differential Revision: https://phab.mercurial-scm.org/D9367

helptext: fix sentence in extensions documentation Differential Revision: https://phab.mercurial-scm.org/D9366

ui: ensure `getpass()` returns bytes Previously, this could return either bytes or str. I'm not sure which direction we should go in, but since the input is bytes, I guess bytes makes sense as output. `debuguigetpass` crashed because it assumed bytes would be returned, `sslcontext.load_cert_chain()` is happy with bytes or str if the type info in PyCharm is correct, and `smtplib.SMTP.login()` wants str. I couldn't figure out how to test this, because the test stalls for input with `echo test | hg debuguigetpass --config ui.interactive=1`, likely because it drains stdin before prompting. The custom input reading with `ui.nontty=1` does not. I'm also a bit concerned with all of this encoding/decoding. The existing code in the mail module uses `encoding.strfromlocal()`, but the username and password are ascii encoded/decoded in `mercurial.url.passwordmgr.find_user_password()` with `pycompat.{str,bytes}url()`. I'm not sure if this inconsistency could cause subtle compatability issues. Differential Revision: https://phab.mercurial-scm.org/D9375