heptapod-ci: move version prints closer to the start This makes debugging a lot easier if anything is to go wrong, and shows output earlier.

pytype: only try the hacky way of finding PYTHON if not provided This allows us to work in more environments, like when using pyenv. This syntax is compatible with all POSIX shells.

dummysmtpd: fix EOF handling on newer versions of OpenSSL Explanations inline.

test-install: add new glob for the upgrade notice in newer versions of pip We only globbed for the old warning, newer versions of pip use a cleaner one.

rust: use `.cargo/config.toml` instead of `.cargo/config` This has been deprecated for a while now and we don't support Rust versions that only understand the old path.

rust: apply clippy lints They are at most harmless and at best make the codebase more readable and simpler.

rust: change minimum supported version everywhere applicable This will help users and downstream packaging.

rustfmt: format the codebase with nightly-2024-07-16 The CI has moved to a newer nightly, which slightly changes how it wraps comments (which is the very option we use nightly for).

hghave: update detection of black version to a newer minimum The CI has moved to version 23.3.0, which is the last one to support 3.7 at runtime.

black: format the codebase with 23.3.0 The CI has moved to 23.3.0, which is the last version that supports 3.7 at runtime, so we should honor this change. # skip-blame mass-reformating only

pytype: work around wrong ImportError flagging As documented in https://github.com/google/pytype/issues/163, newer versions of Pytype do not understand caught `ImportError`, so we temporarily ignore them where applicable.

zeroconf: fix boolean return value This was (wrongly) flagged by Pytype as being undefined since it doesn't seem to understand `try` blocks. However, the caller is expecting a boolean and the fix to appease Pytype is simple, so we do both.

Backout accidental publication of a large range of revisions I accidentally published 25e7f9dcad0f::bd1483fd7088, this is the inverse.

Latest image and pytype fix

dummysmtpd: fix EOF handling on newer versions of OpenSSL Explanations inline.

test-install: add new glob for the upgrade notice in newer versions of pip We only globbed for the old warning, newer versions of pip use a cleaner one.

Try the full CI run

WIP test new CI image

rust: use `.cargo/config.toml` instead of `.cargo/config` This has been deprecated for a while now and we don't support Rust versions that only understand the old path.

rust: apply clippy lints They are at most harmless and at best make the codebase more readable and simpler.

rustfmt: format the codebase with nightly-2024-07-16 The CI has moved to a newer nightly, which slightly changes how it wraps comments (which is the very option we use nightly for).

hghave: update detection of black version to a newer minimum The CI has moved to version 23.3.0, which is the last one to support 3.7 at runtime.

black: format the codebase with 23.3.0 The CI has moved to 23.3.0, which is the last version that supports 3.7 at runtime, so we should honor this change. # skip-blame mass-reformating only

pytype: work around wrong ImportError flagging As documented in https://github.com/google/pytype/issues/163, newer versions of Pytype do not understand caught `ImportError`, so we temporarily ignore them where applicable.

zeroconf: fix boolean return value This was (wrongly) flagged by Pytype as being undefined since it doesn't seem to understand `try` blocks. However, the caller is expecting a boolean and the fix to appease Pytype is simple, so we do both.

convert: fix various leaked file descriptors Some of these only leaked if an exception occurred between the open and close, but a lot of these leaked unconditionally. A type hint is added to `parsesplicemap` because otherwise this change caused pytype to change the return type from this to `Dict[nothing, nothing]`.

convert: stringify `shlex` class argument The documentation is handwavy, but typeshed says this should be `str`[1]. I'm not sure if this is the correct encoding (vs `fsencode` or "latin1" like the tokens returned by the proxy class). While we're here, we can add a few more type hints that would have caused pytype to flag the problem. [1] https://github.com/python/typeshed/blob/6a9b53e719a139c2d6b41cf265ed0990cf438192/stdlib/shlex.pyi#L51

typing: add trivial type hints to the convert extension's common modules This started as ensuring that the `encoding` and `orig_encoding` attributes has a type other than `Any`, so pytype can catch problems where it needs to be str for stdlib encoding and decoding. It turns out that adding the hint in `mercurial.encoding` is what was needed, but I picked a bunch of low hanging fruit while here. There's definitely more to do, and I see a problem where `shlex.shlex` is being fed bytes instead of str, but there are not enough type hints yet to make pytype notice.

convert: drop a duplicate implementation of `dateutil.makedate()` I noticed this because the signature generated by pytype recently changed to be less specific. When the method was introduced back in 337d728e644f, `util.makedate()` didn't take an optional timestamp arg. But now it does, and the methods are the same (except the `dateutil` version validates that the timestamp isn't a negative value). I left the old method in place in case anyone has custom convert code that monkey patches it.

revlog: use mmap by default is pre-population is available Using mmap has a great impact of memory usage on server, and a good impact on performance in multiple case. Now that we pre-populate memory mapping by default, there is case where it using mmap is slower. So we use it by default (if pre-population is available). Further work to reduce the performance impact of the pre-population will be done later. Some benchmark below (using the same setup as 522b4d729e89): As for 522b4d729e89 the impact on small repository like Mercurial or Pypy is tiny, ~1% best. However for large repositories we see some performance improvement without seeing the performance regression that we could have without pre-populate. ##### For netbeans ### data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog ## benchmark.name = hg.command.log # bin-env-vars.hg.flavor = rust # benchmark.variants.limit-rev = 1 # benchmark.variants.patch = yes no-mmap: 0.171579 mmap: 0.166311 (-3.07%, -0.01) # bin-env-vars.hg.flavor = default no-mmap: 0.170716 mmap: 0.165218 (-3.22%, -0.01) # benchmark.variants.patch = no # benchmark.variants.rev = tip no-mmap: 0.140862 mmap: 0.137566 (-2.34%, -0.00) ## benchmark.name = hg.command.unbundle # bin-env-vars.hg.flavor = rust # benchmark.variants.issue6528 = disabled # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle no-mmap: 0.238038 mmap: 0.239912 no-populate: 0.cbd4c9 (+11.71%, +0.03) #### For Mozilla ### data-env-vars.name = mozilla-try-2019-02-18-ds2-pnm # benchmark.name = hg.command.log # bin-env-vars.hg.flavor = rust # bin-env-vars.hg.py-re2-module = default # benchmark.variants.limit-rev = 1 # benchmark.variants.patch = yes no-mmap: 0.258440 mmap: 0.237813 (-7.98%, -0.02) # benchmark.variants.limit-rev = 10 no-mmap: 1.235323 mmap: 1.213578 (-1.76%, -0.02) ## benchmark.name = hg.command.push # bin-env-vars.hg.flavor = rust # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev no-mmap: 4.790135 mmap: 4.668971 (-2.53%, -0.12) no-populate: 4.841141 (+1.06%, +0.05) ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog ## benchmark.name = hg.command.log # bin-env-vars.hg.flavor = default # benchmark.variants.limit-rev = 1000 # benchmark.variants.rev = tip no-mmap: 0.206187 mmap: 0.197348 (-4.29%, -0.01) ## benchmark.name = hg.command.push # bin-env-vars.hg.flavor = default # benchmark.variants.explicit-rev = none # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev no-mmap: 4.768259 mmap: 4.798632 no-populate: 4.953295 (+3.88%, +0.19) # benchmark.variants.revs = any-100-extra-rev no-mmap: 4.785946 mmap: 4.903618 no-populate: 5.014963 (+4.79%, +0.23) ## benchmark.name = hg.command.unbundle # bin-env-vars.hg.flavor = default # benchmark.variants.issue6528 = disabled # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle no-mmap: 1.400121 mmap: 1.423411 no-populate: 1.585365 (+13.23%, +0.19)