tests: remove a sub-test since we are done with it in test-run-tests.t removing test-failure.t to make sure that in next patch when we re-run failed tests using --retest, it won't get considered unnecessarily. Differential Revision: https://phab.mercurial-scm.org/D8963

storageutil: allow modern hash sizes for fileids Differential Revision: https://phab.mercurial-scm.org/D8783

registrar: fix a documentation typo Differential Revision: https://phab.mercurial-scm.org/D8976

documentation: add `zstd` compression to the internal `revlogs` documentation Differential Revision: https://phab.mercurial-scm.org/D8953

extdiff: move single file handling inside `not per-file` conditional If `--per-file` is passed, we will diffing files one by one, hence there is no need to make this optimization around single file in that case. This optimization is only required when `--per-file` is not passed and we will be opening directory-diff for a single file. Differential Revision: https://phab.mercurial-scm.org/D8971

extdiff: remove dir2root and pass full path as dir2 in _runperfilediff() The only use of `dir2root` was to join with `dir2` to generate the path for other side of diff. Like in previous patch, `dir1a` and `dir1b` are full paths and no longer base names, hence we pass `dir2` as full path too and making `dir2root` unrequired. Differential Revision: https://phab.mercurial-scm.org/D8970

extdiff: pass full paths of `dir1a` and `dir1b` to `_runperfilediff()` Earlier we were passing basename instead of the fullpath and then joining with the tmproot. This is wrong because the user can choose `rev1a` as wdir and in those cases, tmproot should not be joined with the path. I am working on refactoring extdiff logic so that we can have options like `diff --tool` and encountered this. Although this patch in itself makes no difference, however in future when a new caller of `_runperfilediff()` will be added, it will be useful to directly pass on the full paths instead. Differential Revision: https://phab.mercurial-scm.org/D8969

hgweb: compatibility of hgweb.cgi template with Python 3 When running Mercurial on Python 3 hgweb expects the config variable to be a bytestring. The solution proposed by Anton Shestakov is to use a byteprefix to the literal string defined cgi script as provided in this patch for hgweb.cgi, contrib/hgweb.fcgi and contrib/hgweb.wsgi. I am not sure whether this covers all relevant templates, but I couldn't find others. Differential Revision: https://phab.mercurial-scm.org/D8978

genosxversion: don't give up if we can't find a path to hg libraries This doesn't work if you have a PyOxidized hg on $PATH, but everything is fine if you just ignore that problem. Differential Revision: https://phab.mercurial-scm.org/D8975

tests: fix test-demandimport.py on Python 3.9 Starting with Python 3.9, importing importlib.resources (indirectly) imports the zipfile module. Therefore, the module is not suitable for the test. Instead, we can use the ftplib module, which is very unlikely to be imported during the test run.

merge with stable

Added signature for changeset f62bb5d07848

Added tag 5.5.1 for changeset f62bb5d07848

mercurial: force LF endings for *.py, *.{c,h} and *.t in .editorconfig PyCharm on Windows otherwise uses CRLF. Differential Revision: https://phab.mercurial-scm.org/D8947

hooklib: update documentation of changeset_obsoletedfor for changed hook type This updates usage example of changeset_obsoleted to reflect the move from pretxnclose to txnclose made in 04ef381000a8 (hooklib: fix detection of successors for changeset_obsoleted). Differential Revision: https://phab.mercurial-scm.org/D8929

worker: don't expose readinto() on _blockingreader since pickle is picky The `pickle` module expects the input to be buffered and a whole object to be available when `pickle.load()` is called, which is not necessarily true when we send data from workers back to the parent process (i.e., it seems like a bad assumption for the `pickle` module to make). We added a workaround for that in https://phab.mercurial-scm.org/D8076, which made `read()` continue until all the requested bytes have been read. As we found out at work after a lot of investigation (I've spent the last two days on this), the native version of `pickle.load()` has started calling `readinto()` on the input since Python 3.8. That started being called in https://github.com/python/cpython/commit/91f4380cedbae32b49adbea2518014a5624c6523 (and only by the C version of `pickle.load()`)). Before that, it was only `read()` and `readline()` that were called. The problem with that was that `readinto()` on our `_blockingreader` was simply delegating to the underlying, *unbuffered* object. The symptom we saw was that `hg fix` started failing sometimes on Python 3.8 on Mac. It failed very relyable in some cases. I still haven't figured out under what circumstances it fails and I've been unable to reproduce it in test cases (I've tried writing larger amounts of data, using different numbers of workers, and making the formatters sleep). I have, however, been able to reproduce it 3-4 times on Linux, but then it stopped reproducing on the following few hundred attempts. To fix the problem, we can simply remove the implementation of `readinto()`, since the unpickler will then fall back to calling `read()`. The fallback was added a bit later, in https://github.com/python/cpython/commit/b19f7ecfa3adc6ba1544225317b9473649815b38. However, that commit also added checking that what `read()` returns is a `bytes`, so we also need to convert the `bytearray` we use into that. I was able to add a test for that failure at least. Differential Revision: https://phab.mercurial-scm.org/D8928

test: add `test-repo` requirement to `test-check-format` (issue6395) Kindly reported by Tristan Seligmann <mithrandi@mithrandi.net>

commit: clear mergestate also with --amend (issue6304) The `hg commit --amend` uses the in-memory code, which naturally doesn't touch the merge state (well, it shouldn't anyway; I think I've fixed bugs in that area recently). We therefore need to clear the mergestate after calling `repo.commitctx()` since we expect that from `hg commit --amend`. Differential Revision: https://phab.mercurial-scm.org/D8932

tests: add test showing that merge state is not cleared by amend This is slightly modified version of the test case I provided in issue6304. Differential Revision: https://phab.mercurial-scm.org/D8931

requirements: introduce constants for `shared` and `relshared` requirements We add them to `WORKING_DIR_REQUIREMENTS` too as they should be stored in `.hg/requires` and have information about the type of working copy. Differential Revision: https://phab.mercurial-scm.org/D8926

mergestate: replace `addmergedother()` with generic `addcommitinfo()` (API) Storing that a file is resolved for the other parent while merging is just one case of things we will like to store in the mergestate. There are more which we will like to store. This patch replaces `addmergedother()` with a much more generic `addcommitinfo()`. Doing this, we also blinding stores the same key value pair generated by the merge code instead of touching them. Differential Revision: https://phab.mercurial-scm.org/D8923

merge: introduce `addcommitinfo()` on mergeresult object This makes code little bit nicer as we directly update information in the mergeresult object instead of building up a dict first and then setting it. Differential Revision: https://phab.mercurial-scm.org/D8922

merge: use collections.defaultdict() for mergeresult.commitinfo We will be storing info from mergeresult.commitinfo to mergestate._stateextras in upcoming patches, let's make them use same structure so that we don't have to make much efforts in transferring info from one to other. Differential Revision: https://phab.mercurial-scm.org/D8921

mergestate: use _stateextras instead of merge records for commit related info There is a set of information related to a merge which is needed on commit. We want to store such information in the mergestate so that we can read it while committing. For this purpose, we are using merge records and introduced a merge entry state for that. However this won't scale and is not clean way to implement this. This patch reworks the existing logic related to this to use _stateextras and read from it. Right now the information stored is not very descriptive but it will be in next patch. Using _stateextras also makes MERGE_RECORD_MERGED_OTHER useless and only to be kept for BC. Differential Revision: https://phab.mercurial-scm.org/D8920

mergestate: use collections.defaultdict(dict) for _stateextras I want to use this _stateextras more in upcoming patches to store some commit time related information. Using defaultdict will help in cleaner code around checking whether a file exists or not. Differential Revision: https://phab.mercurial-scm.org/D8919

hgweb: minimize scope of a try-block in staticfile() I think the exceptions are only relevant for the `os.stat()` and `open()` calls, and maybe to the `fh.read()` call. Differential Revision: https://phab.mercurial-scm.org/D8936

hgweb: ignore web.templates config when guessing mime type for static content Frozen binaries won't have a file-system path for static content, so I'd like to remove dependence on that. From the documentation, it seems like `mimetypes.guess_type()` only cares about the suffix, so I think it should be enough to pass in just path under the `web.templates` directory. Differential Revision: https://phab.mercurial-scm.org/D8935

hgweb: let staticfile() look up path from default location unless provided This reduces duplication between the two callers. Differential Revision: https://phab.mercurial-scm.org/D8934

hgweb: handle None from templatedir() equally bad in webcommands.py The following paragraph is based just on my reading of the code; I have not tried to test it. Before my recent work on templates in frozen binaries, it seems both `hgwebdir_mod.py` and `webcommands.py` would pass in an empty list into `staticfile()` when running in a frozen binary. That would then result in a variable in that function (`path`) not getting bound before its first use. I then changed that without thinking in D8786 so we passed a `None` value into the function, which made it break in another way (trying to iterate over `None`). Then I tried to fix it up in D8810, but I only changed `hgwebdir_mod.py` for some reason, and it still doesn't actually work in frozen binaries (which seems fair, since was broken before my changes too). This patch just replicates the half-assed "fix" from D8810 in `webcommands.py`, so they look more similar so I can start refactoring them in the same way. Differential Revision: https://phab.mercurial-scm.org/D8933

posixworker: avoid creating workers that end up getting no work If `workers` (the detected or configured number of CPUs) is greater than the number of work items, then some of the workers end up getting 0 work items. Let's not create such workers. Differential Revision: https://phab.mercurial-scm.org/D8927