osutil: fix excessive decref on tuple creation failure in listdir() The stat object would be freed on error for the same reason as the previous patch. makestat() can be inlined, but this patch doesn't change it. https://github.com/python/cpython/blob/2.7/Python/modsupport.c#L292 The __APPLE__ code is untested.

revlog: fix excessive decref on tuple creation failure in parse_index2() Since Py_BuildValue() steals the ownership of "N" arguments, these objects would already be freed if Py_BuildValue() returned NULL. https://github.com/python/cpython/blob/2.7/Python/modsupport.c#L292

tests: correctly match clang-format version >= 10 The previous regex only matched the first digit, so "clang-format version 10.0.0" was recognized as version 1.

state: support validated declaration of nested unfinished ops This enables extensions to define commands that delgate to rebase, evolve, etc. one or more times to also have their own unfinished states for the full sequence of operations without monkey-patching _unfinishedstates. Differential Revision: https://phab.mercurial-scm.org/D8714

absorb: improve message for the case when changeset became empty In changeset f55099982bc5, I introduced the message "became empty and became ...", which I was never very happy with. Raphaël Gomès suggested "became empty as ...". That sounds much nicer. Differential Revision: https://phab.mercurial-scm.org/D8765

relnotes: extend Differential Revision: https://phab.mercurial-scm.org/D8767

merge: document return values of manifestmerge() and calculateupdates() In future patches, I want to add one more return value which represents information which needs to stored and used at commit time. Differential Revision: https://phab.mercurial-scm.org/D8741

merge: refactor code to advise fsmonitor in separate function merge.update() is quite hard to understand, found this an easy win. The end goal is to have better organized merge and mergestate handling and then fix some related bugs. Differential Revision: https://phab.mercurial-scm.org/D8740

mergestate: document what mergestate._results is for Understanding that dict is important for understanding how mergestate is performing operations on dirstate. Differential Revision: https://phab.mercurial-scm.org/D8739

mergestate: rename a helpless variable name to bit helpful one The old variable name `r` makes it ~impossible to understand what does it mean. One can only understand that after going to callers and hoping that its documented there. I also documented return value of the function involved while I was there. Differential Revision: https://phab.mercurial-scm.org/D8738

mergestate: remove unnecessary recordactions() from mergestate class This function is updating dirstate which sounds like not something which a method on mergestate class should do. Also this just calls another function. Lets directly call that function and remove this reducing mergestate responsibility a bit. There was single caller which is updated. Differential Revision: https://phab.mercurial-scm.org/D8737

mergestate: document mergestate records in an organized way This makes clear which mergestate record is used for what and group them based on how they are used right now. Differential Revision: https://phab.mercurial-scm.org/D8719

mergestate: remove unused unsupported related mergestate records I tried to find users of this but was unable to find. Seems like RECORD_OVERRIDE is doing for what they were used before. Differential Revision: https://phab.mercurial-scm.org/D8718

mergestate: add comments about couple of record types and minor reorder I am trying to divide the records into certain groups and then have dedicated objects for them. Taking baby steps in that direction. Differential Revision: https://phab.mercurial-scm.org/D8717

mergestate: remove unrequired RECORD_RESOLVED_OTHER record This was introduced in last cycle however while working on refactoring mergestate, I realized it's unncessary. This will break users who did a merge using previous version, did this kind of storage and before commiting updated the mercurial version. Differential Revision: https://phab.mercurial-scm.org/D8716

mergestate: rename addpath() -> addpathonflict() to prevent confusion addpath() seems to imply that we are adding a new path/entry to the mergestate. Differential Revision: https://phab.mercurial-scm.org/D8715

windows: don’t set `softspace` attribute in `winstdout` Python 2 file objects have the `softspace` attribute (https://docs.python.org/2/library/stdtypes.html#file.softspace), which is used by the print statement to track its internal state. The documentation demands from file-like objects only that the attribute is writable and initialized to 0. Method `file.write()` sets it to 0, but this is not documented. Historically, sys.stdout was replaced by an instance of the `winstdout` class, so it needed to behave exactly the same (the softspace fix was introduced in 705278e70457). Nowadays we don’t replace sys.stdout and don’t use the print statement on `winstdout` instances, so we can safely drop it.

windows: handle file-like objects without isatty() method Copying the function is not nice, but moving around stuff to avoid the circular import didn’t seem to be worth the effort.

exchange: check actually missing revs for obsolete / unstable revs (issue6372) The previous code was using `outgoing.ancestorsof`, which was originally called `outgoing.missingheads` although not containing the missing heads. This confusion was probably the reason why the buggy code was written. The actually outgoing changesets are stored in `outgoing.missing`. By checking all outgoing changesets, we avoid the problem and can show the list of all obsolete or unstable changesets, which is more helpful for the user.

tests: test that push doesn’t complain about unstable changesets if no changes When there’re no outgoing changes, push doesn’t complain about unstable changesets. There is currently a bug (see issue6372) that causes that there is an abort on push when the outgoing changes contain another changeset even if that is not obsolete or unstable. A test case and fix for that is sent in the next patch.

diff: move no-eol text constant to a common location Differential Revision: https://phab.mercurial-scm.org/D8763

revert: fix interactive reverting of end-of-file newline changes The chunk reversal used by `revert -i` in Curses mode was not taking this case into account. Differential Revision: https://phab.mercurial-scm.org/D8762

phabricator: clarify the multiple heads/roots error message for phabsend As pointed out in followup feedback for c1c922391314, it's confusing to say there are multiple items and then only list one. Differential Revision: https://phab.mercurial-scm.org/D8761

error: normalize "unresolved conflicts" error messages with a custom class Differential Revision: https://phab.mercurial-scm.org/D8713

error: unify the error message formats for 'rebase' and 'unshelve' Differential Revision: https://phab.mercurial-scm.org/D8730

debugcommands: create new debugantivirusrunning command This writes the EICAR test file to .hg/cache, in an attempt to trigger an AV scanner's scanning engine. This should let us (in theory) detect some cases when a user's slowness is a result of AV scanning. Differential Revision: https://phab.mercurial-scm.org/D8353

windows: always work around EINVAL in case of broken pipe for stdout / stderr In 29a905fe23ae, I missed the fact that the `winstdout` class works around two unrelated bugs (size limit when writing to consoles and EINVAL in case of broken pipe) and that the latter bug happens even when no console is involved. When writing a test for this, I realized that the same problem applies to stderr, so I applied the workaround for EINVAL to both stdout and stderr. The size limit is worked around in the same case as before (consoles on Windows on Python 2). For that, I changed the `winstdout` class.

tests: make pipes / PTYs non-inheritable in test-stdio.py A following patch requires that to test closing the receiving end of the pipe / PTYs. Even for existing tests, it might be safer to make the lifetime of the pipes / PTYs as short as possible.

util: enhance `nullcontextmanager` to be able to return __enter__ result This makes its functionality and signature equivalent to Python 3.7’s contextlib.nullcontext().

windows: augment docstring of `winstdout` class

discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof` The attribute `missingheads` was recently renamed to `ancestorsof`, as it, despite the old name, doesn’t contain the missing heads but the changesets that were requested (including ancestors) for the outgoing operation. Changing all the users enables to print a warning if the old name is used. There is a good chance that some of the users are buggy because of the old name. Changing them to use the new name makes it more obvious that they are buggy. All users need to be reviewed for bugs. When sending patches for fixing them, the change will be more obvious without having to explain again and again the discrepancy of the old attribute name and what it actually contained.

discovery: fix docstring of `outgoing` class Also, introduce a more correct name `ancestorsof` for what was named `missingheads` before. For now, we just forward `ancestorsof` to `missingheads` until all users are changed. There were some mistakes in the old docstring / name: * `missingheads` (new name: `ancestorsof`) contains the revs whose ancestors are included in the outgoing operation. It may contain non-head revs and revs which are already on the remote, so the name "missingheads" is wrong in two ways. * `missing` contains only ancestors of `missingheads`, so not *all nodes* present in local but not in remote. * `common` might not contain all common revs, e.g. not some that are not an ancestor of `missingheads`. It seems like the misleading name have fostered an actual bug (issue6372), where `outgoing.missingheads` was used assuming that it contains the heads of the missing changesets.

discovery: weaken claim about returned common heads if ancestorsof are given As the test case shows, the claim is not true in general.

cext: remove unused variables Differential Revision: https://phab.mercurial-scm.org/D8760

phabupdate: allow resigning from revisions Differential Revision: https://phab.mercurial-scm.org/D8756

phabupdate: allow revisions to be commandeered Differential Revision: https://phab.mercurial-scm.org/D8755

phabupdate: allow revisions to be reopened Differential Revision: https://phab.mercurial-scm.org/D8754

phabupdate: allow revisions to be marked "closed" (Yes, the transaction name and final state in the comment here do differ.) Differential Revision: https://phab.mercurial-scm.org/D8753

phabupdate: allows revisions to be marked "request-review" Differential Revision: https://phab.mercurial-scm.org/D8752

phabupdate: allow revisions to be marked with "plan changes" Differential Revision: https://phab.mercurial-scm.org/D8751

status: match category text in the option description The ! category in the main status help text is defined as "missing", so use the same text for the help text of --delete. Differential Revision: https://phab.mercurial-scm.org/D8759

purge: classify as a "working directory management" command in help It seems to me that `hg purge` is clearly a "working directory management" command and not a "repository management" command. The working copy is all that's affected, after all; the repo is not affected at all. Differential Revision: https://phab.mercurial-scm.org/D8757

extensions: make `hg help extensions` list disabled extensions with PyOxidizer `setup.py` has a mechanism for generating a list of bundled extensions and their documenation and putting it in a `hgext/__index__.py` file. That file is then read from `extensions.py`, which makes `hg help extensions` work in py2exe binaries. This patch makes it so `setup.py` produces the `hgext/__index__.py` file also when called by PyOxidizer. Thanks to Matt Harbison for pointing out the existence of `__index__.py`! Differential Revision: https://phab.mercurial-scm.org/D8758

phases: sparsify phaseroots and phasesets As final step of dealing with the holes in the phase numbers, make phaseroots and phasesets both dictionaries indexed by the phase number. Further adjust the interface of the C module by pushing the node to revision mapping down as it is cheaper on the C side to deal with revision numbers. Overall, the patch series improves a no-change "hg up" for my NetBSD test repository from 4.7s to 1.3s. Differential Revision: https://phab.mercurial-scm.org/D8698

extdiff: add some comments in diffrevs() It was not obvious to understand the code so I added some comments. Differential Revision: https://phab.mercurial-scm.org/D8690

extdiff: add comments and minor variable renames diffpatch() Some variable names were now confusing as we refactored the code in a separate function. For example, `node1a` leads to ideas why `1a` and not `1`. The variable storing path to patch file was named as `dirX` instead of `fileX`. Renamed these variables and added couple of comments. Differential Revision: https://phab.mercurial-scm.org/D8689

extdiff: remove unrequired do3way argument to diffpatch() When we are diffing patches, there is no 3 way diff. Differential Revision: https://phab.mercurial-scm.org/D8688

extdiff: refactor logic to diff revs of versions of files Now that code for both cases, diffing patches or files is in separate function, it will be better to refactor them more and understand. Differential Revision: https://phab.mercurial-scm.org/D8687

extdiff: refactor logic which does diff of patches The current extdiff code is hard to understand on first look. Took me few hours to grasp the code. Before adding more things, decided to do some refactoring. Differential Revision: https://phab.mercurial-scm.org/D8686

absorb: make it explicit if empty changeset was created If the config rewrite.empty-successor=skip is set, a message "became empty and was dropped" is shown if the changeset became empty. If the config rewrite.empty-successor=keep is set, absorb may create changesets even if they became empty. It’s probably a good idea to make that explicit. Therefore the message is changed to be a combination of both: "became empty and became ...". Repeating the word "became" is not very elegant. This results from the fact that "became" was and is overloaded to indicate both the change from non-empty to empty and the successor relation. In the combinated message, both meanings are used in one sentence.

absorb: consider rewrite.empty-successor configuration This adds support for the recently added rewrite.empty-successor configuration.

rebase: consider rewrite.empty-successor configuration This adds support for the recently added rewrite.empty-successor configuration.

rewriteutil: add utility to check whether empty successors should be skipped

config: add option to control creation of empty successors during rewrite The default for many history-rewriting commands (e.g. rebase and absorb) is that changesets which would become empty are not created in the target branch. This makes sense if the source branch consists of small fix-up changes. For more advanced workflows that make heavy use of history-editing to create curated patch series, dropping empty changesets is not as important or even undesirable. Some users want to keep the meta-history, e.g. to make finding comments in a code review tool easier or to avoid that divergent bookmarks are created. For that, obsmarkers from the (to-be) empty changeset to the changeset(s) that already made the changes should be added. If a to-be empty changeset is pruned without a successor, adding the obsmarkers is hard because the changeset has to be found within the hidden part of the history. If rebasing in TortoiseHg, it’s easy to miss the fact that the to-be empty changeset was pruned. An empty changeset will function as a reminder that obsmarkers should be added. Martin von Zweigbergk mentioned another advantage. Stripping the successor will de-obsolete the predecessor. If no (empty) successor is created, this won’t be possible. In the future, we may want to consider other behaviors, like e.g. creating the empty successor, but pruning it right away. Therefore this configuration accepts 'skip' and 'keep' instead of being a boolean configuration.

commands: use any() instead of `if a or b or c` Small cleanup for future when we have an option to show configs from shared rc. Differential Revision: https://phab.mercurial-scm.org/D8658

manifest: use the same logic for handling flags in _parse as elsewhere Differential Revision: https://phab.mercurial-scm.org/D8684

manifest: tigher manifest parsing and flag use In the manifest line, flags are put directly after the hash, so the parser has been guessing the presence of flags based on the length of the hash. Replace this assumption by an enumeration of the valid flags and removing them from the hash first as they are distinct input values. Consistently handle the expected 256bit length of the SHA1-replacement in the pure Python parser. Check that setting flags will use one of the blessed values. Extend write logic in the C version to handle 256bit hashes as well. Verify that hashes always have exactly the expected length. Since 1070df141718 we should no longer depend on the old extra-byte hack. Differential Revision: https://phab.mercurial-scm.org/D8679

phases: sparsify phase lists When the internal and archived phase was added, allphase became a large, sparsely populated list. This dramatically increased the number of lookup operations for public relations in `phasecache.phase`. As a first step, define allphases and related lists explicitly to contain only the actual phases. Make phasenames a dictionary and create corresponding dictionaries for mapping phase names back to numbers. Adjust various list to be sparse as well with the exception of phaseroots and phasesets members of phasecache. Keep those as a separate step as it involves changes to the C module. Differential Revision: https://phab.mercurial-scm.org/D8697

phases: updatephases should not skip internal phase Historically, updatephases used to skip the secret phase, but wasn't updated when archived and internal were added. Differential Revision: https://phab.mercurial-scm.org/D8696

repoview: use the phasecache directly to determine mutable revisions This speeds up no-change updates from 1.3s to 0.85s for my NetBSD test repository. Differential Revision: https://phab.mercurial-scm.org/D8670

phases: provide a test and accessor for non-public phase roots This decouples users from the implementation details of the phasecache. Also document a historic artifact about the stored phaseroots. Differential Revision: https://phab.mercurial-scm.org/D8694

hg-core: define a `dirstate_status` `Operation` This is 3/3 in a series of patches to improve dirstate status' code. Following in the footsteps of a46e36b82461, we move the main status functionality to an `Operation`. This will most likely be subject to change in the future (what function signature, what parameters, etc., but we will see when `rhg` gets `hg status` support. Differential Revision: https://phab.mercurial-scm.org/D8663

rust-status: improve documentation and readability This patch is 2/3 in the series to improve the dirstate status code. It adds a number of common type aliases to add more obvious semantics to function signatures, improves/adds documentation where necessary and improves one or two patterns to be more idiomatic. Differential Revision: https://phab.mercurial-scm.org/D8662

rust-status: refactor status into a struct The code for `dirstate/status` has grown too large for comfort, this is the first of three patches that try to improve maintainability. In this patch, refactoring dirstate's status into a struct allows for slimming down function signatures drastically, keeping the mental (and maintenance) burden lower, since pretty much all of them shared a few common arguments. This had the pleasant side-effect of simplifying lifetimes a little. This has no observable impact on performance. The next patch will add/improve documentation and refactor some types. I tried to keep new code down to a minimum in this patch because it's already pretty big. Differential Revision: https://phab.mercurial-scm.org/D8661

tests: avoid "magic" nodeids in test-rebase-legacy.t This helps with readability. Differential Revision: https://phab.mercurial-scm.org/D8735

tests: avoid a "magic" nodeid in test-wireproto-command-lookup.t This helps with readability. Differential Revision: https://phab.mercurial-scm.org/D8734

run-tests: replace '#' with '-' in temp path of repos created for tests If we have multiple cases in a test, that leads us to a temp path of format "<temp-path>-<case1>#<case2>". This leads to hg.parseurl() parsing the path and take part after `#` as a branch name. I encountered this bug while adding support for share-safe case in next patch. Differential Revision: https://phab.mercurial-scm.org/D8647

debugcommands: introduce new debugrequirements command This for now just prints out the list of current requirements. In future this will be helpful in reading requirements from couple of sources, and checking which requirement comes from where. Differential Revision: https://phab.mercurial-scm.org/D8632

scmutil: add writereporequirements() and route requires writing through it In upcoming patches, to implement Share Safe plan we will be introducing requires file in store. We need to route all callers to a single function to check for a share-safe requirement and if present, write requirements to .hg/store/requires instead. After this patch, callers directly calling scmutil.writerequires() are only those where we don't have the repo object, for example when initializing the repository object itself. Differential Revision: https://phab.mercurial-scm.org/D8631

extensions: make `hg nonexistent` not crash with PyOxidizer When running `hg nonexistent`, we try to look for extensions that provide that command. We do that by looking for files in the `hgext.__file__` directory. However, PyOxidizer doesn't provide a `__file__`, so we crash when running with PyOxidizer. We should be able to look for the command in built-in extensions, but we seem to already have code for skipping the scan when running in a frozen binary, so I just modified that code instead. By the way, it also seems like we should be able to search for extensions in the `hgext3rd` module, but we don't do that yet either (before or after this patch). Differential Revision: https://phab.mercurial-scm.org/D8750

procutil: avoid use of deprecated tempfile.mktemp() In the previous version, I used tempfile.mktemp() because it seemed to be the only way to open a file from two processes (the Python documentation says the file backing NamedTemporaryFile can’t be opened a second time on Windows). However, it’s possible when passing the O_TEMPORARY flag to the second open. Source: https://stackoverflow.com/a/15235559/6366251

procutil: make _make_write_all() function private Because this function isn’t meant for general use (e.g., it’s Python 2-only), make in a module-private function by prefixing it with `_`.

dispatch: handle late KeyboardInterrupt occurred in run() User can press Ctrl+C while flushing streams in dispatch.run(). In such case, I think exiting with 255 is better than printing Python traceback and exiting with 1.

dispatch: indent run() function I'll add KeyboardInterrupt handling there.

debuginstall: don't translate encoding messages While Microsoft trained us to read mojibake text, it doesn't make sense to print mojibake messages when debugging the exact issue.

merge with stable

patch: refactor content diffing part in separate fn so extensions can wrap Right now extdiff uses it's own logic using archival to diff two versions of file using external diff tools. This makes the extdiff functionality non-extensible. This series is an attempt to refactor core patch and diff functionality so that extdiff can wrap and reuse it. This will help us in using external diffing tools at more places and not just extdiff command only then. Differential Revision: https://phab.mercurial-scm.org/D8685

tests: check that procutil.std{out,err}.write() returns correct result On Windows, we currently don’t fully test the case when the stream is connected to a TTY, but we test the child process side by connecting them to NUL, which is recognized as a TTY by Python. To make the large write test a bit more useful besides checking that it doesn’t crash, we can check that the write() method returns the correct result.

tests: add tests for when stdout or stderr is connected to `os.devnull` The original motivation was that creating PTYs on Windows is not possible, but `NUL` is recognized as a TTY, so we can have at least some test coverage for the TTY case. I think it doesn’t hurt to run the test cases on all systems.

procutil: ensure that procutil.std{out,err}.write() writes all bytes Python 3 offers different kind of streams and it’s not guaranteed for all of them that calling write() writes all bytes. When Python is started in unbuffered mode, sys.std{out,err}.buffer are instances of io.FileIO, whose write() can write less bytes for platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could write less if interrupted by a signal; when writing to Windows consoles, it’s limited to 32767 bytes to avoid the "not enough space" error). This can lead to silent loss of data, both when using sys.std{out,err}.buffer (which may in fact not be a buffered stream) and when using the text streams sys.std{out,err} (I’ve created a CPython bug report for that: https://bugs.python.org/issue41221). Python may fix the problem at some point. For now, we implement our own wrapper for procutil.std{out,err} that calls the raw stream’s write() method until all bytes have been written. We don’t use sys.std{out,err} for larger writes, so I think it’s not worth the effort to patch them.

procutil: move assignments This should probably be part of the previous patch, but folding it results in a less useful word diff, so I decided to keep it separate for review.

procutil: distribute code for stdout It makes sense to have the distinction between Python 2 and 3 at the top level, as we have to fight a different kind of battle on each: On Python 3, we get consistent behavior on all platforms, but need to create correctly-behaving binary streams. On Python 2, we have to account for platform differences.

convert: use bytes for value in extra dict The keys and values of the extra dict are expected to be bytes. Before this fix, there was a crash in mercurial.changelog.encodeextra().

overlayworkingctx: remove unused `nofilechanges()` and `_compact()` methods Differential Revision: https://phab.mercurial-scm.org/D8733

rebase: correctly check for empty commit in in-memory mode The new code has a small overhead in the empty commit case, as a `memctx` object is always created, but I don’t think it’s justified here to duplicate code to optimize a relatively unlikely code path. Differential Revision: https://phab.mercurial-scm.org/D8732

memctx: make `parents()` return list of one element if it’s not a merge It is part of the contract of `parents()` that non-merges return a list of one element. `self._parents` is always a list of two elements in `memctx`. Differential Revision: https://phab.mercurial-scm.org/D8731

commit: factor out empty commit check to `basectx.isempty()` This enables reuse in other places, e.g. those dealing with `memctx`. Differential Revision: https://phab.mercurial-scm.org/D8729

overlayworkingctx: rename misleadingly named `isempty()` method This method is only about whether there are file changes, not about whether the commit will be empty or not. One user of the method was incorrectly assuming the latter meaning, leading to the bug for which a test case was added in D8727. I’ve added a FIXME to the code. The original motivation for the rename was that I want to add `committablectx.isempty()`, that properly checks if a commit will be empty, using the exact same logic as in `repo.commit()`, and I wanted to avoid a name clash. Differential Revision: https://phab.mercurial-scm.org/D8728

rebase: add test case that shows that inmemory rebase does not preserve merges Differential Revision: https://phab.mercurial-scm.org/D8727

rebase: remove now unnecessary logic to allow empty commit when branch changes This was a workaround for a bug in the empty commit check in repo.commit(), where the parent branch name was incorrectly compared with the wdir branch name instead of the branch name passed via `extra`. The bug was fixed in D8724. The workaround was introduced in b2415e94b2f5. Differential Revision: https://phab.mercurial-scm.org/D8725

commit: check branch change on to-be-committed changeset This makes a difference when a branch name is passed via the `extra` argument, which will be used as the branch name for the created changeset. The empty commit check should use that branch name instead of whatever was set on the working directory. Besides fixing the bug for which a test case was extended, it enables to remove a workaround in the rebase extension, which will be done in a follow-up patch (D8725). Differential Revision: https://phab.mercurial-scm.org/D8724

merge with stable

procutil: move comment closer to relevant code

procutil: use mercurial.windows.winstdout only on Python 2 and TTYs Python 3 already works around the bug. The workaround is only needed when writing to consoles. If stdout is a console, sys.stdout.isatty() is true.

procutil: split if condition This prepares the code for subsequent changes where we need to differentiate between the two cases.

procutil: explain better why line buffering is not possible The sentence “On Python 3, buffered binary streams can't be set line-buffered.” was imprecise, as all streams are just Python classes and we can implement our own (which we did).

tests: make subprocess handling reusable for different tests in test-stdio.py

procutil: back out 8403cc54bc83 (make ....procutil.stderr unbuffered) Changeset 8403cc54bc83 introduced code that opens a second file object referring to the stderr file descriptor. This broke tests on Windows. The reason is that on Windows, sys.stderr is buffered and procutil.stderr closed the file descriptor when it got garbage collected before sys.stderr had the chance to flush buffered data. `procutil.stdout` had the same problem for a long time, but we didn’t realize, as in CI test runs, stdout is not a TTY and in this case no second file object is opened.

test-clone-uncompressed: use config to silence the lock warning The ui.timeout.warn option can solve the issue6237 in a cleaner way than what was done in 752da6863e39. That way, possible other unexpected error output will not be silenced. Differential Revision: https://phab.mercurial-scm.org/D8726

hooklib: fix detection of successors for changeset_obsoleted Provide a hook for obsutil.getobsolete to be used with either a transaction or the changes item of the transaction, since hooks only have access to the latter. Use that to find the correct list of revisions with obsmarkers, even new ones, and then filter out revisions with known successors. Move the processing from pretxnclose to txnclose as the transaction access itself is no longer necessary. This is more in line with notify and ensures that sanity checks can abort the transaction first. Differential Revision: https://phab.mercurial-scm.org/D8575

fix: prefetch file contents This prevents the worker subprocesses from contacting the server individually, which is either inefficient, or leads to problems if the connection is shared among them. Differential Revision: https://phab.mercurial-scm.org/D8723

fix: obtain base paths before starting workers This moves calculation of base paths to before work is dispatched. While this does mean that copy tracing will be serialized instead of parallel, it is necessary to be able to prefetch the base contents in a batch, which will likely be more efficient. Differential Revision: https://phab.mercurial-scm.org/D8722

scmutil: allowing different files to be prefetched per revision The old API takes a list of revision separate from the file matcher, and thus provides no way to fetch different sets of files from each revision. In preparation for adding one such usage, I'm changing the API to take a list of (revision, file matcher) tuples instead. Differential Revision: https://phab.mercurial-scm.org/D8721

tests: update test-devel-warnings.t output with chg

tests: make names in test-stdio.py more distinctive This way, more tests can be added without name clashes.

tests: terminate subprocess in test-stdio.py in case of exception If an error happened while reading the output of the subprocess, the pipe / TTY buffer can fill up and prevent that the subprocess ends. Therefore we should terminate the subprocess in case of an exception.

tests: proof test-stdio.py against buffer fill-up With the previous code, it could in theory happen that the pipe / PTY buffer of the child stdout / stderr fills up and the process never finishes. To prevent that, we read all of the stream before waiting for the end of the process. To ensure that the stream reaches EOF when the child finishes, we must close the parent "copy" of the child stdout / stderr.

tests: stablize test-serve.t on Windows I forget the reason that the subprocess on Windows doesn't print this, but all other instances of this are similarly conditionalized, so I didn't think too hard about it. Also, the server needs to be killed so it doesn't prevent the next run from working, especially since the port isn't randomized. Differential Revision: https://phab.mercurial-scm.org/D8720

commitctx: document the manifest writing function Same spirit as for `_filecommit` lets document the input and output before making any change or improvement. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8705

commitctx: reorder some conditional for efficiency in _filecommit Checking if a dict is empty will be faster than comparing text. I don't expect it to be a huge performance win, but still a good (but gratuitous) cleanup to do while we are at it. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8701

commitctx: document _filecommit input and output This is the beginning of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8699

phases: replace magic number by constant Differential Revision: https://phab.mercurial-scm.org/D8695

clonebundles: optional memory-requirement attribution The new REQUIREDRAM option allows a client to skip bundles it isn't expected to handle well, e.g. without swapping. This allows a fallback path to be provided e.g. using zstd level 10 instead of 22. Differential Revision: https://phab.mercurial-scm.org/D8645

util: provide a helper function to estimate RAM size For POSIX systems, it uses sysconf. For Windows, it uses the win32 API directly. Differential Revision: https://phab.mercurial-scm.org/D8644

tests: allow timing difference in output Differential Revision: https://phab.mercurial-scm.org/D8676

documentation: add some internals documentation about bid merge This is an important algorithm that was only documented on the wiki so far. Some update to the algorithm (and associated doc) is to expected in the future since the bid merge algorithm is bug-ridden when it comes to file deletion comes to play. Differential Revision: https://phab.mercurial-scm.org/D8711

cmdserver: document message-encodings and channel output options While writing the previous patch, I noticed these options are undocumented. In my testing, a separate status/error message channel works well in GUI frontend as we no longer have to sort out data and message from mixed outputs. So let's mark it as not experimental.

cmdserver: add option to not exit from message loop on SIGINT Sending SIGINT to server is the only way to interrupt a command running in command-server process. SIGINT will be caught at dispatch.dispatch() if we're lucky. Otherwise it will terminate the serer process. This is fundamentally unreliable as signals are delivered asynchronously. "cmdserver.shutdown-on-interrupt=False" mitigate the issue by making the server basically block SIGINT.

pycompat: remove pycompat.{stdin,stdout,stderr} All users have been changed to use procutil.{stdin,stdout,stderr}, which provide consistent behavior across platforms and Python versions.

pycompat: change users of pycompat.{stdin,stdout,stderr} to use procutil.std* On Python 3, pycompat.{stdin,stdout,stderr} are usually block-buffered even if connected to a TTY. procutil.{stdin,stdout,stderr} provide consistent behavior across platforms and Python versions.