nodemap: rename a variable to prevent shawoding "_" This will make pyflakes happy with the next changeset. Differential Revision: https://phab.mercurial-scm.org/D8412

rust-chg: indent process_message() to prepare mass rewrite to futures-0.3 I'll start upgrading the codebase to modern async/await-based implementation, which cannot be done incrementally. This is the last non-breaking patch to prepare for the rewrite. Differential Revision: https://phab.mercurial-scm.org/D8403

rust-chg: move get_umask() call out of run() function run() will be an async function, but get_umask() isn't thread safe. Differential Revision: https://phab.mercurial-scm.org/D8402

rust-chg: leverage impl trait at argument position Differential Revision: https://phab.mercurial-scm.org/D8401

setup: link osutil.so to libsocket on Solaris/illumos (issue6299)

revset: import `unstable()` from the evolve extension Differential Revision: https://phab.mercurial-scm.org/D8404

fastexport: make a diagnostics message more localizable Differential Revision: https://phab.mercurial-scm.org/D8398

rust-chg: use "crate::" to import local modules I feel it's easier to follow to resolve modules from the crate root than relative path from self module. Differential Revision: https://phab.mercurial-scm.org/D8400

rust-chg: upgrade to 2018 edition and remove useless extern crates Differential Revision: https://phab.mercurial-scm.org/D8399

mergestate: store about files resolved in favour of other Committing a merge sometimes wrongly creates a new filenode where it can re-use an existing one. This happens because the commit code does it's own calculation and does not know what happened on merge. This starts storing information in mergestate about files which were automatically merged and the other/remote version of file was used. We need this information at commit to pick the filenode parent for the new commit. This issue was found by Pierre-Yves David and idea to store the relevant parts in mergestate is also suggested by him. Somethings which can be further investigated are: 1) refactoring of commit logic more to depend on this information 2) maybe a more generic solution? Differential Revision: https://phab.mercurial-scm.org/D8392

histedit: add test that an invalid editor doesn't bork client state We had a report of a situation like this borking a user at Google, but I can't reproduce it in a test. Let's at least backstop the issue with a test, so we don't accidentally introduce such a bug... Differential Revision: https://phab.mercurial-scm.org/D8391

rust-chg: silence warning about dated coding style Differential Revision: https://phab.mercurial-scm.org/D8384

rust-chg: update name of the server process This is a copy of updateprocname() of hgclient.c. At this point, rust-chg is basically functional. I did dogfooding for a couple of weeks in 2018. There are a few remaining tasks: a. loop detection by CHGINTERNALMARK b. forward unsupported commands (notably serve -d) to real hg c. better handling of early server exception d. modernize codebase (2018 edition, impl trait, async/await) For (d), we'll probably want to switch to async-std, but I'm thinking of upgrading to Tokio 0.2 as an intermediate step since process API isn't ported to async-std yet. I'm pretty sure future migration to async-std will be painless compared to the mass rewrite from futures-0.1 to 0.3. https://github.com/async-rs/async-std/issues/22 Differential Revision: https://phab.mercurial-scm.org/D8383

rust-chg: send client side umask to server This is equivalent to forwardumask() of hgclient.c. Differential Revision: https://phab.mercurial-scm.org/D8382

rust-chg: add config validation and process returned instructions This is the reimplementation of runinstructions() and main() in chg.c. In Rust version, we only pass in early arguments to the server as the locator doesn't know the full arguments. This should be fine since these arguments are just passed in to _earlyparseopts() and _parseconfig(), which means the server doesn't need full arguments. Another difference is the handling of the "exit <code>" instruction. In Rust version, we can simply reuse the connection instead of "exit(code)" as the command error isn't displayed yet. That's because the client-side stdio is not attached until the connection is validated. This behavior is cleaner than C, but it also means that the early server exception wouldn't be propagated to client because stderr isn't attached. So we might have to reconsider when to attach/detach the server stdio. Differential Revision: https://phab.mercurial-scm.org/D8381

rust-chg: collect server flags from command arguments This is the reimplementation of testsensitiveflag() and setcmdserverargs() of chg.c. Differential Revision: https://phab.mercurial-scm.org/D8380

rust-chg: add interface to run "validate" request Differential Revision: https://phab.mercurial-scm.org/D8379

rust-chg: add helper to parse instructions sent from server This is well structured version of runinstructions() of chg.c. Differential Revision: https://phab.mercurial-scm.org/D8378

tests: move verification closer to setup in test-copies-chain-merge.t I found this test case really hard to read because it requires scrolling back and forth between the setup and the verification. This patch moves them closer together. There was also duplicated descriptions for each test case (duplicated between setup and verification). They had already become out of sync. I picked the description from the verification, partly so the differences would be highlighted in this patch. Differential Revision: https://phab.mercurial-scm.org/D8377

tests: collect all branch creation in one place in test-copies-chain-merge.t I found this test case really hard to read because it requires scrolling back and forth between the setup and the verification. The next patch will move the verification close to the merge commits they test. This patch prepares for that by moving all branch creation first so they are separate from the merge commits (because many merge commits reuse the same branches). Differential Revision: https://phab.mercurial-scm.org/D8376

hgk: remove a "b" used on a kwargs expansion, the keys are strs Differential Revision: https://phab.mercurial-scm.org/D8385

rust-chg: send client-side environment variables to server This is also needed to run config validation. The "validate" request and its response handling will be implemented in the next batch. Differential Revision: https://phab.mercurial-scm.org/D8364

rust-chg: add helper to pack environment variables On my machine, "printenv | wc -c" says 3422. That's the only reason why the initial buffer capacity is set to 4kB. Differential Revision: https://phab.mercurial-scm.org/D8363

rust-chg: move set_current_dir() to Locator This is necessary to run config validation in proper environment. Differential Revision: https://phab.mercurial-scm.org/D8362

rust-chg: abort if server doesn't have required capabilities It's checked at Locator::connect() since we'll have to do more stuff in this function. Differential Revision: https://phab.mercurial-scm.org/D8361

rust-chg: spawn server process if not running This is the minimal reimplementation of gethgcmd(), execcmdserver(), retryconnectcmdserver(), and connectcmdserver() in chg.c. No config validation is implemented yet. And some Py3 workarounds would be missing as this is the code I wrote in 2018. Differential Revision: https://phab.mercurial-scm.org/D8360

rust-chg: update dependencies Differential Revision: https://phab.mercurial-scm.org/D8359

rust-chg: depend on released version of tokio-process Still it is futures-0.1 based. Differential Revision: https://phab.mercurial-scm.org/D8358

rust-chg: add brief comment about initial capacity of temp_sock_path() I don't know if it can be expressed as a compile-time constant, so it's a comment for now. About this series: This is quite old patches for rust-chg. I heard from Octobus people that there's a plan to do an experiment on merging hgcli + chg + some Rust?, so I decided to respin the rust-chg series. Maybe we'll rewrite the core to leverage the recent async/await functionality, but I want to first make my old patches in so the rust-chg can be a drop-in replacement for the chg of C. Compiler warnings will be removed later, and the codebase will be upgraded to the 2018 edition later. Differential Revision: https://phab.mercurial-scm.org/D8357

discovery: avoid wrongly saying there are nothing to pull We can get in a situation where a revision passed through `hg pull --rev REV` are available on the server, but not a descendant of the advertised server heads. For example the server could lying be during heads advertisement, to hide some pull request. Or obsolete/hidden content could be explicitly pulled. So in this case the lookup associated to `REV` returned successfully, but the normal discovery will find all advertised heads already known locally. This flip a special boolean `anyinc` that will prevent any fetch attempt, preventing `REV` to be pulled over. We add three line of code to detect this case and make sure a pull actually happens. My main target is to make some third party extensions happy (I expect the associated test to move upstream with the extension). However this fix already make some of the `infinitepush` test happier.

rebase: don't create merge when continuing rebase interrupted by old hg This fixes the bug described and demonstrated in the previous commit. It does so by practically undoing 8082a77cc3a2 (rebase: remove some redundant setting of dirstate parents, 2020-01-10). Differential Revision: https://phab.mercurial-scm.org/D8356

tests: demonstrate how continuing rebase after upgrade can result in merge If the user starts a rebase with an hg version before 9c9cfecd4600 (rebase: don't use rebased node as dirstate p2 (BC), 2020-01-10) and then runs into conflicts, they will be dropped out to the shell with the rebased node set as the dirstate's second parent. If they then upgrade to a later hg version, it will respect the dirstate's parents and will create a merge commit even if the user was rebasing a non-merge commit. Differential Revision: https://phab.mercurial-scm.org/D8355

manifest: also declare treemanifest as implementing imanifestdict It always has been intended to be, but it had a mismatched flags() method until the previous change. Differential Revision: https://phab.mercurial-scm.org/D8366

manifest: introduce new exception to signal unavailability of fastdelta() I've spent some time reflecting on this, and I think this is the best we can do in this API contract for now. This opens the door to adding treemanifest's implementation to the list of implementers of imanifestdict. Differential Revision: https://phab.mercurial-scm.org/D8365

copies-tests: remove spurious `]` in the template Fixing this typo happily impact all the tests output. Differential Revision: https://phab.mercurial-scm.org/D8258

merge with stable

debugcommands: fix typo in debuguigetpass

dagop: fix subsetparentswalker to set p1/p2 chains at merge revision The previous implementation was wrong because the '1'/'2' key would be appended at a fork revision. Since we traverse the graph from heads, a merge revision is actually a branching point, where the sort key must be generated.

dagop: simplify dict/set reuse condition in subsetparentswalker Prepares for fixing the calculation of p1/p2 sort keys. With this change, there will be one more copying on merge&fork case. I think the copying cost is negligible since we'll have to update each item in the dict on merge/fork.

extensions: refactor function for obtaining disabled extension help The way this worked before was hgext.__index__ was consulted. This file appears to only be present on some Windows distributions. This file contains a dict mapping extension name to its summary line, not its full docstring. The problem with this is that code in the help system was calling this function to resolve help text. If hgext.__index__ was present, only the summary line would be displayed. If not, the full extension help would be printed. This commit changes the function to not use hgext.__index__ such that it always returns the full extension help text. As a result of this change, test-extension.t and test-qrecord.t now pass when run from environments that have an hgext.__index__. Differential Revision: https://phab.mercurial-scm.org/D8344

tests: perform grep manually in test-doctest.py This test has been failing on Windows since 0af56d3ee24c introduced the `hg files` invocation. Specifically, Windows seems to be choking on special characters in the fileset pattern. I believe at least \n and > were causing issues. I attempted various incantations to make the Windows command line parser accept the fileset but couldn't get anything working. I declared bankruptcy and just reimplemented the grepping code in Python. After this change, the test now passes on Windows again. Differential Revision: https://phab.mercurial-scm.org/D8343

tests: prevent printing \r to stdout Like we've done in other recent commits, we need to change sys.stdout on Python 3 to not use os.linesep so output is consistent on Python 3 on Windows. With this change, test-notify.t now passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8342

tests: force \n newlines when writing to sys.stdout Without this, Python 3 on Windows inserts some \r that aren't present in the input, causing test-http-bad-server.t to fail. After this change, the test passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8341

dispatch: force \n for newlines on sys.std* streams (BC) The sys.std* streams behave differently on Python 3. On Python 3, these streams are an io.TextIOWrapper that wraps a binary buffer stored on a .buffer attribute. These TextIOWrapper instances normalize \n to os.linesep by default. On Windows, this means that \n is normalized to \r\n. So functions like print() which have an implicit end='\n' will actually emit \r\n for line endings. While most parts of Mercurial go through the ui.write() layer to print output, some code - notably in extensions and hooks - can use print(). If this code was using print() or otherwise writing to sys.std* on Windows, Mercurial would emit \r\n. In reality, pretty much everything on Windows reacts to \n just fine. Mercurial itself doesn't emit \r\n when going through the ui layer. Changing the sys.std* streams to not normalize line endings sounds like a scary change. But I think it is safe. It also makes Mercurial on Python 3 behave similarly to Python 2, which did not perform \r\n normalization in print() by default. .. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3 Differential Revision: https://phab.mercurial-scm.org/D8339

hook: move stdio redirection to context manager The old code was checking stdio redirection in a loop. This didn't make sense. The pattern is better expressed as a context manager IMO, so this commit refactors it to be one. Differential Revision: https://phab.mercurial-scm.org/D8338

pycompat: change argv conversion semantics Use of os.fsencode() to convert Python's sys.argv back to bytes was not correct because it isn't the logically inverse operation from what CPython was doing under the hood. This commit changes the logic for doing the str -> bytes conversion. This required a separate implementation for POSIX and Windows. The Windows behavior is arguably not ideal. The previous behavior on Windows was leading to failing tests, such as test-http-branchmap.t, which defines a utf-8 branch name via a command argument. Previously, Mercurial's argument parser looked to be receiving wchar_t bytes in some cases. After this commit, behavior on Windows is compatible with Python 2, where CPython did not implement `int wmain()` and Windows was performing a Unicode to ANSI conversion on the wchar_t native command line. Arguably better behavior on Windows would be for Mercurial to preserve the original Unicode sequence coming from Python and to wrap this in a bytes-like type so we can round trip safely. But, this would be new, backwards incompatible behavior. My goal for this commit was to converge Mercurial behavior on Python 3 on Windows to fix busted tests. And I believe I was successful, as this commit fixes 9 tests on my Windows machine and 14 tests in the AWS CI environment! Differential Revision: https://phab.mercurial-scm.org/D8337

Added signature for changeset 8fca7e8449a8

Added tag 5.3.2 for changeset 8fca7e8449a8

phabricator: extract logic to print the status when posting a commit This will make it easier to list each commit when folding. That makes the output less confusing because it matches the output of `--confirm` and the revisions listed on the command line. Differential Revision: https://phab.mercurial-scm.org/D8313

phabricator: extract the logic to amend diff properties to a function This will be needed on a separate code path when dealing with folding revisions. And since we know that will involve adding multiple local commmits to the diff properties instead of just one, restructure the logic slightly to allow it. Differential Revision: https://phab.mercurial-scm.org/D8312

phabricator: teach `getoldnodedrevmap()` to handle folded reviews The tricky part here is reasoning through all of the possible predecessor scenarios. In the typical case of submitting a folded range and then resubmitting it (also folded), filtering the list of commits for the diff stored on Phabricator through the local predecessor list for each single node will result in the typical 1:1 mapping to the old node. There are edge cases like using `hg fold` within the range prior to resubmitting, that will result in mapping to multiple old nodes. In that case, the first direct predecessor is needed for the base of the diff, and the last direct predecessor is needed for the head of the diff in order to make sure that the entire range is included in the diff content. And none of this matters for commits in the middle of the range, as they are never used. Fortunately the only crucial thing here is the `drev` number for each node. For these complicated cases where there are multiple old nodes, simply ignore them all. This will cause `createdifferentialrevision()` to generate a new diff (within the same Differential), and avoids complicating the code. Differential Revision: https://phab.mercurial-scm.org/D8311

phabricator: teach createdifferentialrevision() to allow a folded commit range No visible changes here, until an option to enable it is added to `phabsend`. Differential Revision: https://phab.mercurial-scm.org/D8310

phabricator: combine commit messages into the review when folding commits No visible changes here, until an option to enable it is added to `phabsend`. This combines the Differential fields like Arcanist does, rather than simply concatenating the text blocks. Aside from populating everything properly in the web interface, Phabricator fails the review create/update if repeated fields are seen as would happen with simple concatenation. On the flip side, now that the Summary and Test Plan fields can contain data from multiple commits, we can't just join these fields together to determine if an amend is needed. If that were to happen, every single commit in the folded range would get amended with the combined commit message, which seems clearly wrong. Aside from making a minor assumption about the content of the Differential Revision field (it seems they allow some minor variances with spacing), this means that for folded reviews, you can't post it, go to the web page add a missing Test Plan, and then get it added to the commit message by re-posting it. I don't think that's a big deal. Differential Revision: https://phab.mercurial-scm.org/D8309

phabricator: record all local commits used to create a Differential revision Arcanist records all of the commits that it squashes into a single review, and that info will be helpful when adding similar functionality. This info is used when submitting an updated review, so that the extension can recalculate the old diff and see if a new one is necessary, or if it is just a property update. It also shows on the `commits` tab in the `Revision Contents` section. When submitting in the usual 1:1 commit to review mode, the wire protocol is unchanged. The content of `hg:meta` is a bit odd, but such is the problem when folding several commits. The choice for the parent node is obvious, but the `node` value uses the tip commit because that seems more natural, and is used elsewhere to look up the previous diff when updating. The rest of the attributes follow from there. Differential Revision: https://phab.mercurial-scm.org/D8308

tests: use `f --hexdump` to print file content The inline print.py in this test wasn't fully compatible with Python 3 because it was reading from sys.stdin, which already normalized line endings since it operates in the realm of str on Python 3. To do this correctly, we'd need to read from sys.stdin.buffer on Python 3. This would entail conditional code. I felt this was too much effort. So I just replaced the custom script with `f`, which already knows how to do the right thing. test-mactext.t now passes on Python 3 on Windows. Differential Revision: https://phab.mercurial-scm.org/D8336

url: pass str to pathname2url This is needed to appease Python 3. This fixes test-extdata.t and test-url-download.t on Python 3 on Windows. Differential Revision: https://phab.mercurial-scm.org/D8335

tests: pass str to matchoutput() It accepts a str, not bytes. This fixes a failure in test-hghave.t on Windows. Why it wasn't failing on Linux, I don't know. I suspect the Windows process code in Python doesn't accept bytes and the POSIX code does? Differential Revision: https://phab.mercurial-scm.org/D8334

hgcli: customize for Mercurial Now that we have a shiny new PyOxidizer-based hgcli project, let's customize it for Mercurial! This commit replaces the auto-generated pyoxidizer.bzl with one that installs Mercurial from the local source repository. A README.md with build instructions has been added. The Cargo.toml file has been updated to reflect the proper license and reference the added README.md. In my Linux environment, running the test suite yields 27 failures. It's worth noting the run time of the test harness on Linux on my Ryzen 3950X: before: 378s wall; 9982s user; 1195s sys after: 353s wall; 8996s user; 958s sys % orig: 93.4 wall; 90.1 user; 80.2 sys While I haven't measured explicitly, I suspect the performance win is due to in-memory resource loading (which is known to be faster than Python's filesystem importer). Differential Revision: https://phab.mercurial-scm.org/D8351

hgcli: add stub PyOxidizer project Using commit c772a1379c3026314eda1c8ea244b86c0658951d of PyOxidizer, I ran `pyoxidizer init-rust-project hgcli` to create a stub Rust project. The only modifications I made from what that command produced are: * Update location of pyembed crate to PyOxidizer's Git repository. * Removed some trailing whitespace from pyoxidizer.bzl * Added auto-generated Cargo.lock file Subsequent commits will modify the stub project to Mercurial's needs. Differential Revision: https://phab.mercurial-scm.org/D8350

hgcli: remove legacy project This code is a logical precursor to PyOxidizer. It is now defunct. Differential Revision: https://phab.mercurial-scm.org/D8349