tests: perform grep manually in test-doctest.py This test has been failing on Windows since 0af56d3ee24c introduced the `hg files` invocation. Specifically, Windows seems to be choking on special characters in the fileset pattern. I believe at least \n and > were causing issues. I attempted various incantations to make the Windows command line parser accept the fileset but couldn't get anything working. I declared bankruptcy and just reimplemented the grepping code in Python. After this change, the test now passes on Windows again. Differential Revision: https://phab.mercurial-scm.org/D8343

tests: prevent printing \r to stdout Like we've done in other recent commits, we need to change sys.stdout on Python 3 to not use os.linesep so output is consistent on Python 3 on Windows. With this change, test-notify.t now passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8342

tests: force \n newlines when writing to sys.stdout Without this, Python 3 on Windows inserts some \r that aren't present in the input, causing test-http-bad-server.t to fail. After this change, the test passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8341

dispatch: force \n for newlines on sys.std* streams (BC) The sys.std* streams behave differently on Python 3. On Python 3, these streams are an io.TextIOWrapper that wraps a binary buffer stored on a .buffer attribute. These TextIOWrapper instances normalize \n to os.linesep by default. On Windows, this means that \n is normalized to \r\n. So functions like print() which have an implicit end='\n' will actually emit \r\n for line endings. While most parts of Mercurial go through the ui.write() layer to print output, some code - notably in extensions and hooks - can use print(). If this code was using print() or otherwise writing to sys.std* on Windows, Mercurial would emit \r\n. In reality, pretty much everything on Windows reacts to \n just fine. Mercurial itself doesn't emit \r\n when going through the ui layer. Changing the sys.std* streams to not normalize line endings sounds like a scary change. But I think it is safe. It also makes Mercurial on Python 3 behave similarly to Python 2, which did not perform \r\n normalization in print() by default. .. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3 Differential Revision: https://phab.mercurial-scm.org/D8339

hook: move stdio redirection to context manager The old code was checking stdio redirection in a loop. This didn't make sense. The pattern is better expressed as a context manager IMO, so this commit refactors it to be one. Differential Revision: https://phab.mercurial-scm.org/D8338

pycompat: change argv conversion semantics Use of os.fsencode() to convert Python's sys.argv back to bytes was not correct because it isn't the logically inverse operation from what CPython was doing under the hood. This commit changes the logic for doing the str -> bytes conversion. This required a separate implementation for POSIX and Windows. The Windows behavior is arguably not ideal. The previous behavior on Windows was leading to failing tests, such as test-http-branchmap.t, which defines a utf-8 branch name via a command argument. Previously, Mercurial's argument parser looked to be receiving wchar_t bytes in some cases. After this commit, behavior on Windows is compatible with Python 2, where CPython did not implement `int wmain()` and Windows was performing a Unicode to ANSI conversion on the wchar_t native command line. Arguably better behavior on Windows would be for Mercurial to preserve the original Unicode sequence coming from Python and to wrap this in a bytes-like type so we can round trip safely. But, this would be new, backwards incompatible behavior. My goal for this commit was to converge Mercurial behavior on Python 3 on Windows to fix busted tests. And I believe I was successful, as this commit fixes 9 tests on my Windows machine and 14 tests in the AWS CI environment! Differential Revision: https://phab.mercurial-scm.org/D8337

Added signature for changeset 8fca7e8449a8

Added tag 5.3.2 for changeset 8fca7e8449a8

phabricator: extract logic to print the status when posting a commit This will make it easier to list each commit when folding. That makes the output less confusing because it matches the output of `--confirm` and the revisions listed on the command line. Differential Revision: https://phab.mercurial-scm.org/D8313

phabricator: extract the logic to amend diff properties to a function This will be needed on a separate code path when dealing with folding revisions. And since we know that will involve adding multiple local commmits to the diff properties instead of just one, restructure the logic slightly to allow it. Differential Revision: https://phab.mercurial-scm.org/D8312

phabricator: teach `getoldnodedrevmap()` to handle folded reviews The tricky part here is reasoning through all of the possible predecessor scenarios. In the typical case of submitting a folded range and then resubmitting it (also folded), filtering the list of commits for the diff stored on Phabricator through the local predecessor list for each single node will result in the typical 1:1 mapping to the old node. There are edge cases like using `hg fold` within the range prior to resubmitting, that will result in mapping to multiple old nodes. In that case, the first direct predecessor is needed for the base of the diff, and the last direct predecessor is needed for the head of the diff in order to make sure that the entire range is included in the diff content. And none of this matters for commits in the middle of the range, as they are never used. Fortunately the only crucial thing here is the `drev` number for each node. For these complicated cases where there are multiple old nodes, simply ignore them all. This will cause `createdifferentialrevision()` to generate a new diff (within the same Differential), and avoids complicating the code. Differential Revision: https://phab.mercurial-scm.org/D8311

phabricator: teach createdifferentialrevision() to allow a folded commit range No visible changes here, until an option to enable it is added to `phabsend`. Differential Revision: https://phab.mercurial-scm.org/D8310

phabricator: combine commit messages into the review when folding commits No visible changes here, until an option to enable it is added to `phabsend`. This combines the Differential fields like Arcanist does, rather than simply concatenating the text blocks. Aside from populating everything properly in the web interface, Phabricator fails the review create/update if repeated fields are seen as would happen with simple concatenation. On the flip side, now that the Summary and Test Plan fields can contain data from multiple commits, we can't just join these fields together to determine if an amend is needed. If that were to happen, every single commit in the folded range would get amended with the combined commit message, which seems clearly wrong. Aside from making a minor assumption about the content of the Differential Revision field (it seems they allow some minor variances with spacing), this means that for folded reviews, you can't post it, go to the web page add a missing Test Plan, and then get it added to the commit message by re-posting it. I don't think that's a big deal. Differential Revision: https://phab.mercurial-scm.org/D8309

phabricator: record all local commits used to create a Differential revision Arcanist records all of the commits that it squashes into a single review, and that info will be helpful when adding similar functionality. This info is used when submitting an updated review, so that the extension can recalculate the old diff and see if a new one is necessary, or if it is just a property update. It also shows on the `commits` tab in the `Revision Contents` section. When submitting in the usual 1:1 commit to review mode, the wire protocol is unchanged. The content of `hg:meta` is a bit odd, but such is the problem when folding several commits. The choice for the parent node is obvious, but the `node` value uses the tip commit because that seems more natural, and is used elsewhere to look up the previous diff when updating. The rest of the attributes follow from there. Differential Revision: https://phab.mercurial-scm.org/D8308

tests: use `f --hexdump` to print file content The inline print.py in this test wasn't fully compatible with Python 3 because it was reading from sys.stdin, which already normalized line endings since it operates in the realm of str on Python 3. To do this correctly, we'd need to read from sys.stdin.buffer on Python 3. This would entail conditional code. I felt this was too much effort. So I just replaced the custom script with `f`, which already knows how to do the right thing. test-mactext.t now passes on Python 3 on Windows. Differential Revision: https://phab.mercurial-scm.org/D8336