py3: port f to Python 3 This involved a lot of b'' literals, conversion from %s to %d, and using a hashing mechanism that returns bytes instead of str. Differential Revision: https://phab.mercurial-scm.org/D2299

py3: replace file() with open() in test-convert-git.t file() is not present in Python 3. It also makes sure we write bytes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2133

bundle: updates the help text for hg bundle (issue5744) Differential Revision: https://phab.mercurial-scm.org/D2278

py3: get bytes-repr of network errors portably This resolves a lot of weird issues in Python 3 around error strings. Differential Revision: https://phab.mercurial-scm.org/D2295

hgweb: open server logs in binary mode This is consistent with when we're logging to stdout, so we don't have to do something annoyingly complicated in the logging infrastructure. Differential Revision: https://phab.mercurial-scm.org/D2294

tests: add some b prefixes in test-http-bundle1.t # skip-blame just some b prefixes Differential Revision: https://phab.mercurial-scm.org/D2293

hgweb: correctly bytes-ify status, not string-ify Differential Revision: https://phab.mercurial-scm.org/D2291

printenv: port to python3 Differential Revision: https://phab.mercurial-scm.org/D2290

py3: whitelist another eight passing tests Differential Revision: https://phab.mercurial-scm.org/D2281

remotenames: port partway to python3 by using collections.MutableMapping test-logexchange.t doesn't pass after this, but at least the remotenames extension can be imported. Differential Revision: https://phab.mercurial-scm.org/D2280

tests: avoid referring to pvec in demandimport test Nothing else currently uses pvec, so we may want to delete it (but Augie Fackler says he may want to use it). To enable deletion, this patch replaces it by the error module in the demandimport test (any module works). However, since the error module had already been loaded at this point in the test (via the util module), I moved it earlier in the test so it's still not loaded (although I'm not sure if that's even relevant to the test). Differential Revision: https://phab.mercurial-scm.org/D2287

help: use cmdutil.parsealiases() to resolve command name This seems slightly better than parsing '^command|name' string by using an ad-hoc pattern.

help: list deprecated and experimental extensions if --verbose This is common in the other help topics.

hgweb: translate Abort in help command to 404 error c3784e3c3e8d changed the exception type to be raised if unknowncmd=False.

templatekw: add {reporoot} keyword The "%b" in filename format string will be replaced with "{reporoot|basename}".

templatefilters: add dirname() filter This is different from stripdir() in that the last component is always removed.

templater: add function to help substituting patterns in template string This will be used to rewrite a filename pattern to a template string.

templater: extract function scanning template string This provides a tokenizer-level view of template fragments, and will be used to substitute patterns in outermost 'string' tokens.

tests: expand test coverage for updating phases Consolidating the tests demonstrated that there are behavior differences when pushing phases between bundle1 and bundle2. A reason for this is the behavior of legacy pushes where the client queries the state of phases and then conditionally updates phases after an "unbundle" is processed. This behavior is expected. The tests were incomplete because they only tested the case of a publishing repo. In this commit, we add a variant for a non-publishing repo. We still see some differences between the legacy and bundle2 exchanges. But they are less pronounced. The behavior of not firing a pushkey hook when phases are updated as part of changegroup application feels weird to me. I'm not sure if this is a feature or a bug. By the time the "pushkey" or "phases" bundle2 part is applied, the phases have already been moved on a publishing repository. We fire the "pushkey" hook regardless, even though it would be a no-op. This is the part that feels the most buggy.

tests: consolidate test-push-http.t and test-push-http-bundle1.t These tests were initially copies of each other. Now that we have #testcases support in .t tests, we can consolidate them. The changes to test-push-http.t reflect the differences between that file and test-push-http-bundle1.t. The variances in phases push behavior are the biggest differences. The test will be updated in a subsequent commit to make the differences more clear and to expand test coverage. For now, let's just port the differences verbatim to get the tests consolidated.

tests: port value-less unbundle capability test to test-push-http.t This test is present in test-push-http-bundle1.t. Let's add it to test-push-http.t to further unify the tests.

tests: add phase testing to test-push-http-bundle1.t test-push-http.t and test-push-http-bundle1.t were initially copies. Now that we have support for inline test variants, we can combine them. One of the variances between the tests is testing of phase moving. We add the missing code to test-push-http-bundle1.t.

cmdutil: narrow scope of KeyError in makefilename() It seemed too broad to catch a standard Python exception.

node: make bin() be a wrapper instead of just an alias This includes a full backout of 59affe7e and 30d0cb27. Per the review of the former, we'd rather adapt the API to behave like it used to (at least for now), and take a second run at it if it shows up in our performance numbers. I ran perfrevlogindex with and without this change and it didn't make a measurable difference, so maybe it's fine (despite my intuition to the contrary). Differential Revision: https://phab.mercurial-scm.org/D2279

tests: actually check that HGDEMANDIMPORT=disable disables demandimport At the point in the test where we were checking that the 'node' got eagerly loaded, it had already been loaded (via the pvec module), so our check wasn't doing anything (i.e. the test would pass even if you removed the line that set HGDEMANDIMPORT=disable). Let's move this test earlier so it tests what it was meant to test. Differential Revision: https://phab.mercurial-scm.org/D2286

debugcommands: print out the editor that was searched for (post shlexsplit) A user was in #mercurial and had the following settings in their hgrc: [ui] editor = C:\home\npp\notepad++.exe -multiInst -nosession After shlexsplit, the first argument was mangled into C:homenppnotepad++.exe, which was quite unlikely to exist. It took many back-and-forths to identify that adding " characters around the exe would fix the issue; we were thinking that it's because something was incorrectly *not* splitting and adding/moving the " characters fixed the split boundaries, but when testing afterward it appears that it's just mangled. I considered adding an informational if pycompat.iswindows and \ in the string about this issue, but was worried that might have too many false positives and did not do so at this time. Differential Revision: https://phab.mercurial-scm.org/D1808

merge with stable

date: fix parsing months Thanks nemo for discovering this on #mercurial IRC channel. Test Plan: Add a test. It fails before this patch: ``` + hg: parse error: invalid date: 'Feb 2018' + hg: parse error: invalid date: 'Apr 2018' + hg: parse error: invalid date: 'Jun 2018' + hg: parse error: invalid date: 'Sep 2018' + hg: parse error: invalid date: 'Nov 2018' ``` Differential Revision: https://phab.mercurial-scm.org/D2289

hgweb: show dates recorded in obsolescence markers Dates of operations that obsolete commits are also important enough to be shown, but maybe not as important as original commit dates, that and also for brevity is why they are wrapped in a <span> with only "age" class. Provided that JS is enabled, such elements only show age (e.g. "3 months ago"), and the exact date is visible in a tooltip. Commits dates, on the other hand, show both date and age (the latter in parenthesis). Example result for f1a0933ce59e from hg-committed: obsolete: pruned by Yuya Nishihara <yuya@tcha.org> 13 days ago

gitweb: make span.age CSS selector more specific This set of CSS properties is only used on /changelog page to make age elements float to the left and be fixed in size, but span.a is too broad of a selector, because we don't want to apply these properties to all <span> elements that have age class.

wireprotoserver: return to using iscmd() method This was teased out in part so remotefilelog could disable an old protocol method over http. It got dropped accidentally in the recent refactor, but the code was all still present so it's easy to support for now. I think once we land remotefilelog, we should probably inline this function. Differential Revision: https://phab.mercurial-scm.org/D2285

githelp: cast commands to bytes This is more compatible with Python 3. Differential Revision: https://phab.mercurial-scm.org/D2165

py3: use util.forcebytestr instead of str for converting errors to bytes Differential Revision: https://phab.mercurial-scm.org/D2269

py3: use pycompat.bytestr to convert str to bytes Differential Revision: https://phab.mercurial-scm.org/D2261

mpatch: allow clang-format oversight Differential Revision: https://phab.mercurial-scm.org/D2183

base85: allow clang-format oversight Differential Revision: https://phab.mercurial-scm.org/D2182

charencode: allow clang-format oversight Nice and easy. Differential Revision: https://phab.mercurial-scm.org/D2181

formatting: enforce system headers before local headers Differential Revision: https://phab.mercurial-scm.org/D2275

wireprotoserver: add version to HTTP protocol name (API) This matches what we did for the SSH protocol handler in ac33dc94e1d5. .. api:: HTTP protocol handlers now advertises its internal name as ``http-v1`` instead of ``http``. Differential Revision: https://phab.mercurial-scm.org/D2219

wireprotoserver: rename webproto to httpv1protocolhandler This matches our naming convention for the SSH server's protocol handler. Differential Revision: https://phab.mercurial-scm.org/D2218

wireproto: improve docstring for "hello" Differential Revision: https://phab.mercurial-scm.org/D2217

httppeer: remove httpspeer All it did was verify at construction time that Mercurial supports TLS. instance() is what's used to construct peer instances. So we can just inline this check into that function and do away with the type variant. Differential Revision: https://phab.mercurial-scm.org/D2216

httppeer: remove redundant code to fetch capabilities _fetchcaps() is called by httppeer.instance(), which is the only instantiator of httppeer. Since _fetchcaps() always sets self._caps and since 197d10e157ce removed the fallback for cases where the remote doesn't support capabilities, we can remove some dead code from httppeer.capabilities(). Differential Revision: https://phab.mercurial-scm.org/D2215

httppeer: change logic around argument handling The code to process arguments only makes sense if there are arguments. So change an "else" to "elif args", remove an "if" that isn't necessary, and add some docs for good measure. Differential Revision: https://phab.mercurial-scm.org/D2214

tests: test using both versions of SSH protocol Now that the version 2 of the SSH protocol is usable in core, we can start actively testing it more widely outside of low-level protocol tests. We add #testcases variants to a handful of tests so we exercise both version 1 and version 2 of the SSH protocol when testing. This will allow us to more easily find regressions and variances as protocol 2 is developed. It will also make it easier to continue testing with protocol version 1 once version 2 is enabled by default. There are a handful of tests using ssh:// that should also gain test variances. One - test-push-race.t - already has a #testcases. This would require combinatorial cases. I didn't want to go down that rabbit hole, so that test is unchanged. Thinking aloud, there is probably an opportunity to automatically run tests with multiple server/protocol implementations. Ideally any test that performed server interaction would run with all supported server implementations and protocols so we could find variances between servers and protocols. But this has been a long-standing issue with our test harness. I don't think it is an easily solved problem. But it would be nice... Differential Revision: https://phab.mercurial-scm.org/D2206

sshpeer: log remote capabilities after protocol upgrade This helps reduce variance with version 1 and will help prevent test output divergence as we start testing protocol version 2 more widely. Differential Revision: https://phab.mercurial-scm.org/D2205

wireprotoserver: handle SSH protocol version 2 upgrade requests This commit teaches the SSH server to recognize the "upgrade" request line that clients send when they wish to switch the channel to version 2 of the SSH protocol. Servers don't honor upgrade requests unless an experimental config option is set. Since the built-in server now supports upgrade requests, our test server to test the handshake has been deleted. Existing tests use the built-in server and their output doesn't change. The upgrade is handled in our state machine. The end result is a bit wonky, as the server transitions back to version 1 state immediately after upgrading. But this will change as soon as version 2 has an actual protocol that differs from version 1. Tests demonstrating that the new server is a bit more strict about the upgrade handshake have been added. Differential Revision: https://phab.mercurial-scm.org/D2204

wireprotoserver: move SSH server operation to a standalone function The server-side processing logic will soon get a bit more complicated in order to handle protocol switches. We will use a state machine to help make the transitions clearer. To prepare for this, we move SSH server operation into a standalone function. We structure it as a very simple state machine. It only has two states for now, with one state containing the bulk of the logic. But things will evolve shortly. Differential Revision: https://phab.mercurial-scm.org/D2203

py3: stringify integer with %d instead of bytes() The unbundle wire protocol command now returns a properly formatted reply in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2274

py3: add b'' to test-sshserver.py # skip-blame because adding b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D2273

py3: add b'' to config options in test extension # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D2272

manifest: add support for including directories outside narrowspec When using tree manifests and the client doesn't have a directory, we have two choices for what to do with "hg manifest" output: 1) ignore the directory, and 2) include the directory (not files within it). For "hg files", we decided to ignore the directories (and files) outside the narrowspec. If we choose to not include directories outside the narrowspec, then I think we should also make sure we don't include files outside the narrowspec. I also think we should add --outside-narrow flag (or other name). Thus, whichever way we go, I think we should have a way of displaying paths (files or directories) outside the narrowspec. For that we'll need to handle the 't' flag that narrowhg uses, and that's what this patch adds support for. Differential Revision: https://phab.mercurial-scm.org/D2235

narrow: restrict manifest iteration by using manifest.walk(matcher) This is only for root nodes, so it shouldn't really matter (they're rarely huge), but seems cleaner. Differential Revision: https://phab.mercurial-scm.org/D2234

narrow: only diff manifest part within narrowspec when generating changegroup Since 959ebff3505a (manifest: add match argument to diff and filesnotin, 2017-03-07), we have a more efficient way of diffing manifests while applying a matcher. Let's use that while generating narrowed changegroups, so we avoid diffing parts of the manifest that don't match the narrowspec. Differential Revision: https://phab.mercurial-scm.org/D2233

cmdutil: build "%m" (desc|firstline) in makefilename()

cmdutil: rewrite makefilename() to use ctx methods