merge with stable

match: teach diffmatcher.visitdir() to return 'all' if possible This patch teaches differencematcher.visitdir() to return 'all' when m1.visitdir() returns 'all' and m2 does not matches. Before this patch, from a differencematcher.visitdir(), we always returned either True or False. We never returned 'all' even when we can. This causes problem when m1 and m2 of a differencematcher are themselves differencematcher. In that case, we try to check: `if self._m2_.visitdir(dir) == 'all'` which will never be 'all' even though it can be. This leads to iterating over a lot of sub-directory manifest, even though we don't want to while extending a narrow clone. I am yet to measure the impact of this but calculating manifest was taking ~50-60 seconds, so this should definitely save some of time there. Differential Revision: https://phab.mercurial-scm.org/D5814

py3: use integer division instead of `int(...)` call Changeset 38a82e0333c9 and 7f853549823b introduced explicit conversion to integer to work around the division behavior change from python2 to python3. Using the integer division operator is a simpler and clearer way to achieve this.

py3: add some b'' prefixes in test-notify.t # skip-blame as just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5811

py3: use pycompat.long in hgext/convert/monotone.py Differential Revision: https://phab.mercurial-scm.org/D5810

py3: add three new passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D5809

py3: do a fsdecode(), fsencode() dance in posix.py We have to use `fsdecode()` instead of sysstr() because they are abusing `PyUnicode_EncodeFSDefault()` to get back bytes. Differential Revision: https://phab.mercurial-scm.org/D5808

revert: add prompt before undeleting a file in -i (issue6008) This adds a prompt that asks whether or not a removed file is to be undeleted in `hg revert -i`. Differential Revision: https://phab.mercurial-scm.org/D5803

py3: add alternate output on Python 3 This is basically the same deal as D5806. Python 3's exception printing output is different. We had to tweak the regular expression to match appropriately, hence the added line of output. Differential Revision: https://phab.mercurial-scm.org/D5807

py3: add Python 3 output for test-flagprocessor.t The exception is being generated from ui.traceback(), which simply calls traceback.format_*() to format exceptions. Since the output from the standard library has changed, there is seemingly not much we can do about it. So this commit adds conditional output depending on the Python version. I'm not thrilled about b'' appearing in user-facing output. Can we do something better here? Differential Revision: https://phab.mercurial-scm.org/D5806

zeroconf: Python 3 porting of vendored library A quick glance through this module reveals that most of it "just works" on Python 3 with the source transformer active. There are a few places where we need to ensure we're using str. Differential Revision: https://phab.mercurial-scm.org/D5804

relnotes: more improvements

py3: don't use universal_newlines in svnsubrepo On Python 3, it means text=True without any specific encoding, which is pretty bad. Instead, use util.fromnativeeol() to translate CR+LF to LF without encoding conversion.

py3: remove unneeded fsencode() from gitsubrepo Here p.stdout isn't a TextIO. read() must return bytes.

debugcommands: add a debugpathcopies command I've been working on storing copy metadata in the changelog instead of the filelog and this has been useful for debugging. Do we usually add tests for these? Differential Revision: https://phab.mercurial-scm.org/D5791

py3: record several more passes from the buildbot ratchet Differential Revision: https://phab.mercurial-scm.org/D5799

grep: respect ui.relative-paths Differential Revision: https://phab.mercurial-scm.org/D5777

grep: move writing of path outside of column loop This will make the next patch simpler. Differential Revision: https://phab.mercurial-scm.org/D5776

resolve: respect ui.relative-paths Differential Revision: https://phab.mercurial-scm.org/D5748

merge: respect ui.relative-paths We print file paths in a lot of places in this code and I've probably missed a few places. We can fix them as we discover them (I'm also happy to fix anything reviewers notice, of course). Differential Revision: https://phab.mercurial-scm.org/D5747

histedit: add templating support to histedit's rule file generation This will allow users to customize the display of the rule list for the free-form segment that we don't interpret. We've had users want to add things like bookmark names or similar to the rule list as a convenience, which seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D5742

py3: account for demand import difference between Python versions Our lazy importer for Python 3 will validate that modules are loadable before returning a stub module object. This is different from Python 2, which will always return a stub module object. While we could change behavior of the Python 3 demand importer, that seems like a problem for another day. This commit teaches test-extension.t about that difference in behavior. Differential Revision: https://phab.mercurial-scm.org/D5798

tests: use unimported modules in test-demandimport.py contextlib isn't a good module to test because it is likely already imported by code above. Let's use modules that shouldn't have been imported. And let's verify that with asserts. Differential Revision: https://phab.mercurial-scm.org/D5797

py3: conditionalize test-demandimport.py for Python 3 The Python 3 lazy importer uses the LazyLoader that is part of importlib. On Python 3 and later, LazyLoader is implemented using a custom module type that defines a __getattribute__ which triggers module loading. Furthermore, there are additional differences as well. For example, it appears that Python 3 will return an existing sys.modules entry instead of constructing a new module object. This commit adds additional test coverage for lazy importing behavior to cover the differences between Python 2 and 3. This reveals that the test and some lazy import functionality is kinda busted. For example, the test assumes "contextlib" will be lazy. But in reality an import before it has already imported contextlib! There's definitely room to improve the behavior of the demand importer code, both for Python 2 and 3. But at least the test passes on Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D5796

py3: replace print() with assert in test-demandimport.py Behavior of demand imports behaves differently between Python 2 and 3. .out files do not support conditional output the way that .t files do. In order to make this test work on Python 3, we'll need to make the test itself conditional. The first step of this is to port the test to not use a .out file to compare output. Unfortunately, we can't easily use the unittest framework for defining this test because putting import statements in functions changes the behavior of the demand importer (at least on Python 2). So, we effectively replace a bunch of print() with assert statements. This makes the test a bit annoying to debug, as the test will stop at first assertion failure. But we don't exactly have a good alternative. Differential Revision: https://phab.mercurial-scm.org/D5795

py3: pass str into grp.getgrnam grp.getgrnam expects str on Python 3. This fixes test-acl.t on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5794

montone: fix addition to list by using .append() instead of '+' Differential Revision: https://phab.mercurial-scm.org/D5739

tests: give up and make setsockopt() calls optional in the output I can't figure out what causes these calls to happen or not, and I weary of dealing with them. End the madness for now by marking them all as optional lines of output. Differential Revision: https://phab.mercurial-scm.org/D5762

py3: fix test-remotefilelog-repack.t This is uglier, but more obviously correct in my opinion. I guess Python 3 doesn't hang on to the exception as long, which seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D5781

tests: alter email `From` line to a value that's consistently parsed Python2: >>> email.header.decode_header('=?UTF-8?q?Rapha=C3=ABl=20Hertzog?= <hertzog@debian.org>') [('Rapha\xc3\xabl Hertzog', 'utf-8'), ('<hertzog@debian.org>', None)] Python3: >>> email.header.decode_header('=?UTF-8?q?Rapha=C3=ABl=20Hertzog?= <hertzog@debian.org>') [(b'Rapha\xc3\xabl Hertzog', 'utf-8'), (b' <hertzog@debian.org>', None)] So alter the input to an input that parses to the same result consistently. After skimming the relevant RFC (1342), I'm not sure if what we had was valid, or how I could modify it to be more consistent. Differential Revision: https://phab.mercurial-scm.org/D5769