hgdemandimport: disable on Python 3.5 The demand importer functionality isn't working at all on Python 3.5. I'm not sure what's wrong. Since it isn't working, let's disable it completely. ``` $ HGRCPATH= hyperfine -w 1 -r 50 -- "~/.pyenv/versions/3.5.9/bin/python ./hg version" \ "HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version" Benchmark #1: ~/.pyenv/versions/3.5.9/bin/python ./hg version Time (mean ± σ): 163.7 ms ± 2.2 ms [User: 148.5 ms, System: 15.7 ms] Range (min … max): 161.0 ms … 170.2 ms 50 runs Benchmark #2: HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version Time (mean ± σ): 164.3 ms ± 1.4 ms [User: 148.2 ms, System: 16.6 ms] Range (min … max): 161.4 ms … 169.8 ms 50 runs ``` Differential Revision: https://phab.mercurial-scm.org/D7953

py3: suppress unraisable exceptions in test-worker.t Python 3.8 calls sys.unraisablehook when an unraisable exception is encountered. The default behavior is to print a warning. test-worker.t was triggering this hook due to a race between a newly forked process exiting and that process's _os.register_at_fork handlers running. I was seeing the stdlib's random module in the stack re-seeding itself. Although there could be other after-fork handlers in the mix. This commit defines sys.unraisablehook to effectively no-op. This suppresses the warning and makes test output on Python 3.8 consistent with prior versions. test-worker.t now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7949

rust: add a README In particular to explain how to build any of the rust. It's neither obvious, nor easy to find out, nor easy to determine if you did it right without some documentation. Differential Revision: https://phab.mercurial-scm.org/D7952

rust: move hgcli's README out of the way My understanding is that it's not meant to be used in the current form. Differential Revision: https://phab.mercurial-scm.org/D7951

verify: avoid spurious integrity warnings in verbose mode (issue6172) The issue seems to revolve around renames in filtered commits, and only occurred in verbose mode. The problem occurs in the `# check renames` stage, around line 577. Without using the unfiltered repo, this test would have printed: $ hg verify -v repository uses revlog format 1 checking changesets checking manifests crosschecking files in changesets and manifests checking files foo@25: checking rename of 71ec0570c325: filtered revision '25' foobar@26: checking rename of 1b549296015b: filtered revision '26' checked 28 changesets with 16 changes to 11 files 2 integrity errors encountered! (first damaged changeset appears to be 25) [1] Differential Revision: https://phab.mercurial-scm.org/D7950

py3: glob over exception in test-check-py3-compat.t Python 3.6+ raise ModuleNotFoundError and older versions raise ImportError. Glob over the exception differences. For whatever reason, we were already doing this for one failure. But not all occurrences of ModuleNotFoundError were changed. Who knows. This test should now pass on all Python versions (although I didn't check Windows). Differential Revision: https://phab.mercurial-scm.org/D7939

py3: string normalization and I/O tweaks in test-lfs.t The print was inserting b'' on Python 3. In addition, since we weren't writing to the ui instance (which isn't readily available in this function), output order could get mixed up. We add some pycompat casts and a stdout flush to make the test happy on all Python versions. Differential Revision: https://phab.mercurial-scm.org/D7938

help: minor copy editing to the `config.format` section Differential Revision: https://phab.mercurial-scm.org/D7936

changectx: mark parent of changesets as non filtered If a node is not filtered, its parents cannot be filtered. Differential Revision: https://phab.mercurial-scm.org/D7502

changectx: use unfiltered changelog to walk ancestors in annotate Since we are only walking ancestors, it is safe to use an unfiltered repository. (Because if the original rev is not filtered, none of its ancestors will be). Differential Revision: https://phab.mercurial-scm.org/D7501

localrepo: also fast past the parents of working copies parents There are descent odds that they will be needed too. So we also cache and fastpath them. Differential Revision: https://phab.mercurial-scm.org/D7498

localrepo: recognize trivial request for '.' Same logic as for `null`, this is a command request and skipping the revset logic can avoid triggering the changelog filtering logic. Differential Revision: https://phab.mercurial-scm.org/D7495

localrepo: fastpath access to "." "." is just an alias for `p1(wdir())`, let us handle it that way. Differential Revision: https://phab.mercurial-scm.org/D7494

localrepo: also fastpath access to working copy parents when possible If the filter level guarantee that the working copy parents will be visible, we allow fast path access to them. With this change multiple commands can now run without triggering filtering. After using the quick access mechanism introduced, the whole series results in pretty good performance gain: ``` All benchmarks: before after ratio [8e095512] [36b2f659] - 711±0.8ms 60.7±0.2ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 712±0.8ms 61.6±0.2ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 690±1ms 93.5±0.3ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 688±1ms 93.8±0.3ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 714±1ms 60.7±0.8ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 713±1ms 60.9±0.3ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±1ms 93.7±0.2ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 687±2ms 92.8±0.2ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 799±2ms 98.1±0.6ms 0.12 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 800±0.8ms 100.0±0.4ms 0.12 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 711±0.9ms 111±0.2ms 0.16 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 711±1ms 112±0.3ms 0.16 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 760±1ms 59.8±0.1ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 763±2ms 62.2±0.3ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±1ms 93.1±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 688±1ms 94.3±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 763±1ms 60.1±0.2ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 763±1ms 62.1±0.4ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±0.8ms 93.2±0.2ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 687±0.9ms 94.1±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] ``` Differential Revision: https://phab.mercurial-scm.org/D7492

examples: refer to nightly rustfmt in Windows-compatible way Thanks to Jun Wu for the tip. I found that the new form also gave better error messages when the nightly rustfmt wasn't installed (it told me which command to run instead of just saying "error: not a file: <some path>"). Differential Revision: https://phab.mercurial-scm.org/D7911

convert: refactor authormap into separate function for outside use Differential Revision: https://phab.mercurial-scm.org/D7732

remotefilelog: fix opening validatecachelog in text mode

cext: fix compiler warning about sign changing line.len is a Py_ssize_t, and we're casing to size_t (unsigned). On my compiler, this causes a warning to be emitted: ``` mercurial/cext/manifest.c: In function 'pathlen': mercurial/cext/manifest.c:48:44: warning: operand of ?: changes signedness from 'Py_ssize_t' {aka 'long int'} to 'long unsigned int' due to unsignedness of other operand [-Wsign-compare] return (end) ? (size_t)(end - l->start) : l->len; ^~~~~~ ``` Differential Revision: https://phab.mercurial-scm.org/D7913

sha1dc: avoid including the nonexistent stdint.h with Visual Studio 2008 Differential Revision: https://phab.mercurial-scm.org/D7903

py3: fix curses chunkselector fallback (when diffs are too large) on py3 Previously we showed the message using Exception.message, which is removed in py3. Since crecordmod.fallbackerror inherits from error.Abort, we can just use `b'%s' % exception` to print the message. This does not print the hint, but that's fine - we don't set one. We inherit from error.Abort so that if a codepath doesn't handle fallback specially, it exits to the terminal with a sane message instead of an unknown exception error. Differential Revision: https://phab.mercurial-scm.org/D7912

transaction: allow finalizer to add finalizer It will make some code (persistent nodemap related) simpler to write, because higher level code can blindly queue finalization without thinking too hard about the context. Differential Revision: https://phab.mercurial-scm.org/D7833

tests: stabilize test-subrepo-svn.t on Windows This partially reverts 991e4404e910, because the URL form of `C:\...` gets escaped to `C%3A/...`, which breaks the substitution of $TESTTMP. The forget command on 'notafile*' errored out with: notafile*: The filename, directory name, or volume label syntax is incorrect which I think is because '*' isn't a legal character in a file name (though I can't trigger this directly from MSYS or cmd.exe for some reason). Differential Revision: https://phab.mercurial-scm.org/D7816

rebase: fix bug where `--collapse` would apply diff on missing file Even though the file was missing, the rebase would succeed. Differential Revision: https://phab.mercurial-scm.org/D7897

rebase: extract a variable for a repeated `repo[p1]` I'll add another use site in the next patch. Differential Revision: https://phab.mercurial-scm.org/D7896

graftcopies: document why the function is useful at all Despite having spent a significant amount on time on the copy-tracing code, I thought `graftcopies()` (formerly known as `duplicatecopies()`) was needed to duplicate copies after calling `merge.update()` to do a merge (as `merge.graft()` does), but it's actually usually not needed; `merge.update()` takes care of most copies. This patch documents what the function is for. Differential Revision: https://phab.mercurial-scm.org/D7861

graftcopies: remove `skip` and `repo` arguments The `skip` argument was added in 2ba6c9b4e0eb (rebase: fix bug that caused transitive copy records to disappear (issue4192), 2014-06-07) in order to fix https://bz.mercurial-scm.org/show_bug.cgi?id=4192. I ran tests at that commit without the `skiprev` argument and the only difference I noticed was that `test-rebase-collapse.t` failed differently, in the call that is now on line 501. Without the `skiprev` argument, that call would end up creating another commit because it tried to record an invalid copy. With the previous patch in this series, such invalid copies are no longer recorded, so it seems we don't need the `skip` argument anymore. I also removed the `repo` argument since that also becomes unused with the removal of the `skip` argument. Differential Revision: https://phab.mercurial-scm.org/D7860

graftcopies: use _filter() for filtering out invalid copies `graftcopies()` (formerly called `duplicatecopies()`) checked that the copy destination existed in the working copy, but it didn't check that copy source existed in the parent of the working copy. In `test-graft.t` we can see that as warnings about not finding ancestors of the copied files, and also empty commits getting created. This patch uses the existing `_filter()` function for filtering out invalid copies. In addition to the aforementioned types, that also includes copies where source and destination is the same. Differential Revision: https://phab.mercurial-scm.org/D7859

copies: replace duplicatecopies() by function that takes contexts The callers mostly have context objects, so let's avoid looking up the same context objects inside `duplicatecopies()`. I also renamed the function to `graftcopies()` since I think that better matches its purpose. I did it in the same commit so it's easier for extensions to switch between the functions. Differential Revision: https://phab.mercurial-scm.org/D7858

graft: extract repo[None] to a variable I plan to allow the caller pass in an overlayworkingctx, so this prepares for that. Differential Revision: https://phab.mercurial-scm.org/D7857

rust-core: fix typo in comment Differential Revision: https://phab.mercurial-scm.org/D7895

sha1dc: use buffer protocol when parsing arguments Without this, functions won't accept bytearray, memoryview, or other types that can be exposed as bytes to the C API. The most resilient way to obtain a bytes-like object from the C API is using the Py_buffer interface. This commit converts use of s#/y# to s*/y* and uses Py_buffer for accessing the underlying bytes array. I checked how hashlib is implemented in CPython and the the implementation agrees with its use of the Py_buffer interface as well as using BufferError in cases of bad buffer types. Sadly, there's no good way to test for ndim > 1 without writing our own C-backed Python type. Differential Revision: https://phab.mercurial-scm.org/D7879

lfs: avoid quadratic performance in processing server responses This is also adapted from the Facebook repo[1]. Unlike there, we were already reading the download stream in chunks and immediately writing it to disk, so we basically avoided the problem on download. There shouldn't be a lot of data to read on upload, but it's better to get rid of this pattern. [1] https://github.com/facebookexperimental/eden/commit/82df66ffe97e21f3ee73dfec093c87500fc1f6a7 Differential Revision: https://phab.mercurial-scm.org/D7882