phabricator: use .arcconfig for the callsign if not set locally (issue6243) This makes things easier for people working with more than one repository because this file can be committed to each repository. The bug report asks to read <repo>/.arcrc, but AFAICT, that file lives in ~/ and holds the credentials. And we already track an .arcconfig file. Any callsign set globally is still used if that is all that is present, but .arcconfig will override it if available. The idea behind letting the local hgrc override .arcconfig is that the developer may need to do testing against another server, and not dirty the working directory. Originally I was going to just try to read the callsign in `getrepophid()` if it wasn't present in the hg config. That works fine, but I think it also makes sense to read the URL from this file too. That would have worked less well because `readurltoken()` doesn't have access to the repo object to know where to find the file. Supplimenting the config mechanism is less magical because it reports the source and value of the properties used, and it doesn't need to read the file twice. Invalid hgrc files generally cause the program to abort. I only flagged it as a warning here because it's not our config file, not crucial to the whole program operating, and really shouldn't be corrupt in the typical case where it is checked into the repo. Differential Revision: https://phab.mercurial-scm.org/D7934

config: add a function to insert non-file based, but overridable settings This will be used in the next patch. Until relatively recently (473510bf0575), there was no official way for extensions to inject per-repo config data, so it probably makes sense that `ui.setconfig()` items are sticky, and not affected by loading more config files. But that makes it cumbersome if the extension wants to allow the data it might add to be overridden by any data in the local hgrc file. The only thing I could get to work was to load the local hgrc first, and then check if the source for the config item that should be overridden was *not* the local hgrc file name. But that's brittle because in addition to the file name, the source contains the line number, there are the usual '\' vs '/' platform differences, etc. Differential Revision: https://phab.mercurial-scm.org/D7933

tests: restore phabricator tests and regenerate the recordings These contain the new API chatter. Most of the changes are because some new commits were created, but they're pretty obviously equivalent. I have no idea why the last recording contains real data, whereas it previously looked fake. Differential Revision: https://phab.mercurial-scm.org/D7920

hgrc: introduce HGRCSKIPREPO to skip reading the repository's hgrc We had a way to change the behavior regarding reading the global and user config, but we had nothing regarding the repository hgrc itself. This option is useful in situation where scripts need to be able to work around strange configuration set by the user in his repository. (and were HGPLAIN is not enough). Differential Revision: https://phab.mercurial-scm.org/D7807

debugcommands: move away from line buffered output on binary stream Line buffering on binary file objects is apparently undefined behavior in Python and emits a RuntimeWarning on Python 3.8. See https://bugs.python.org/issue32236. This commit changes the I/O logging file descriptor from line buffered to unbuffered to work around this. I'm no fan of unbuffered I/O for performance reasons. But I don't think it is an issue here given the nature of the code. With this change, test-ssh-proto.t now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7948

py3: conditionalize test-lfs-serve-access.t for Python 3.8 This is another case where Python 3.8's traceback printing varies subtly from older Python versions. Again, I'm not sure why. But this is apparently the new behavior. With this change, test-lfs-serve-access.t now passes on Python 3.8! Differential Revision: https://phab.mercurial-scm.org/D7947

py3: add extra traceback line present on Python 3.8 I'm not sure why Python 3.8 is outputting this line. It appears to be a change in behavior of formatting tracebacks on Python 3.8. So let's add it to expected output. With this change, test-hook.t now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7946

py3: conditionalize test-flagprocessor.t on Python 3.8 For reasons I don't understand, Python 3.8 is outputting a different lint in the traceback than prior Pythons. The lines in question are: flagutil.addflagprocessor( REVIDX_NOOP, (noopdonothingread, noopdonothing, validatehash,) ) Python <3.8 prints the 2nd line but 3.8 the first line. Perhaps Python changed its traceback logic to always print the first line of a multiple line expression? Whatever the case, with this change, the test now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7945

tests: conditionalize test-hightlight.t on pygments version Output changed slightly in pygments 2.5. I thought it was easier to copy the line and add a version check than to add a regular expression because the line has some special characters. I also like tests explicitly calling out where they vary. Differential Revision: https://phab.mercurial-scm.org/D7943

hgdemandimport: apply lazy module loading to sys.meta_path finders Python's `sys.meta_path` finders are the primary objects whose job it is to find a module at import time. When `import` is called, Python iterates objects in this list and calls `o.find_spec(...)` to find a `ModuleSpec` (or None if the module couldn't be found by that finder). If no meta path finder can find a module, import fails. One of the default meta path finders is `PathFinder`. Its job is to import modules from the filesystem and is probably the most important importer. This finder looks at `sys.path` and `sys.path_hooks` to do its job. The `ModuleSpec` returned by `MetaPathImporter.find_spec()` has a `loader` attribute, which defines the concrete module loader to use. `sys.path_hooks` is a hook point for teaching `PathFinder` to instantiate custom loader types. Previously, we injected a custom `sys.path_hook` that told `PathFinder` to wrap the default loaders with a loader that creates a module object that is lazy. This approach worked. But its main limitation was that it only applied to the `PathFinder` meta path importer. There are other meta path importers that are registered. And in the case of PyOxidizer loading modules from memory, `PathFinder` doesn't come into play since PyOxidizer's own meta path importer was handling all imports. This commit changes our approach to lazy module loading by proxying all meta path importers. Specifically, we overload the `find_spec()` method to swap in a wrapped loader on the `ModuleSpec` before it is returned. The end result of this is all meta path importers should be lazy. As much as I would have loved to utilize .__class__ manipulation to achieve this, some meta path importers are implemented in C/Rust in such a way that they cannot be monkeypatched. This is why we use __getattribute__ to define a proxy. Also, this change could theoretically open us up to regressions in meta path importers whose loader is creating module objects which can't be monkeypatched. But I'm not aware of any of these in the wild. So I think we'll be safe. According to hyperfine, this change yields a decent startup time win of 5-6ms: ``` Benchmark #1: ~/.pyenv/versions/3.6.10/bin/python ./hg version Time (mean ± σ): 86.8 ms ± 0.5 ms [User: 78.0 ms, System: 8.7 ms] Range (min … max): 86.0 ms … 89.1 ms 50 runs Time (mean ± σ): 81.1 ms ± 2.7 ms [User: 74.5 ms, System: 6.5 ms] Range (min … max): 77.8 ms … 90.5 ms 50 runs Benchmark #2: ~/.pyenv/versions/3.7.6/bin/python ./hg version Time (mean ± σ): 78.9 ms ± 0.6 ms [User: 70.2 ms, System: 8.7 ms] Range (min … max): 78.1 ms … 81.2 ms 50 runs Time (mean ± σ): 73.4 ms ± 0.6 ms [User: 65.3 ms, System: 8.0 ms] Range (min … max): 72.4 ms … 75.7 ms 50 runs Benchmark #3: ~/.pyenv/versions/3.8.1/bin/python ./hg version Time (mean ± σ): 78.1 ms ± 0.6 ms [User: 70.2 ms, System: 7.9 ms] Range (min … max): 77.4 ms … 80.9 ms 50 runs Time (mean ± σ): 72.1 ms ± 0.4 ms [User: 64.4 ms, System: 7.6 ms] Range (min … max): 71.4 ms … 74.1 ms 50 runs ``` Differential Revision: https://phab.mercurial-scm.org/D7954

hgdemandimport: disable on Python 3.5 The demand importer functionality isn't working at all on Python 3.5. I'm not sure what's wrong. Since it isn't working, let's disable it completely. ``` $ HGRCPATH= hyperfine -w 1 -r 50 -- "~/.pyenv/versions/3.5.9/bin/python ./hg version" \ "HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version" Benchmark #1: ~/.pyenv/versions/3.5.9/bin/python ./hg version Time (mean ± σ): 163.7 ms ± 2.2 ms [User: 148.5 ms, System: 15.7 ms] Range (min … max): 161.0 ms … 170.2 ms 50 runs Benchmark #2: HGDEMANDIMPORT=disable ~/.pyenv/versions/3.5.9/bin/python ./hg version Time (mean ± σ): 164.3 ms ± 1.4 ms [User: 148.2 ms, System: 16.6 ms] Range (min … max): 161.4 ms … 169.8 ms 50 runs ``` Differential Revision: https://phab.mercurial-scm.org/D7953

py3: suppress unraisable exceptions in test-worker.t Python 3.8 calls sys.unraisablehook when an unraisable exception is encountered. The default behavior is to print a warning. test-worker.t was triggering this hook due to a race between a newly forked process exiting and that process's _os.register_at_fork handlers running. I was seeing the stdlib's random module in the stack re-seeding itself. Although there could be other after-fork handlers in the mix. This commit defines sys.unraisablehook to effectively no-op. This suppresses the warning and makes test output on Python 3.8 consistent with prior versions. test-worker.t now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7949

rust: add a README In particular to explain how to build any of the rust. It's neither obvious, nor easy to find out, nor easy to determine if you did it right without some documentation. Differential Revision: https://phab.mercurial-scm.org/D7952

rust: move hgcli's README out of the way My understanding is that it's not meant to be used in the current form. Differential Revision: https://phab.mercurial-scm.org/D7951

verify: avoid spurious integrity warnings in verbose mode (issue6172) The issue seems to revolve around renames in filtered commits, and only occurred in verbose mode. The problem occurs in the `# check renames` stage, around line 577. Without using the unfiltered repo, this test would have printed: $ hg verify -v repository uses revlog format 1 checking changesets checking manifests crosschecking files in changesets and manifests checking files foo@25: checking rename of 71ec0570c325: filtered revision '25' foobar@26: checking rename of 1b549296015b: filtered revision '26' checked 28 changesets with 16 changes to 11 files 2 integrity errors encountered! (first damaged changeset appears to be 25) [1] Differential Revision: https://phab.mercurial-scm.org/D7950

py3: glob over exception in test-check-py3-compat.t Python 3.6+ raise ModuleNotFoundError and older versions raise ImportError. Glob over the exception differences. For whatever reason, we were already doing this for one failure. But not all occurrences of ModuleNotFoundError were changed. Who knows. This test should now pass on all Python versions (although I didn't check Windows). Differential Revision: https://phab.mercurial-scm.org/D7939

py3: string normalization and I/O tweaks in test-lfs.t The print was inserting b'' on Python 3. In addition, since we weren't writing to the ui instance (which isn't readily available in this function), output order could get mixed up. We add some pycompat casts and a stdout flush to make the test happy on all Python versions. Differential Revision: https://phab.mercurial-scm.org/D7938

help: minor copy editing to the `config.format` section Differential Revision: https://phab.mercurial-scm.org/D7936

changectx: mark parent of changesets as non filtered If a node is not filtered, its parents cannot be filtered. Differential Revision: https://phab.mercurial-scm.org/D7502

changectx: use unfiltered changelog to walk ancestors in annotate Since we are only walking ancestors, it is safe to use an unfiltered repository. (Because if the original rev is not filtered, none of its ancestors will be). Differential Revision: https://phab.mercurial-scm.org/D7501

localrepo: also fast past the parents of working copies parents There are descent odds that they will be needed too. So we also cache and fastpath them. Differential Revision: https://phab.mercurial-scm.org/D7498

localrepo: recognize trivial request for '.' Same logic as for `null`, this is a command request and skipping the revset logic can avoid triggering the changelog filtering logic. Differential Revision: https://phab.mercurial-scm.org/D7495

localrepo: fastpath access to "." "." is just an alias for `p1(wdir())`, let us handle it that way. Differential Revision: https://phab.mercurial-scm.org/D7494

localrepo: also fastpath access to working copy parents when possible If the filter level guarantee that the working copy parents will be visible, we allow fast path access to them. With this change multiple commands can now run without triggering filtering. After using the quick access mechanism introduced, the whole series results in pretty good performance gain: ``` All benchmarks: before after ratio [8e095512] [36b2f659] - 711±0.8ms 60.7±0.2ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 712±0.8ms 61.6±0.2ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 690±1ms 93.5±0.3ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 688±1ms 93.8±0.3ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 714±1ms 60.7±0.8ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 713±1ms 60.9±0.3ms 0.09 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±1ms 93.7±0.2ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 687±2ms 92.8±0.2ms 0.14 simple_command.read.diff.empty.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 799±2ms 98.1±0.6ms 0.12 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 800±0.8ms 100.0±0.4ms 0.12 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 711±0.9ms 111±0.2ms 0.16 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 711±1ms 112±0.3ms 0.16 simple_command.read.export.bare.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 760±1ms 59.8±0.1ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 763±2ms 62.2±0.3ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±1ms 93.1±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 688±1ms 94.3±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 1) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 763±1ms 60.1±0.2ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 763±1ms 62.1±0.4ms 0.08 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py2.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] - 689±0.8ms 93.2±0.2ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYc-HGWITHRUSTEXTcpython] - 687±0.9ms 94.1±0.3ms 0.14 simple_command.read.status.wc_clean.default.time_bench('mercurial-filtered-2019-11-22', 'zstd', 'default', True, True, True, True, True, 2) [citrea/virtualenv-py3.7-pyyaml-HGMODULEPOLICYrust+c-HGWITHRUSTEXTcpython] ``` Differential Revision: https://phab.mercurial-scm.org/D7492

examples: refer to nightly rustfmt in Windows-compatible way Thanks to Jun Wu for the tip. I found that the new form also gave better error messages when the nightly rustfmt wasn't installed (it told me which command to run instead of just saying "error: not a file: <some path>"). Differential Revision: https://phab.mercurial-scm.org/D7911

convert: refactor authormap into separate function for outside use Differential Revision: https://phab.mercurial-scm.org/D7732

remotefilelog: fix opening validatecachelog in text mode

cext: fix compiler warning about sign changing line.len is a Py_ssize_t, and we're casing to size_t (unsigned). On my compiler, this causes a warning to be emitted: ``` mercurial/cext/manifest.c: In function 'pathlen': mercurial/cext/manifest.c:48:44: warning: operand of ?: changes signedness from 'Py_ssize_t' {aka 'long int'} to 'long unsigned int' due to unsignedness of other operand [-Wsign-compare] return (end) ? (size_t)(end - l->start) : l->len; ^~~~~~ ``` Differential Revision: https://phab.mercurial-scm.org/D7913

sha1dc: avoid including the nonexistent stdint.h with Visual Studio 2008 Differential Revision: https://phab.mercurial-scm.org/D7903

py3: fix curses chunkselector fallback (when diffs are too large) on py3 Previously we showed the message using Exception.message, which is removed in py3. Since crecordmod.fallbackerror inherits from error.Abort, we can just use `b'%s' % exception` to print the message. This does not print the hint, but that's fine - we don't set one. We inherit from error.Abort so that if a codepath doesn't handle fallback specially, it exits to the terminal with a sane message instead of an unknown exception error. Differential Revision: https://phab.mercurial-scm.org/D7912

transaction: allow finalizer to add finalizer It will make some code (persistent nodemap related) simpler to write, because higher level code can blindly queue finalization without thinking too hard about the context. Differential Revision: https://phab.mercurial-scm.org/D7833

tests: stabilize test-subrepo-svn.t on Windows This partially reverts 991e4404e910, because the URL form of `C:\...` gets escaped to `C%3A/...`, which breaks the substitution of $TESTTMP. The forget command on 'notafile*' errored out with: notafile*: The filename, directory name, or volume label syntax is incorrect which I think is because '*' isn't a legal character in a file name (though I can't trigger this directly from MSYS or cmd.exe for some reason). Differential Revision: https://phab.mercurial-scm.org/D7816

rebase: fix bug where `--collapse` would apply diff on missing file Even though the file was missing, the rebase would succeed. Differential Revision: https://phab.mercurial-scm.org/D7897

rebase: extract a variable for a repeated `repo[p1]` I'll add another use site in the next patch. Differential Revision: https://phab.mercurial-scm.org/D7896

graftcopies: document why the function is useful at all Despite having spent a significant amount on time on the copy-tracing code, I thought `graftcopies()` (formerly known as `duplicatecopies()`) was needed to duplicate copies after calling `merge.update()` to do a merge (as `merge.graft()` does), but it's actually usually not needed; `merge.update()` takes care of most copies. This patch documents what the function is for. Differential Revision: https://phab.mercurial-scm.org/D7861

graftcopies: remove `skip` and `repo` arguments The `skip` argument was added in 2ba6c9b4e0eb (rebase: fix bug that caused transitive copy records to disappear (issue4192), 2014-06-07) in order to fix https://bz.mercurial-scm.org/show_bug.cgi?id=4192. I ran tests at that commit without the `skiprev` argument and the only difference I noticed was that `test-rebase-collapse.t` failed differently, in the call that is now on line 501. Without the `skiprev` argument, that call would end up creating another commit because it tried to record an invalid copy. With the previous patch in this series, such invalid copies are no longer recorded, so it seems we don't need the `skip` argument anymore. I also removed the `repo` argument since that also becomes unused with the removal of the `skip` argument. Differential Revision: https://phab.mercurial-scm.org/D7860

graftcopies: use _filter() for filtering out invalid copies `graftcopies()` (formerly called `duplicatecopies()`) checked that the copy destination existed in the working copy, but it didn't check that copy source existed in the parent of the working copy. In `test-graft.t` we can see that as warnings about not finding ancestors of the copied files, and also empty commits getting created. This patch uses the existing `_filter()` function for filtering out invalid copies. In addition to the aforementioned types, that also includes copies where source and destination is the same. Differential Revision: https://phab.mercurial-scm.org/D7859

copies: replace duplicatecopies() by function that takes contexts The callers mostly have context objects, so let's avoid looking up the same context objects inside `duplicatecopies()`. I also renamed the function to `graftcopies()` since I think that better matches its purpose. I did it in the same commit so it's easier for extensions to switch between the functions. Differential Revision: https://phab.mercurial-scm.org/D7858

graft: extract repo[None] to a variable I plan to allow the caller pass in an overlayworkingctx, so this prepares for that. Differential Revision: https://phab.mercurial-scm.org/D7857

rust-core: fix typo in comment Differential Revision: https://phab.mercurial-scm.org/D7895

sha1dc: use buffer protocol when parsing arguments Without this, functions won't accept bytearray, memoryview, or other types that can be exposed as bytes to the C API. The most resilient way to obtain a bytes-like object from the C API is using the Py_buffer interface. This commit converts use of s#/y# to s*/y* and uses Py_buffer for accessing the underlying bytes array. I checked how hashlib is implemented in CPython and the the implementation agrees with its use of the Py_buffer interface as well as using BufferError in cases of bad buffer types. Sadly, there's no good way to test for ndim > 1 without writing our own C-backed Python type. Differential Revision: https://phab.mercurial-scm.org/D7879

lfs: avoid quadratic performance in processing server responses This is also adapted from the Facebook repo[1]. Unlike there, we were already reading the download stream in chunks and immediately writing it to disk, so we basically avoided the problem on download. There shouldn't be a lot of data to read on upload, but it's better to get rid of this pattern. [1] https://github.com/facebookexperimental/eden/commit/82df66ffe97e21f3ee73dfec093c87500fc1f6a7 Differential Revision: https://phab.mercurial-scm.org/D7882

lfs: check content length after downloading content Adapted from the Facebook repo[1]. The intent is to distinguish between the connection dying and getting served a corrupt blob. The original message: HTTP makes no provision to tell your client that you failed halfway through producing your response and won't have the answer they're looking for. So, if a LFS server fails while producing a response, then we'll report an OID mismatch. We can do a little better and disambiguate between "the server sent us the wrong blob" (very scary) and "the server crashed" (merely annoying) by looking at the content length of the response we got back. If it's not what was advertised, we can reasonably safely assume the server crashed. [1] https://github.com/facebookexperimental/eden/commit/2a4a6fab4e882ed89b948bfc1e7d56d7c3c99dd2 Differential Revision: https://phab.mercurial-scm.org/D7881

lfs: rename a variable to clarify its use This is the response object, not a request. Differential Revision: https://phab.mercurial-scm.org/D7880

sha1dc: use proper string functions on Python 2/3 PyString_FromStringAndSize doesn't exist on Python 3: we need to use PyUnicode_FromStringAndSize. The extension now compiles without warnings on Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D7878

sha1dc: declare all variables at begininng of block This is required to appease ancient C language standards, which msvc 2008 still requires for Python 2.7 on Windows. Differential Revision: https://phab.mercurial-scm.org/D7877

sha1dc: manually define integer types on msvc 2008 Python 2.7 on Windows builds with MSVC 2008, which doesn't include stdint.h. So we need to check for the compiler version and manually define missing types when it is ancient. Differential Revision: https://phab.mercurial-scm.org/D7876

packaging: leverage os.path.relpath() in setup.py `os.path.relpath()` has existed since Python 2.6, so we can safely use it. This fixes a bug in the current code when the common prefix is "/" (in which case `uplevel` would be one less than it should). Differential Revision: https://phab.mercurial-scm.org/D7875

rust-utils: add util to find a slice in another slice Differential Revision: https://phab.mercurial-scm.org/D7863

dirstate: move rust fast-path calling code to its own method This logic is about to get bigger, this will make it easier to read and not pollute the main Python logic. Differential Revision: https://phab.mercurial-scm.org/D7862

lfs: add "bytes" as the unit to the upload/download progress bar Facebook also passes `util.bytecount()` as a pretty formatter here, but our progress bar doesn't support that. Differential Revision: https://phab.mercurial-scm.org/D7872

phabricator: post revisions in ascending topological order (issue6241) The parent in phabricator ends up being the last revision posted, so sorting the user input into ascending order should be enough to preserve the proper relationships. Differential Revision: https://phab.mercurial-scm.org/D7874

doc: fix references to `revset.abstractsmartset` Differential Revision: https://phab.mercurial-scm.org/D7873

fsmonitor: properly handle str ex.msg ex.msg is always a str, since pywatchman uses str for exception messages. This commit removes a b'' from a string compare to avoid types mismatch and adds a coercion to bytes before stuffing the exception message on our local exception type, which uses bytes for the message elsewhere in this file. Differential Revision: https://phab.mercurial-scm.org/D7855

verify: allow the storage to signal when renames can be tested on `skipread` This applies the new marker in the lfs handler to show it in action, and adds the test mentioned at the beginning of the series to show that fulltext isn't necessary in the LFS case. The existing `skipread` isn't enough, because it is also set if an error occurs reading the revlog data, or the data is censored. It could probably be cleared, but then it technically violates the interface contract. That wouldn't matter for the existing verify algorithm, but it isn't clear how that will change as alternate storage support is added. The flag is probably pretty revlog specific, given the comments in verify.py. But there's already filelog specific stuff in there and I'm not sure what future storage will bring, so I don't want to over-engineer this. Likewise, I'm not sure that we want the verify method for each storage type to completely drive the bus when it comes to detecting renames, so I don't want to go down the rabbithole of having verifyintegrity() return metadata hints at this point. Differential Revision: https://phab.mercurial-scm.org/D7713

lfs: don't skip locally available blobs when verifying The `skipflags` config was introduced in a2ab9ebcd85b, which specifically calls out downloading and storing all blobs as potentially too expensive. But I don't see any reason to skip blobs that are already available locally. Hashing the blob is the only way to indirectly verify the rawdata content stored in the revlog. (The note in that commit about skipping renamed is still correct, but the reason given about needing fulltext isn't.) Differential Revision: https://phab.mercurial-scm.org/D7712