perf: measure slicing time in perfrevlogrevision Slicing a sparse delta chain can be expensive. We now benchmark the associated time.

perf: teach perfrevlogrevision about sparse reading Before this change, chunks were always read in a single block. Even in the sparse-read/sparse-revlog case. This gave a false view of the performance and could lead to memory consumption issue.

perf: use the same timer for all section of perfrevlogrevision Otherwise the -T json output is invalid.

obsutil: clarify the access to "repo" We use the variable multiple times and we might use it even more in the future. We use a temporary variable instead.

pycompat: adding Linux detection and fixing Mac Python 3 recommends detecting OSs with the prefix of the platform, but we were comparing the full string for macOS. We also didn't have Linux detection, which is convenient for extensions to use (rather than have some OSs detected by hg and some by the extension). Reference: https://docs.python.org/3/library/sys.html#sys.platform Differential Revision: https://phab.mercurial-scm.org/D5227

remotefilelog: fix various minor py3 problems # skip-blame b prefixes and pycompat.long, nothing remotely interesting Differential Revision: https://phab.mercurial-scm.org/D5223

remotefilelog: rip out a missed mention of lz4 Differential Revision: https://phab.mercurial-scm.org/D5222

ui: wrap whole _write() block with timeblockedsection I think the cost of color labeling is negligible compared to the I/O syscalls. Let's simply wrap the whole write() function so that we can eliminate _write() and _write_err() in later changeset.

ui: indent _writenobuf() to prepare moving bits from _write() functions

ui: simply concatenate messages before applying color labels This should be cheaper in space than applying labels for each message.

ui: simplify interface of low-level write() functions _write() and _write_err() will be replaced with fout.write() and ferr.write() respectively. This is the first step.

ui: factor out function that writes data to fout/ferr with labeling I'm thinking of adding an option to send status messages to stderr (or a dedicated command-server channel) so that structured output (e.g. JSON) would never be interleaved with non-formatter output. A unified write() interface helps to do that.

ui: consolidate places where _progclear() is called The progress bar has to be cleared when we start writing some data to the output stream. Let's make it always triggered immediately before switching by _colormode, so that we can easily factor out helper functions.

templater: compute revset lazily This speeds up e.g. "{ifcontains(rev, revset('::.'), ...)}" in common cases where 'rev' is near the working parent. The templater API is ugly, but it helps here. 'f' can be either a generator or a function returning a generator.

remotefilelog: drop compat code for "getbundle_shallow" wireprotocol command Doing some annotate on hgexperimental shows that getbundle_shallow used to exist in 2013 or before. We don't have any pre-2013 remotefilelog users except Fb themselves and I doubt they are going to use in-core remotefilelog. So it's safe to remove this. Differential Revision: https://phab.mercurial-scm.org/D5193

tests: remove lz4 as dependency while running tests One of the previous patch authored by Augie rips out the lz4 dependency and things should work without it. Now there are just 2-3 tests failing because of same change in emitrevisions() API. Differential Revision: https://phab.mercurial-scm.org/D5192

remotefilelogserver: add a matcher argument to _walkstreamfiles() Implementing narrow stream clones in core, I added an optional matcher argument. The function in remotefilelogserver.py does not know about that argument and does not accept that and hence some tests fails. Differential Revision: https://phab.mercurial-scm.org/D5191

shallowutil: introduce a helper function isenabled() This patch introduces a inenabled() function which will check whether remotefilelog is enabled or not. The function is then also used at all the places where check whether remotefilelog is enabled or not. The new function makes code easy to read without need to understand what is the constant involved and why we are checking repo.requirements. Differential Revision: https://phab.mercurial-scm.org/D5190

remotefilelog: remove some BC code related to streamclone Since remotefilelog is now moved to core and we have streamclone._walkstreamfiles() in core, we don't need to have the backward compatibility code. People with old mercurial version should use remotefilelog from hg-experimental repo as IMO remotefilelog will go under a good refactoring and old clients will break. # no-check-commit foo_bar function name Differential Revision: https://phab.mercurial-scm.org/D5189

remotefilelog: add some docstring Differential Revision: https://phab.mercurial-scm.org/D5131

remotefilelog: rename wireproto methods and remaining capabilities This is a mess, in part because there should be more constants throughout. I know we typically do exp- instead of the x_ business in this change, but I also had to use this in some function names, so I figured until I can break that coupling I'd go with this. If it's too unpleasant during review, let me know and I can probably clean it up some more. # no-check-commit due to new foo_bar naming - too hard to avoid right now :( Differential Revision: https://phab.mercurial-scm.org/D5129

remotefilelog: consolidate and rename on-disk store requirement The value of this constant appeared in too many places. While we're here, rename it to be more consistent with our naming conventions for experimental functionality. Differential Revision: https://phab.mercurial-scm.org/D5128

remotefilelog: consolidate and rename bundle2 capability Bonus: we catch a spot where the shallowrepo requirement was being misused as the bundle2 capability. Differential Revision: https://phab.mercurial-scm.org/D5127

remotefilelog: rename capability for legacy ssh file fetching method I think I want to delete this, but for now we'll just rename this so it's easy to tell apart from other strings that have the same value. Differential Revision: https://phab.mercurial-scm.org/D5126

remotefilelog: rip out lz4 support Some methods are left teased out so it's easier to build a migration extension from v1 packs to v2. I also anticipate those methods will be of use if we get around to adding configurable compression engines before we end up jettisoning packs. I think it would make sense to register one-shot `compress` and `decompress` methods on our compression engines in util.py, but indygreg mentioned that might not be straightforward. Differential Revision: https://phab.mercurial-scm.org/D4944

remotefilelog: bump pack file formats to version 2 Drop support for format 0 and 1 packs. Version 2 will be different in that it'll use zlib compression instead of lz4, as the latter isn't really suitable for core. If I can make it work, I'll make compression engines pluggable, but the logic for handling these files is spread out enough it's a little tricky to get that right. Test changes are only due to pack files being renamed. The contents of the packs are the same. Differential Revision: https://phab.mercurial-scm.org/D4943

remotefilelog: avoid hard-coding pack names in tests I'm about to alter the pack format, which alters the content-addressed name of the pack file. In order to reduce my own confusion I found it helpful to use an ls | head pipeline since we always want to inspect the newest pack. Differential Revision: https://phab.mercurial-scm.org/D4942

remotefilelog: remove now-unused wirepack code Differential Revision: https://phab.mercurial-scm.org/D5124

remotefilelogserver: remove pack-serving functionality Differential Revision: https://phab.mercurial-scm.org/D5123

remotefilelog: remove support for fetching packs We've never used this at Google, and it's strictly an optimization that indygreg and I are confident we can handle more elegantly later. Let's rip it out to keep the surface area of the imported remotefilelog small. Some tests have changes because the tests for regular packs and repacking were mixed with tests of fetching packs. I'm pretty sure I've done the right thing here. Differential Revision: https://phab.mercurial-scm.org/D5122