largefiles: add support for 'largefiles://' url scheme This changesets allows Mercurial to transparently download content from the configured largefile store. This handle all authentication and largefile protocol details. The target usecase is to leverage largefile infrastructure for clone bundle. See next changeset for details

largefiles: add a 'debuglfput' command to put largefile into the store We add a new debug command able to add arbitrary new files into the largefile store. This will be useful for debugging, testing and other unholy usecase.

debug: add a 'debugdownload' command This command resolve and fetch and URL through the Mercurial logic. Mercurial logic add various headers (including authentication) while resolving an URL so the commands helps with building the same request Mercurial would be doing. A new test file is created because we'll add more logic regarding Mercurial download logic and it will grow to a reasonable size.

revsetlang: raise ParseError to report invalid format character It's more common in revset and templater than raising Abort. I have a couple more patches to address exceptions caused by bad format string passed to revset() template function.

revsetlang: use iterator to track current argument in formatspec()

revsetlang: unnest "if True" in formatrevspec()

revsetlang: use str.find() to scan expr in formatspec() There should be no need to walk character one by one in Python.

revsetlang: avoid string concatenation in formatspec()

test-glog: hook cmdutil.getlogrevs() so -frREV is rewritten accordingly Before, these tests didn't match the real behavior of "log -frREV".

test-glog: dump computed set It's more important than the revset expression built from command options.

test-glog: drop uninteresting nodes from AST output This makes future test changes more readable.

lfs: remove the verification option when writing to the local store This partially reverts 417e8e040102 and bb6a80fc969a. But since there's now a dedicated download function, there's no functional change. The last sentence in the commit message of the latter is wrong- write() didn't need the one time hash check if verification wasn't requested. I suspect I missed 'read()' in there ("... but _read()_ also needs to do a one time check..."), because that did fail without the hash check before linking to the usercache. The write() method simply took the same check for consistency. While here, clarify that the write() method is *only* for storing content directly from filelog, which has already checked the hash. If someone can come up with a way to bridge the differences between writing to a file and sending a urlreq.request across the wire, we can create an upload() function and cleanup read() in a similar way. About the only common thread I see is an open() that verifies the content before returning a file descriptor.

lfs: use the localstore download method to transfer from remote stores Both gitlfsremote and file based remotes benefit from not requiring the whole file in memory (though the whole file is still loaded when passing through the revlog interface). With a method specific to downloading from a remote store, the misleading 'use hg verify' hint is removed. The behavior is otherwise unchanged, in that a download from both remote store types will yield a copy of the blob via util.atomictempfile. There's no response payload defined for the non 'download' actions, but the previous code attempted to read the payload in this case anyway. This refactored code made that more obvious, so any payload is printed as a debug message, just in case.

lfs: introduce a localstore method for downloading from remote stores The current local.write() method requires the full data, which means concatenating file chunks in memory when downloading from a git server. The dedicated method downloads in chunks, verifies the content on the fly, and creates the usercache hardlink if successful. It can also be used for the file system based remotestore. An explicit division of labor between downloading from a remote store (which should be verified) and writing to the store because of a commit or similar (which doesn't need verification), seems clearer. I can't figure out how to make a similar function for upload, because for a file remote store, it's a simple open/read/write operation. For a gitremote store, it's open the file and a urlreq.request(), and process that.

win32: do not call GetVolumePathName() with the minimum buffer length It fails on Windows XP even though the doc says "a safer but slower way to set the size of the return buffer is to call the GetFullPathName function, and then make sure that the buffer size is at least the same size as the full path that GetFullPathName returns." https://msdn.microsoft.com/en-us/library/windows/desktop/aa364996(v=vs.85).aspx Well, more "safe" way would be to simply rely on MAX_PATH for common scenarios.

win32: allocate buffer of maximum length for GetVolumeInformation() It's documented that "the maximum buffer size is MAX_PATH+1", which is slightly larger than 256. https://msdn.microsoft.com/en-us/library/windows/desktop/aa364993(v=vs.85).aspx

log: drop unused expr from return value of getlogrevs() Future patches will move some processing of the --follow option out of _makelogrevset(), where the returned 'expr' value will be less consistent with the 'revs'. So let's remove it from the public interface.

log: don't expand aliases in revset built from command options Only -rREV should be rewritten with user aliases.

graphlog: remove redundant check for empty revs

log: remove redundant handling of --limit A returned 'revs' set is pre-filtered by cmdutil.getlogrevs(). The result of displayer.flush() isn't used anymore, so removed.

log: use smartset.slice() to limit number of revisions to be displayed

log: merge getlogrevs() and getgraphlogrevs() cmdutil.graphlog() is updated to receive (revs, filematcher) as arguments to make sure that opts['graph'] is set when getlogrevs() is invoked.

revset: parse x^:: as (x^):: (issue5764) We have to make '::' a valid primary expression to parse 'x^::' as '(x)^(::)' first, but that doesn't change the language because a prefix operator '::y' precedes a primary '::'. I can't think of an intuitive meaning of '::', so it's just rejected. Given 'x::y' can be considered to default to {x = roots(), y = heads()}, '::' could be 'roots()::heads()', which seems not any useful.

test-run-tests: stabilize the test (issue5735) Previously there is a race condition because things happen in this order: 1. Check shouldStop 2. If shouldStop is false, print the diff 3. Call fail() -> set shouldStop The check and set should really happen in a same critical section. This patch adds a lock to address the issue. Test Plan: Run `run-tests.py -l test-run-tests.t` 20 times on gcc112 and the race condition does not reproduce. Differential Revision: https://phab.mercurial-scm.org/D1830

run-tests: avoid set PYTHONUSERBASE environment variable to None According to python documentation, get_config_var can return None and it was during running mercurial tests under a virtual environment with the local built mercurial installed as a python package with `pip install -e .`. On the other hand, if mercurial is installed in the user library folders with `pip install --local -e .` the issue doesn't reproduce. The fallback is that it will just set an empty string and everything should work as before, according to https://www.python.org/dev/peps/pep-0370/: The path to the user base directory can be overwritten with the environment variable PYTHONUSERBASE. The default location is used when PYTHONUSERBASE is not set or empty.

templatefilters: fix doc of basename() os.path.basename() does NOT strip trailing slashes. > Note that the result of this function is different from the Unix basename > program; where basename for '/foo/bar/' returns 'bar', the basename() > function returns an empty string (''). https://docs.python.org/2.7/library/os.path.html#os.path.basename

identify: document -r. explicitly how to disable wdir scanning (issue5622)

lfs: add a comment to describe subtle local blobstore open() behavior

tests: make #testcase available as env var in test It can be useful for e.g. helper functions to behave differently depending on which test case is being executed. This patch helps with that by making the #testcase case available as a TESTCASE environment variable. Differential Revision: https://phab.mercurial-scm.org/D1816

archive: add support to specify hidden revs if directaccess config is set Differential Revision: https://phab.mercurial-scm.org/D1812