Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 16:17:43 -0500] rev 35967
lfs: deduplicate oids in the transfer
Apparently, we can't rely on the server to deduplicate for us.
Sadly, the pointer object isn't hashable, so it can't be reduced by converting
it to a set. In order to be hashable, it needs to be immutable. I had a bunch
of code to change it to composition and forward the readonly dict methods to a
member dict. But the pointer is updated via __setitem__() when creating the
pointer file. So it didn't see worth adding all of that code to the class.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 15:26:49 -0500] rev 35966
cat: call the storage prefetch hook
It's not important to call in the case of a single file, but maybe it's better
to do so for consistency.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 14:31:32 -0500] rev 35965
archive: call the storage prefetch hook
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 00:33:28 -0500] rev 35964
lfs: prefetch lfs blobs during revert
The revert command oddly prints out what it will do before requesting the files
to be prefetched. But the 'need to transfer' line indicates the blobs are being
grouped.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 14:14:28 -0500] rev 35963
cmdutil: convert _revertprefetch() to a generic stored file hook (API)
This will be used by LFS to fetch required files in a group for multiple
commands, prior to being accessed. That avoids the one-at-a-time fetch when the
filelog wrapper goes to access it, and it is missing locally (which costs two
round trips to the server.) The core command list that needs this is probably
at least:
- annotate
- archive (which is also used by extdiff)
- cat
- diff
- export
- grep
- verify (sadly)
- anything that has the '{data}' template
There are no core users of the revert prefetch hook, and never have been since
it was introduced in 45e02cfad4bd for remotefilelog. Thanks to Yuya for
figuring out a way to reliably trigger the deprecated warning. Unfortunately,
it wanted to blame the caller of revert. Passing along an adjusted stack level
seemed the least bad choice (although it still blames a core function).
One thing to note is that the store lock isn't being held when this is called.
I'm not at all familiar with remotefilelog or its locking requirements, so this
may not be a big deal. Currently, LFS doesn't hold a lock when downloading
files. Even though largefiles doesn't either, I'm starting to think it should,
and maybe the .hg/store/lock isn't good enough to cover the globally shared
cache.
.. api::
The cmdutil._revertprefetch() hook point for prefetching stored files has
been replaced by the command agnostic cmdutil._prefetchfiles(). The new
function takes a list of files, instead of a list of lists of files.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Feb 2018 21:26:12 -0500] rev 35962
lfs: prefetch lfs blobs when applying merge updates
In addition to merge, this method ultimately gets called by many commands:
- backout
- bisect
- clone
- fetch
- graft
- import (without --bypass)
- pull -u
- rebase
- strip
- share
- transplant
- unbundle
- update
Additionally, it's also called by histedit, shelve, unshelve, and split, but it
seems that the related blobs should always be available locally for these.
For `hg update`, it happens after the normal argument checking and pre-update
hook processing, and remote corruption is detected prior to manipulating the
working directory. Other commands could use this treatment (archive, cat,
revert, etc), but this covers so many of the frequently used bulk commands, it
seems like a good starting point.
Losing the verbose message that prints the file name before a corrupt blob
aborts the command is a little sad, because there's no easy way to go from oid
to file name. I'd like to change that message to list the file name so it looks
cleaner and less cryptic, but the pointer object is nowhere near where it needs
to be to do this. So punt on that for now.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 14:53:16 -0500] rev 35961
lfs: factor out a method for extracting the pointer of a single file
This will be useful for filesets, among other things, instead of traversing the
whole context.
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:24:02 +0530] rev 35960
py3: add b'' to literals in check-config.py
# skip-blame because we are just adding b''
Differential Revision: https://phab.mercurial-scm.org/D2046
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 29 Dec 2017 05:40:49 +0530] rev 35959
check-config: specify the mode 'rb' to open the file
Differential Revision: https://phab.mercurial-scm.org/D2045
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:12:36 +0530] rev 35958
py3: use open() instead of file()
file() is not present in Python 3
Differential Revision: https://phab.mercurial-scm.org/D2044
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:12:01 +0530] rev 35957
py3: use pycompat.strkwargs() to convert kwargs' key to str
Differential Revision: https://phab.mercurial-scm.org/D2043
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:10:33 +0530] rev 35956
py3: add __bytes__() for mq.patchheader and make sure __str__ returns str
Before this patch, __str__() function for mq.patchheader class return bytes
which is not str on Python 3. So let's move that logic to __bytes__() and for
__str__() convert the return value of __bytes__() to str.
Differential Revision: https://phab.mercurial-scm.org/D2042
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:01:35 +0530] rev 35955
py3: use "%d" to convert integer to bytes
Differential Revision: https://phab.mercurial-scm.org/D2041
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 12:59:57 +0530] rev 35954
py3: use .startswith() instead of bytes[0]
Differential Revision: https://phab.mercurial-scm.org/D2040
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 12:58:51 +0530] rev 35953
py3: slice on bytes to prevent getting the ascii values
Differential Revision: https://phab.mercurial-scm.org/D2039
Pulkit Goyal <7895pulkit@gmail.com> [Sun, 04 Feb 2018 20:46:26 +0530] rev 35952
py3: add r'' to convert keys to keyword arguments to str
# skip-blame as we are just adding r''
Differential Revision: https://phab.mercurial-scm.org/D2038