Mon, 13 Jan 2020 11:11:20 -0800 rebase: extract a variable for a repeated `repo[p1]`
Martin von Zweigbergk <martinvonz@google.com> [Mon, 13 Jan 2020 11:11:20 -0800] rev 44094
rebase: extract a variable for a repeated `repo[p1]` I'll add another use site in the next patch. Differential Revision: https://phab.mercurial-scm.org/D7896
Sun, 29 Dec 2019 17:53:48 -0800 graftcopies: document why the function is useful at all
Martin von Zweigbergk <martinvonz@google.com> [Sun, 29 Dec 2019 17:53:48 -0800] rev 44093
graftcopies: document why the function is useful at all Despite having spent a significant amount on time on the copy-tracing code, I thought `graftcopies()` (formerly known as `duplicatecopies()`) was needed to duplicate copies after calling `merge.update()` to do a merge (as `merge.graft()` does), but it's actually usually not needed; `merge.update()` takes care of most copies. This patch documents what the function is for. Differential Revision: https://phab.mercurial-scm.org/D7861
Fri, 27 Dec 2019 13:47:59 -0800 graftcopies: remove `skip` and `repo` arguments
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Dec 2019 13:47:59 -0800] rev 44092
graftcopies: remove `skip` and `repo` arguments The `skip` argument was added in 2ba6c9b4e0eb (rebase: fix bug that caused transitive copy records to disappear (issue4192), 2014-06-07) in order to fix https://bz.mercurial-scm.org/show_bug.cgi?id=4192. I ran tests at that commit without the `skiprev` argument and the only difference I noticed was that `test-rebase-collapse.t` failed differently, in the call that is now on line 501. Without the `skiprev` argument, that call would end up creating another commit because it tried to record an invalid copy. With the previous patch in this series, such invalid copies are no longer recorded, so it seems we don't need the `skip` argument anymore. I also removed the `repo` argument since that also becomes unused with the removal of the `skip` argument. Differential Revision: https://phab.mercurial-scm.org/D7860
Fri, 27 Dec 2019 15:14:19 -0800 graftcopies: use _filter() for filtering out invalid copies
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Dec 2019 15:14:19 -0800] rev 44091
graftcopies: use _filter() for filtering out invalid copies `graftcopies()` (formerly called `duplicatecopies()`) checked that the copy destination existed in the working copy, but it didn't check that copy source existed in the parent of the working copy. In `test-graft.t` we can see that as warnings about not finding ancestors of the copied files, and also empty commits getting created. This patch uses the existing `_filter()` function for filtering out invalid copies. In addition to the aforementioned types, that also includes copies where source and destination is the same. Differential Revision: https://phab.mercurial-scm.org/D7859
Mon, 06 Jan 2020 15:24:36 -0800 copies: replace duplicatecopies() by function that takes contexts
Martin von Zweigbergk <martinvonz@google.com> [Mon, 06 Jan 2020 15:24:36 -0800] rev 44090
copies: replace duplicatecopies() by function that takes contexts The callers mostly have context objects, so let's avoid looking up the same context objects inside `duplicatecopies()`. I also renamed the function to `graftcopies()` since I think that better matches its purpose. I did it in the same commit so it's easier for extensions to switch between the functions. Differential Revision: https://phab.mercurial-scm.org/D7858
Fri, 27 Dec 2019 13:03:40 -0800 graft: extract repo[None] to a variable
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Dec 2019 13:03:40 -0800] rev 44089
graft: extract repo[None] to a variable I plan to allow the caller pass in an overlayworkingctx, so this prepares for that. Differential Revision: https://phab.mercurial-scm.org/D7857
Thu, 16 Jan 2020 00:30:08 +0800 rust-core: fix typo in comment
Aay Jay Chan <aayjaychan@itopia.com.hk> [Thu, 16 Jan 2020 00:30:08 +0800] rev 44088
rust-core: fix typo in comment Differential Revision: https://phab.mercurial-scm.org/D7895
Tue, 14 Jan 2020 18:59:49 -0800 sha1dc: use buffer protocol when parsing arguments
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 14 Jan 2020 18:59:49 -0800] rev 44087
sha1dc: use buffer protocol when parsing arguments Without this, functions won't accept bytearray, memoryview, or other types that can be exposed as bytes to the C API. The most resilient way to obtain a bytes-like object from the C API is using the Py_buffer interface. This commit converts use of s#/y# to s*/y* and uses Py_buffer for accessing the underlying bytes array. I checked how hashlib is implemented in CPython and the the implementation agrees with its use of the Py_buffer interface as well as using BufferError in cases of bad buffer types. Sadly, there's no good way to test for ndim > 1 without writing our own C-backed Python type. Differential Revision: https://phab.mercurial-scm.org/D7879
Tue, 14 Jan 2020 20:05:37 -0500 lfs: avoid quadratic performance in processing server responses
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 20:05:37 -0500] rev 44086
lfs: avoid quadratic performance in processing server responses This is also adapted from the Facebook repo[1]. Unlike there, we were already reading the download stream in chunks and immediately writing it to disk, so we basically avoided the problem on download. There shouldn't be a lot of data to read on upload, but it's better to get rid of this pattern. [1] https://github.com/facebookexperimental/eden/commit/82df66ffe97e21f3ee73dfec093c87500fc1f6a7 Differential Revision: https://phab.mercurial-scm.org/D7882
Tue, 14 Jan 2020 19:42:24 -0500 lfs: check content length after downloading content
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 19:42:24 -0500] rev 44085
lfs: check content length after downloading content Adapted from the Facebook repo[1]. The intent is to distinguish between the connection dying and getting served a corrupt blob. The original message: HTTP makes no provision to tell your client that you failed halfway through producing your response and won't have the answer they're looking for. So, if a LFS server fails while producing a response, then we'll report an OID mismatch. We can do a little better and disambiguate between "the server sent us the wrong blob" (very scary) and "the server crashed" (merely annoying) by looking at the content length of the response we got back. If it's not what was advertised, we can reasonably safely assume the server crashed. [1] https://github.com/facebookexperimental/eden/commit/2a4a6fab4e882ed89b948bfc1e7d56d7c3c99dd2 Differential Revision: https://phab.mercurial-scm.org/D7881
Tue, 14 Jan 2020 18:02:20 -0500 lfs: rename a variable to clarify its use
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 18:02:20 -0500] rev 44084
lfs: rename a variable to clarify its use This is the response object, not a request. Differential Revision: https://phab.mercurial-scm.org/D7880
Tue, 14 Jan 2020 17:53:43 -0800 sha1dc: use proper string functions on Python 2/3
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 14 Jan 2020 17:53:43 -0800] rev 44083
sha1dc: use proper string functions on Python 2/3 PyString_FromStringAndSize doesn't exist on Python 3: we need to use PyUnicode_FromStringAndSize. The extension now compiles without warnings on Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D7878
Tue, 14 Jan 2020 17:39:12 -0800 sha1dc: declare all variables at begininng of block
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 14 Jan 2020 17:39:12 -0800] rev 44082
sha1dc: declare all variables at begininng of block This is required to appease ancient C language standards, which msvc 2008 still requires for Python 2.7 on Windows. Differential Revision: https://phab.mercurial-scm.org/D7877
Tue, 14 Jan 2020 17:37:04 -0800 sha1dc: manually define integer types on msvc 2008
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 14 Jan 2020 17:37:04 -0800] rev 44081
sha1dc: manually define integer types on msvc 2008 Python 2.7 on Windows builds with MSVC 2008, which doesn't include stdint.h. So we need to check for the compiler version and manually define missing types when it is ancient. Differential Revision: https://phab.mercurial-scm.org/D7876
Tue, 14 Jan 2020 14:18:11 -0800 packaging: leverage os.path.relpath() in setup.py
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 Jan 2020 14:18:11 -0800] rev 44080
packaging: leverage os.path.relpath() in setup.py `os.path.relpath()` has existed since Python 2.6, so we can safely use it. This fixes a bug in the current code when the common prefix is "/" (in which case `uplevel` would be one less than it should). Differential Revision: https://phab.mercurial-scm.org/D7875
Tue, 14 Jan 2020 18:00:05 +0100 rust-utils: add util to find a slice in another slice
Raphaël Gomès <rgomes@octobus.net> [Tue, 14 Jan 2020 18:00:05 +0100] rev 44079
rust-utils: add util to find a slice in another slice Differential Revision: https://phab.mercurial-scm.org/D7863
Tue, 14 Jan 2020 16:00:57 +0100 dirstate: move rust fast-path calling code to its own method
Raphaël Gomès <rgomes@octobus.net> [Tue, 14 Jan 2020 16:00:57 +0100] rev 44078
dirstate: move rust fast-path calling code to its own method This logic is about to get bigger, this will make it easier to read and not pollute the main Python logic. Differential Revision: https://phab.mercurial-scm.org/D7862
Tue, 14 Jan 2020 00:52:53 -0500 lfs: add "bytes" as the unit to the upload/download progress bar
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 00:52:53 -0500] rev 44077
lfs: add "bytes" as the unit to the upload/download progress bar Facebook also passes `util.bytecount()` as a pretty formatter here, but our progress bar doesn't support that. Differential Revision: https://phab.mercurial-scm.org/D7872
Tue, 14 Jan 2020 16:37:45 -0500 phabricator: post revisions in ascending topological order (issue6241)
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 16:37:45 -0500] rev 44076
phabricator: post revisions in ascending topological order (issue6241) The parent in phabricator ends up being the last revision posted, so sorting the user input into ascending order should be enough to preserve the proper relationships. Differential Revision: https://phab.mercurial-scm.org/D7874
Tue, 14 Jan 2020 16:29:03 -0500 doc: fix references to `revset.abstractsmartset`
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Jan 2020 16:29:03 -0500] rev 44075
doc: fix references to `revset.abstractsmartset` Differential Revision: https://phab.mercurial-scm.org/D7873
Mon, 13 Jan 2020 20:09:32 -0800 fsmonitor: properly handle str ex.msg
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 13 Jan 2020 20:09:32 -0800] rev 44074
fsmonitor: properly handle str ex.msg ex.msg is always a str, since pywatchman uses str for exception messages. This commit removes a b'' from a string compare to avoid types mismatch and adds a coercion to bytes before stuffing the exception message on our local exception type, which uses bytes for the message elsewhere in this file. Differential Revision: https://phab.mercurial-scm.org/D7855
Mon, 23 Dec 2019 01:12:20 -0500 verify: allow the storage to signal when renames can be tested on `skipread`
Matt Harbison <matt_harbison@yahoo.com> [Mon, 23 Dec 2019 01:12:20 -0500] rev 44073
verify: allow the storage to signal when renames can be tested on `skipread` This applies the new marker in the lfs handler to show it in action, and adds the test mentioned at the beginning of the series to show that fulltext isn't necessary in the LFS case. The existing `skipread` isn't enough, because it is also set if an error occurs reading the revlog data, or the data is censored. It could probably be cleared, but then it technically violates the interface contract. That wouldn't matter for the existing verify algorithm, but it isn't clear how that will change as alternate storage support is added. The flag is probably pretty revlog specific, given the comments in verify.py. But there's already filelog specific stuff in there and I'm not sure what future storage will bring, so I don't want to over-engineer this. Likewise, I'm not sure that we want the verify method for each storage type to completely drive the bus when it comes to detecting renames, so I don't want to go down the rabbithole of having verifyintegrity() return metadata hints at this point. Differential Revision: https://phab.mercurial-scm.org/D7713
Sun, 22 Dec 2019 23:50:19 -0500 lfs: don't skip locally available blobs when verifying
Matt Harbison <matt_harbison@yahoo.com> [Sun, 22 Dec 2019 23:50:19 -0500] rev 44072
lfs: don't skip locally available blobs when verifying The `skipflags` config was introduced in a2ab9ebcd85b, which specifically calls out downloading and storing all blobs as potentially too expensive. But I don't see any reason to skip blobs that are already available locally. Hashing the blob is the only way to indirectly verify the rawdata content stored in the revlog. (The note in that commit about skipping renamed is still correct, but the reason given about needing fulltext isn't.) Differential Revision: https://phab.mercurial-scm.org/D7712
Fri, 20 Dec 2019 01:11:35 -0500 lfs: add a switch to `hg verify` to ignore the content of blobs
Matt Harbison <matt_harbison@yahoo.com> [Fri, 20 Dec 2019 01:11:35 -0500] rev 44071
lfs: add a switch to `hg verify` to ignore the content of blobs Trying to validate the fulltext of an external revision causes missing blobs to be downloaded and cached. Since the downloads aren't batch prefetched[1] and aren't compressed, this can be expensive both in terms of time and space. I made this a tri-state instead of a simple bool because there's an existing (undocumented) config to handle this, and it would be weird if `hg verify` were to suddenly start ignoring that config but an `hg recover` initiated verify honors it. Since this uses the same config setting, it too will skip rename verification (which requires fulltext, but not for LFS). [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116118.html Differential Revision: https://phab.mercurial-scm.org/D7708
(0) -30000 -10000 -3000 -1000 -300 -100 -50 -24 +24 +50 +100 +300 +1000 +3000 tip