diff tests/test-lfs-test-server.t @ 35474:16660fd4428d

test-lfs: add tests around corrupted lfs objects

These are mostly tests against file:// based remote stores, because that's what
we have the most control over.

The test uploading a corrupt blob to lfs-test-server demonstrates an overly
broad exception handler in the retry loop.  A corrupt blob is actually
transferred in a download, but eventually caught when it is accessed (only after
it leaves the corrupt file in a couple places locally).  I don't think we want
to trust random 3rd party implementations, and this would be a problem if there
were a `debuglfsdownload` command that simply cached the files.  And given the
cryptic errors, we should probably validate the file hash locally before
uploading, and also after downloading.

--- a/tests/test-lfs-test-server.t	Tue Dec 19 17:53:44 2017 -0500
+++ b/tests/test-lfs-test-server.t	Thu Nov 16 22:52:53 2017 -0500
@@ -105,6 +105,55 @@
   lfs: found 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 in the local lfs store
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
+Test a corrupt file download, but clear the cache first to force a download
+
+XXX: ideally, the validation would occur before polluting the usercache and
+local store, with a clearer error message.
+
+  $ rm -rf `hg config lfs.usercache`
+  $ cp $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 blob
+  $ echo 'damage' > $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
+  $ rm ../repo1/.hg/store/lfs/objects/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
+  $ rm ../repo1/*
+  $ hg --repo ../repo1 update -C tip -v
+  resolving manifests
+  getting a
+  lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
+  getting b
+  lfs: found 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b in the local lfs store
+  getting c
+  lfs: downloading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
+  lfs: adding d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 to the usercache
+  lfs: processed: d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
+  lfs: found d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 in the local lfs store
+  abort: integrity check failed on data/c.i:0!
+  [255]
+
+BUG: the corrupted blob was added to the usercache and local store
+
+  $ cat ../repo1/.hg/store/lfs/objects/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 | $TESTDIR/f --sha256
+  sha256=fa82ca222fc9813afad3559637960bf311170cdd80ed35287f4623eb2320a660
+  $ cat `hg config lfs.usercache`/d1/1e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 | $TESTDIR/f --sha256
+  sha256=fa82ca222fc9813afad3559637960bf311170cdd80ed35287f4623eb2320a660
+  $ cp blob $TESTTMP/lfs-content/d1/1e/1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998
+
+Test a corrupted file upload
+
+  $ echo 'another lfs blob' > b
+  $ hg ci -m 'another blob'
+  $ echo 'damage' > .hg/store/lfs/objects/e6/59058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0
+  $ hg push -v ../repo1
+  pushing to ../repo1
+  searching for changes
+  lfs: uploading e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0 (17 bytes)
+  lfs: failed: LfsRemoteError('HTTP error: HTTP Error 500: Internal Server Error (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)',) (remaining retry 5)
+  lfs: failed: LfsRemoteError('HTTP error: HTTP Error 404: Not Found (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)',) (remaining retry 4)
+  lfs: failed: LfsRemoteError('HTTP error: HTTP Error 404: Not Found (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)',) (remaining retry 3)
+  lfs: failed: LfsRemoteError('HTTP error: HTTP Error 404: Not Found (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)',) (remaining retry 2)
+  lfs: failed: LfsRemoteError('HTTP error: HTTP Error 404: Not Found (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)',) (remaining retry 1)
+  abort: HTTP error: HTTP Error 404: Not Found (oid=e659058e26b07b39d2a9c7145b3f99b41f797b6621c8076600e9cb7ee88291f0, action=upload)!
+  [255]
+
 Check error message when the remote missed a blob:
 
   $ echo FFFFF > b