Commit graph

34 commits

Author SHA1 Message Date
Jeffrey Morgan
ac5076ce1e exponential backoff up to 30s 2023-11-19 14:32:19 -05:00
Michael Yang
cb42589792 adjust download/upload parts 2023-11-19 14:32:19 -05:00
Jeffrey Morgan
e1d7056496 update progress statuses 2023-11-19 09:21:13 -05:00
Michael Yang
d6ecaa2cbf update progress responses 2023-11-17 10:06:19 -08:00
Daniel Hiltgen
cc54a416c6 Resume chunk download on UnexpectedEOF errors
If the chunk download is interrupted, resume from where we left off
2023-11-10 08:29:42 -08:00
Michael Yang
434a6f9d46 return last error 2023-11-03 16:49:51 -07:00
Michael Yang
d39709260f download with retry 2023-11-02 13:16:11 -07:00
Michael Yang
60bb3c03a1 use http.Method 2023-11-02 13:12:45 -07:00
Michael Yang
4e09aab8b9 concurrent uploads 2023-10-27 17:07:33 -07:00
Michael Yang
910816a532 fix(download): no retry when out of space 2023-10-26 11:34:07 -07:00
Michael Yang
257ffeb997 fix download 2023-10-12 12:52:43 -07:00
Michael Yang
c413a55093 download: handle inner errors 2023-10-11 14:15:30 -07:00
Michael Yang
630bb75d2a dynamically size download parts based on file size 2023-10-11 14:10:25 -07:00
Michael Yang
a2055a1e93 update download 2023-10-11 14:10:25 -07:00
Jeffrey Morgan
65dcd0ce35
always cleanup blob download (#747) 2023-10-10 13:12:29 -04:00
Michael Yang
0560b28a8d names 2023-10-06 12:56:56 -07:00
Michael Yang
10199c5987 replace done channel with file check 2023-10-06 12:56:56 -07:00
Michael Yang
288814d3e4 fix ref counts 2023-10-06 12:56:43 -07:00
Michael Yang
04733438da check head request response 2023-10-06 12:56:43 -07:00
Michael Yang
711e891f0f fix resumable downloads
glob returns files in lexical order which is not appropriate when
rebuilding the parts list
2023-10-06 12:56:43 -07:00
Michael Yang
090d08422b handle unexpected eofs 2023-10-06 12:56:43 -07:00
Michael Yang
5b84404c64 handle concurrent requests for the same blobs 2023-10-06 12:56:43 -07:00
Michael Yang
8544edca21 parallel chunked downloads 2023-10-06 12:56:43 -07:00
Michael Yang
f40b3de758 use int64 consistently 2023-09-28 11:07:24 -07:00
Michael Yang
b58d5d16b0 fix mkdir on windows 2023-09-19 09:41:13 -07:00
Michael Yang
246dc65417 loosen http status code checks 2023-08-28 18:34:53 -04:00
Michael Yang
2cc634689b use url.URL 2023-08-22 10:49:07 -07:00
Michael Yang
2c7f956b38 add version 2023-08-22 09:40:58 -07:00
Bruce MacDonald
f0d7c2f5ea retry download on network errors 2023-08-15 15:07:19 -03:00
Bruce MacDonald
f020e1d519 always remove from in progress map on download 2023-08-14 13:09:20 -03:00
Patrick Devine
d9cf18e28d
add maximum retries when pushing (#334) 2023-08-11 15:41:55 -07:00
Bruce MacDonald
8228d166ce pr comments 2023-08-09 11:31:54 -04:00
Bruce MacDonald
907e6c56b3 unlock downloadu in case or requestDownload err 2023-08-09 11:31:54 -04:00
Bruce MacDonald
868e3b31c7 allow for concurrent pulls of the same files 2023-08-09 11:31:54 -04:00