mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 13:37:49 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,169 Commits 32 Branches 296 Tags 152 MiB
edef485333
Commit Graph

67 Commits

Author SHA1 Message Date
han
65fb8fad99
Update filer_multipart.go (#6188) 
fix the #6177 bug
 2024-11-01 00:38:35 -07:00
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part (#5476) 
* try fix s3  test_multipart_get_part

* add passed s3 tests

* fix SeaweedFSUploadId

* rm spaces

* convert part request to range

* add passed s3 tests of multipart
 2024-04-14 10:41:32 -07:00
Konstantin Lebedev
3e25ed1b11
[s3] add s3 pass test_multipart_upload_size_too_small (#5475) 
* add s3 pass test_multipart_upload_size_too_small

* refactor metric names

* return ErrNoSuchUpload if empty parts

* fix test
 2024-04-07 11:52:35 -07:00
Konstantin Lebedev
35cba720a5
[s3] add s3 pass test_multipart_upload (#5474) 
add s3 pass test_multipart_upload
 2024-04-07 11:51:22 -07:00
Konstantin Lebedev
d42a04cceb
[s3] fix s3 test_multipart_resend_first_finishes_last (#5471) 
* try fix s3 test
https://github.com/seaweedfs/seaweedfs/pull/5466

* add error handler metrics

* refactor

* refactor multipartExt

* delete bad entry parts
 2024-04-06 10:56:39 -07:00
Nikita Borzykh
7aa25c1137
fix: correctly form partNumber from new entry.Name format in "listObjectParts" (#5470) 2024-04-04 09:16:16 -07:00
skycope
b19c9847c6
fix completed multiupload lost data (#5460) 
If there are putObjectPart requests with the same uploadId during
completeMultiPart, it can result in data loss. putObjectPart requests
might be due to timeout retries.

Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-04-02 20:18:42 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"" 
This reverts commit 8cb42c39
 2023-09-25 09:35:16 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs" 
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
 2023-09-18 16:12:50 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850) 
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2023-09-18 14:43:05 -07:00
chrislu
6792db21dc ErrInvalidPart when completed parts has extra parts 
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
 2023-08-17 00:51:42 -07:00
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows (#4666) 
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
 2023-07-11 09:05:14 -07:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks() 
for later locking on reading chunks
 2022-11-15 06:33:36 -08:00
famosss
df0662038c
feat:add a uuid after uploadid (#3963) 2022-11-10 07:17:29 -08:00
chrislu
8b9957d461 add back "/" prefix if it is missing in object 
fix https://github.com/seaweedfs/seaweedfs/issues/3737
 2022-10-29 17:54:30 -07:00
chrislu
ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
ImFantuan
b64411bda8
fix 0 size (#3683) 2022-09-15 01:27:02 -07:00
famosss
bb880d6253
IsTruncated returned by the ListMultipartUploads method is set to false by default (#3682) 
* simplify a bit

* fix: IsTruncated returned by the ListMultipartUploads method is set to false by default
 2022-09-15 00:27:15 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
f17cd0d5cd return false if not found 
fix https://github.com/chrislusf/seaweedfs/issues/3011
 2022-05-03 07:18:34 -07:00
Konstantin Lebedev
306cf70c4a avoid empty listMultipartUploads response 2022-04-27 19:27:44 +05:00
leyou240
89eb87c1d1
Merge branch 'master' into slices.SortFunc 2022-04-18 10:39:29 +08:00
justin
3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
guol-fnst
180aa88a92 check uploadid using object name hash string 2022-04-12 11:04:38 +08:00
Konstantin Lebedev
993283bb1d revert sort.Search 2022-03-29 19:53:12 +05:00
Konstantin Lebedev
5b90a39954 fix s3 tests: 
multipart_upload_incorrect_etag
multipart_resend_first_finishes_last
 2022-03-29 19:21:09 +05:00
chrislu
e48764be75 s3: multipart upload verifies uploaded parts 2022-03-23 01:05:14 -07:00
chrislu
61811dc2f1 comments 2022-02-18 22:14:40 -08:00
chrislu
9014d00fd0 Revert "s3: listObjectParts return ErrNoSuchUpload if does not exist" 
This reverts commit 6cf2e7d493.
 2022-02-18 20:54:54 -08:00
chrislu
6cf2e7d493 s3: listObjectParts return ErrNoSuchUpload if does not exist 
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api abort-multipart-upload --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api list-parts --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
{
    "Initiator": null,
    "Owner": null,
    "StorageClass": "STANDARD"
}

If we abort a multipart upload, it appears that records are left behind. We should get a 404 NoSuchKey error.
 2022-02-03 12:34:16 -08:00
chrislu
5c3b783310 s3: copy object API needs to escape special characters 
fix https://github.com/chrislusf/seaweedfs/issues/2482
 2021-12-04 23:24:53 -08:00
Chris Lu
309f46e3af S3: fix upload limit if the size is more than 8GB 2021-10-18 14:14:18 -07:00
Konstantin Lebedev
8de7915950 s3 mime detect 2021-10-14 15:03:11 +05:00
Konstantin Lebedev
458145425e S3 MultipartUpload pass contentType to meta 2021-10-12 17:14:54 +05:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
7359193e97 go fmt 2021-07-21 14:38:12 -07:00
lyg
1a4db87e19 fix s3 metadata error with multipart upload 2021-07-02 11:00:42 +08:00
Chris Lu
c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu
b9c1f3e9de s3: fixes for list multipart upload 2020-09-21 10:51:24 -07:00
Chris Lu
5b40a2690a refactoring 2020-09-19 14:09:58 -07:00
Chris Lu
3984c3962f add comment 2020-09-11 15:07:19 -07:00
Chris Lu
baa6bdf4d4 s3: listMultipartUploads fix output format 2020-09-11 15:04:01 -07:00
Chris Lu
3eda8d6dfc s3: ListParts output xml format 
fix https://github.com/chrislusf/seaweedfs/issues/1461
 2020-09-11 14:53:50 -07:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
83012f543a s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file 2020-08-26 08:50:24 -07:00
Chris Lu
c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu
a25a8d8822 s3: print out time in UTC format 
fix https://github.com/chrislusf/seaweedfs/issues/1297
 2020-04-29 21:35:24 -07:00
Chris Lu
d57559d3f7 chunk rememober encryption or not 2020-04-29 02:41:47 -07:00
Chris Lu
ec2eb8bc48 add If-None-Match and If-Modified-Since 
fix https://github.com/chrislusf/seaweedfs/issues/1269
 2020-04-08 08:12:00 -07:00
Chris Lu
c07bcd5065 refactoring 2020-03-23 00:30:02 -07:00