Commit Graph

49 Commits

Author SHA1 Message Date
skycope
b19c9847c6
fix completed multiupload lost data (#5460)
If there are putObjectPart requests with the same uploadId during
completeMultiPart, it can result in data loss. putObjectPart requests
might be due to timeout retries.

Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-04-02 20:18:42 -07:00
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix (#4465)
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
2023-05-16 09:39:43 -07:00
LHHDZ
f84c13dabd
fix response not allowed (#4057) 2022-12-13 05:48:50 -08:00
LHHDZ
39f0246deb
fix 'NoSuchUpload' when upload part (#3968) 2022-11-11 04:53:53 -08:00
famosss
df0662038c
feat:add a uuid after uploadid (#3963) 2022-11-10 07:17:29 -08:00
chrislu
21c0587900 go fmt 2022-09-14 23:06:44 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev
bcbdc4cb37 use const multipart uploads folder
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
石昌林
15bfc60a88 Remove the collection query param of s3api and let the collection be determined by the filer 2022-06-27 13:47:53 +08:00
guol-fnst
1d77deccd0 AbortMultipartUploadHandler should return 204 instead of 200
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
2022-06-20 13:26:22 +08:00
chrislu
27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu
596c3860ca use final destination to resolve fs configuration
related to https://github.com/chrislusf/seaweedfs/issues/3075
2022-05-30 22:47:26 -07:00
chrislu
00c1dfec4f go fmt 2022-05-01 23:16:29 -07:00
guol-fnst
44d810d163 rename functions and remove uncessary check 2022-04-12 13:40:53 +08:00
guol-fnst
180aa88a92 check uploadid using object name hash string 2022-04-12 11:04:38 +08:00
guol-fnst
2232cfb5b7 Check object name and uploadID when processing multipart uploading 2022-04-11 19:53:44 +08:00
chrislu
e48764be75 s3: multipart upload verifies uploaded parts 2022-03-23 01:05:14 -07:00
chrislu
81f86c381d s3: avoid nil response
fix https://github.com/chrislusf/seaweedfs/issues/2636
2022-02-07 11:37:59 -08:00
Konstantin Lebedev
4ec8715f20 audit log 2021-12-07 12:15:48 +05:00
Chris Lu
f17fa400d5 refactoring 2021-10-31 18:05:34 -07:00
Chris Lu
006b4ec0f8 refactoring 2021-10-31 18:02:08 -07:00
Chris Lu
54b6e0f3fd adjust logs 2021-10-27 23:46:07 -07:00
Konstantin Lebedev
8de7915950 s3 mime detect 2021-10-14 15:03:11 +05:00
Konstantin Lebedev
458145425e S3 MultipartUpload pass contentType to meta 2021-10-12 17:14:54 +05:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
f0042f62dd readable logs 2021-07-15 12:17:48 -07:00
Chris Lu
c6d4c16079 S3: add metadata with multipart upload
fix https://github.com/chrislusf/seaweedfs/issues/2173
2021-07-01 19:12:11 -07:00
Chris Lu
8b382a8209 refactor 2021-06-10 21:50:21 -07:00
Chris Lu
33b87244ef refactoring 2021-06-10 16:54:36 -07:00
Chris Lu
86cdb2a3e5 adjust logs 2020-11-11 02:01:24 -08:00
Chris Lu
b9c1f3e9de s3: fixes for list multipart upload 2020-09-21 10:51:24 -07:00
Chris Lu
5b40a2690a refactoring 2020-09-19 14:09:58 -07:00
Chris Lu
828a5ae429 check signature only when auth is enabled 2020-08-08 09:11:40 -07:00
Chris Lu
54cd582653 refactor 2020-07-26 12:58:58 -07:00
Chris Lu
c50ec1599c auth for presigned put object part 2020-07-26 12:09:50 -07:00
Chris Lu
97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu
986d63cd5e s3: increase list objects or parts limit from 1000 to 10000
10x of Amazon S3 limits
2020-02-25 11:45:40 -08:00
Chris Lu
0a57db7026
Merge pull request #1200 from chrislusf/add_s3_acl
Add s3 ACL
2020-02-17 12:53:30 -08:00
Chris Lu
cf5064d702 properly close http response 2020-02-14 09:09:15 -08:00
Chris Lu
f3ce3166ad add streaming v4 2020-02-09 17:42:17 -08:00
Chris Lu
62843991f2 s3: adjust object key 2019-07-08 12:37:20 -07:00
Chris Lu
c5a3ff1c35 weed s3: multipart upload goes to the right bucket
fix https://github.com/chrislusf/seaweedfs/issues/908
2019-03-31 11:10:19 -07:00
Chris Lu
55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu
d9b32db8d7 adjust uploads to a hidden folder 2018-09-19 22:03:16 -07:00
Chris Lu
f6d8525d1d working S3 multipart uploads 2018-09-12 00:46:12 -07:00
Chris Lu
296acc4a0a go fmt 2018-09-09 16:26:11 -07:00
Chris Lu
164091c269 add s3 multipart upload 2018-09-09 16:25:43 -07:00
Chris Lu
ce6a2140a2 preparing to support S3 multipart uploads 2018-09-04 00:42:44 -07:00
Chris Lu
0b0ece9649 add place holders for multiplarts upload 2018-09-03 11:38:10 -07:00