mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-03 00:49:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,705 Commits 32 Branches 296 Tags 149 MiB
1494221b08
Commit Graph

483 Commits

Author SHA1 Message Date
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
Konstantin Lebedev
d8b424d123
[s3] optimization iam lookup for reducing algorithm complexity (#4857) 
optimization iam lookup for reducing algorithm complexity
https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 08:19:11 -07:00
chrislu
cf28108ea4 fix grpd dial option 2023-09-19 14:02:51 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs" 
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
 2023-09-18 16:12:50 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850) 
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2023-09-18 14:43:05 -07:00
Patrick Schmidt
98dcec0ee2 Clean up old signature hash pools 2023-09-05 10:33:27 -07:00
Patrick Schmidt
cdd817edf9 Improve S3 request signing performance 
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
 2023-09-05 10:33:27 -07:00
SmsS4
f61490966f
Add time to first byte metric for s3 (#4768) 
* Add time to first byte metric for s3

* Change second to millisecond
 2023-08-21 00:42:39 -07:00
chrislu
6792db21dc ErrInvalidPart when completed parts has extra parts 
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
 2023-08-17 00:51:42 -07:00
SmoothDenis
51bcc219ea
s3api should return 500 code from filer (#4699) 2023-07-22 07:22:38 -07:00
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows (#4666) 
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
 2023-07-11 09:05:14 -07:00
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix (#4465) 
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
 2023-05-16 09:39:43 -07:00
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion (#4430) 2023-04-25 08:31:14 -07:00
Konstantin Lebedev
5614ad0000
fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter (#4399) 
* fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter

* fix list with delimiter and start token

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-04-12 16:53:49 -07:00
Konstantin Lebedev
88b2bbadd1
try fix s3test test_bucket_listv2_delimiter_prefix (#4396) 2023-04-12 08:47:20 -07:00
Konstantin Lebedev
095cdb0c7f
fix test prefix is a directory (#4393) 
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-04-12 00:08:13 -07:00
Konstantin Lebedev
44ad07276c
s3 fix get list of dir object key with slash suffix (#4391) 
* s3 fix get list of dir object key with slash suffix
https://github.com/seaweedfs/seaweedfs/issues/3086

* list only entry dir eq prefix

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-04-11 23:52:35 -07:00
Konstantin Lebedev
130bc3e668
s3 fix get fake dir object key (#4390) 2023-04-11 07:36:22 -07:00
chrislu
6bfb91b539 update entry Attributes.Mtime = time.Now().Unix() 
fix https://github.com/seaweedfs/seaweedfs/issues/4337
 2023-03-25 22:02:57 -07:00
chrislu
5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
zemul
0bf56298d5
fix chunk.ModifiedTsNs (#4264) 
* fix

* fix mtime s > ns

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2023-03-02 08:24:36 -08:00
LHHDZ
db5515eada
Fix s3api_object_list_handlers returning contents less than the specified limit when more data actually exists (#4240) 
Fix when the stored data is actually enough but s3api_object_list_handlers returns less than the specified limit

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2023-02-21 23:08:52 -08:00
chrislu
79d68a81fd text change 2023-02-07 15:00:41 -08:00
chrislu
999a126a3f typo 2023-02-07 14:59:47 -08:00
chrislu
1c2364936a detect invalid bucket name 
fix https://github.com/seaweedfs/seaweedfs/issues/4143
 2023-02-07 14:57:29 -08:00
chrislu
5de93fe442 refactoring 2023-02-07 14:45:20 -08:00
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
Guo Lei
5f993a5f20
Fix ide warning (#4082) 
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
 2022-12-26 00:03:47 -08:00
LHHDZ
f84c13dabd
fix response not allowed (#4057) 2022-12-13 05:48:50 -08:00
LHHDZ
bb289d7d69
sort tags by key (#4047) 
Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-12-11 22:49:57 -08:00
Konstantin Lebedev
2b910d1cf8
avoid recursive deleting newly created empty directories (#4016) 2022-11-25 08:45:47 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks() 
for later locking on reading chunks
 2022-11-15 06:33:36 -08:00
LHHDZ
39f0246deb
fix 'NoSuchUpload' when upload part (#3968) 2022-11-11 04:53:53 -08:00
famosss
df0662038c
feat:add a uuid after uploadid (#3963) 2022-11-10 07:17:29 -08:00
pavelzhurov
abe4a61659
Bug fix: empty key in DeleteMultipleObjects request caused bucket delete (#3939) 2022-11-02 05:51:13 -07:00
chrislu
8b9957d461 add back "/" prefix if it is missing in object 
fix https://github.com/seaweedfs/seaweedfs/issues/3737
 2022-10-29 17:54:30 -07:00
chrislu
ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
famosss
34132b2c9f
fix: isTruncated returns problematic (#3907) 2022-10-24 21:15:37 -07:00
famosss
874fd197b5
feat: simplify a bit (#3905) 2022-10-24 17:58:26 -07:00
famosss
25e012d30b
fix: set user metadata key to lowercase (#3894) 
* fix: set user metadata key to lowercase

* feat: simplify a bit
 2022-10-24 17:29:52 -07:00
Guo Lei
97edb40275
Fix errinfo (#3893) 
* types packages is imported more than onece

* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
 2022-10-24 09:25:48 -07:00
LHHDZ
a5b867af69
don't skip empty value at AssembleEntryWithAcp (#3855) 
* add acl helper functionalities

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* add tests

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* remove 0 when create map

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* delete when empty at `AssembleEntryWithAcp`

`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp`  shouldn't skip empty value

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-13 22:50:44 -07:00
LHHDZ
bf5e45b66a
add acl helper functionalities (#3831) 2022-10-12 20:15:16 -07:00
LHHDZ
d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824) 
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-11 20:14:14 -07:00
LHHDZ
3550692afc
change s3_account.go package to avoid cycle dependency (#3813) 2022-10-10 06:44:29 -07:00
Konstantin Lebedev
ec46a34f33
avoid increasing the coordinates of metric labels (#3785) 2022-10-04 01:20:01 -07:00
LHHDZ
e9584d9661
add ownership rest apis (#3765) 2022-10-01 19:18:00 -07:00
LHHDZ
3de1e19780
s3: sync bucket info from filer (#3759) 2022-09-29 12:29:01 -07:00
LHHDZ
aacdcc4cad
s3: add account (#3753) 
associate `Account` and `Identity` by accountId
 2022-09-28 12:45:18 -07:00
chrislu
31922b2bf2 s3 to watch specific directories 2022-09-20 09:30:05 -07:00