Lapshinn Vitaly
6bdc274d4d
add s3api error for copy in file, not directory
2022-02-04 03:28:37 +03:00
chrislu
6cf2e7d493
s3: listObjectParts return ErrNoSuchUpload if does not exist
...
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api abort-multipart-upload --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
ubuntu@prod-master-1:~$ aws --endpoint http://10.244.15.66:8333 s3api list-parts --bucket prod-cache --key multipart-test --upload-id 5347f936-6adc-43de-8e5c-1fd137c3b2bc
{
"Initiator": null,
"Owner": null,
"StorageClass": "STANDARD"
}
If we abort a multipart upload, it appears that records are left behind. We should get a 404 NoSuchKey error.
2022-02-03 12:34:16 -08:00
zerospiel
b54a65ba5a
weed/s3api: added new bucket handlers for more compatibility with AWS S3
...
Protocol
Otherwise any requests to the underlying handlers results in calls to
ListObjects (v1) that may intensively load gateway and volume servers.
Added the following handlers with default responses:
- GetBucketLocation
- GetBucketRequestPayment
Added the following handlers with NotFound and NotImplemented responses:
- PutBucketAcl
- GetBucketPolicy
- PutBucketPolicy
- DeleteBucketPolicy
- GetBucketCors
- PutBucketCors
- DeleteBucketCors
2022-02-03 17:17:05 +03:00
chrislu
77362700e1
S3: fail fast when "X-Amz-Copy-Source" is a folder
...
fix #2593
2022-01-18 12:04:40 -08:00
chrislu
c87b8f4c30
S3: fail fast when "X-Amz-Copy-Source" is a folder
...
fix https://github.com/chrislusf/seaweedfs/issues/2593
2022-01-17 23:09:37 -08:00
Konstantin Lebedev
edb753ab4d
https://github.com/chrislusf/seaweedfs/issues/2583
2022-01-12 16:04:59 +05:00
chrislu
e76105e2ab
fix auth permission checking
2022-01-03 21:05:20 -08:00
chrislu
a7887166cf
wildcard prefix to restrict access to directories in s3 bucket
...
https://github.com/chrislusf/seaweedfs/discussions/2551
2022-01-03 15:39:36 -08:00
Chris Lu
42c849e0df
Merge branch 'master' into metadata_follow_with_client_id
2022-01-02 01:07:30 -08:00
Chris Lu
9b94177380
Merge pull request #2543 from skurfuerst/seaweedfs-158
...
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
2022-01-01 22:34:13 -08:00
Sebastian Kurfuerst
c35660175d
BUGFIX: ensure Authorization header is only added once
2021-12-31 22:06:18 +01:00
Sebastian Kurfuerst
10404c4275
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
...
- one JWT for reading and one for writing, analogous to how the JWT
between Master and Volume Server works
- I did not implement IP `whiteList` parameter on the filer
Additionally, because http_util.DownloadFile now sets the JWT,
the `download` command should now work when `jwt.signing.read` is
configured. By looking at the code, I think this case did not work
before.
## Docs to be adjusted after a release
Page `Amazon-S3-API`:
```
# Authentication with Filer
You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as
explained in [Security-Configuration](Security-Configuration) -
controlled by the `grpc.*` configuration in `security.toml`.
Starting with version XX, it is also possible to authenticate the HTTP
operations between the S3-API-Proxy and the Filer (especially
uploading new files). This is configured by setting
`filer_jwt.signing.key` and `filer_jwt.signing.read.key` in
`security.toml`.
With both configurations (gRPC and JWT), it is possible to have Filer
and S3 communicate in fully authenticated fashion; so Filer will reject
any unauthenticated communication.
```
Page `Security Overview`:
```
The following items are not covered, yet:
- master server http REST services
Starting with version XX, the Filer HTTP REST services can be secured
with a JWT, by setting `filer_jwt.signing.key` and
`filer_jwt.signing.read.key` in `security.toml`.
...
Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer.
Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).**
...
# Securing Filer HTTP with JWT
To enable JWT-based access control for the Filer,
1. generate `security.toml` file by `weed scaffold -config=security`
2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string
3. copy the same `security.toml` file to the filers and all S3 proxies.
If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`.
If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`.
The S3 API Gateway reads the above JWT keys and sends authenticated
HTTP requests to the filer.
```
Page `Security Configuration`:
```
(update scaffold file)
...
[filer_jwt.signing]
key = "blahblahblahblah"
[filer_jwt.signing.read]
key = "blahblahblahblah"
```
Resolves : #158
2021-12-30 14:45:27 +01:00
chrislu
5c87fcc6d2
add client id for all metadata listening clients
2021-12-30 00:23:57 -08:00
chrislu
5788bf2270
s3: increase timeout limit
...
https://github.com/chrislusf/seaweedfs/issues/2541
2021-12-29 22:21:02 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
chrislu
7210558c7b
s3: pass through s3 presigned headers
...
fix https://github.com/chrislusf/seaweedfs/discussions/2502
2021-12-15 13:18:53 -08:00
Konstantin Lebedev
969f513265
disable audit log for s3 statusHandler
2021-12-13 16:08:19 +05:00
Konstantin Lebedev
34779e8f38
force enable asynchronous I/O sending events to Fluentd
2021-12-13 13:39:39 +05:00
kmlebedev
4f98553ba9
audit log SignatureVersion
2021-12-10 19:40:32 +05:00
Konstantin Lebedev
98251fe16a
non blocking audit log
2021-12-09 19:47:16 +05:00
Chris Lu
2ba08afed1
Merge pull request #2498 from kmlebedev/s3_audit_log
2021-12-07 09:35:48 -08:00
Konstantin Lebedev
10678cde81
audit log config
2021-12-07 18:20:52 +05:00
Konstantin Lebedev
4ec8715f20
audit log
2021-12-07 12:15:48 +05:00
Konstantin Lebedev
3ac48cd540
audit log
2021-12-07 12:13:21 +05:00
chrislu
5c3b783310
s3: copy object API needs to escape special characters
...
fix https://github.com/chrislusf/seaweedfs/issues/2482
2021-12-04 23:24:53 -08:00
Konstantin Lebedev
40abae7caa
Avoid forbiddening for HeadBucketHandler if owners are specified
...
https://github.com/chrislusf/seaweedfs/issues/2434
2021-11-17 14:42:24 +05:00
Konstantin Lebedev
42f4aa824b
force overwrite s3-is-admin
...
https://github.com/chrislusf/seaweedfs/issues/2433
2021-11-11 11:34:01 +05:00
Bl1tz23
c683409e92
s3: add RWMutex to iam, use RLock for concurrent reading
2021-11-09 18:11:06 +03:00
Bl1tz23
5018b22f36
s3: fix potencial iam identities data race
2021-11-09 12:19:50 +03:00
Chris Lu
59d1435d65
s3: avoid possible attacks by version
2021-11-08 17:47:56 -08:00
Chris Lu
244841f080
adjust logs
2021-11-07 13:27:57 -08:00
Chris Lu
6e999f1176
s3: fix ListObject if more than 10000 objects
2021-11-07 12:39:36 -08:00
Chris Lu
55973e8572
log unknown access key
2021-11-07 12:37:46 -08:00
Chris Lu
35c37562bc
check auth only when enabled
2021-11-03 12:11:36 -07:00
Chris Lu
e1ab8b01d0
s3: adjust permission for HEAD bucket operation
...
fix https://github.com/chrislusf/seaweedfs/issues/2417#issuecomment-958391856
2021-11-02 16:28:55 -07:00
Chris Lu
b25661c6df
s3: skip permission checking for creating bucket if the bucket already exists
...
fix https://github.com/chrislusf/seaweedfs/issues/2417
Rclone was trying to create the bucket even though the bucket already exists.
2021-11-02 13:13:36 -07:00
Chris Lu
9cf756b188
S3: support CORS
...
related to https://github.com/chrislusf/seaweedfs/issues/1271
2021-10-31 18:06:43 -07:00
Chris Lu
f17fa400d5
refactoring
2021-10-31 18:05:34 -07:00
Chris Lu
006b4ec0f8
refactoring
2021-10-31 18:02:08 -07:00
Konstantin Lebedev
dc3ae5e8a6
DeleteBucketLifecycleConfiguration return 204
2021-10-28 19:34:37 +05:00
Konstantin Lebedev
2afb5a13af
fix DeleteBucketLifecycleConfiguration
2021-10-28 18:30:33 +05:00
Chris Lu
54b6e0f3fd
adjust logs
2021-10-27 23:46:07 -07:00
Konstantin Lebedev
235329a92a
fix Tagging add xmlns
2021-10-20 19:40:22 +05:00
Konstantin Lebedev
dc7e525cb9
fix Tagging test
2021-10-20 19:12:00 +05:00
Konstantin Lebedev
0b834600d5
fix PutObjectTaggingHandler Unmarshal
2021-10-20 17:58:06 +05:00
Konstantin Lebedev
c78220a7f2
fix object tagging https://github.com/chrislusf/seaweedfs/issues/2389
2021-10-20 16:01:06 +05:00
Chris Lu
309f46e3af
S3: fix upload limit if the size is more than 8GB
2021-10-18 14:14:18 -07:00
Chris Lu
a6bb509cb7
adjust mime detection
2021-10-18 10:47:39 -07:00
Konstantin Lebedev
dd2cc1acaf
fix https://github.com/chrislusf/seaweedfs/issues/2387
2021-10-18 16:27:57 +05:00
Chris Lu
004e56c1a6
save updated lastTsNs
2021-10-17 02:01:20 -07:00
Chris Lu
71a8c80f6f
s3: continuously listen to file updates
...
fix https://github.com/chrislusf/seaweedfs/issues/2342
2021-10-17 01:04:19 -07:00
Konstantin Lebedev
8de7915950
s3 mime detect
2021-10-14 15:03:11 +05:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
b693a8d6b9
Merge pull request #2380 from kmlebedev/bucket_policy
...
Bucket policy
2021-10-13 15:29:53 -07:00
Konstantin Lebedev
9d6ffa0ea1
GetBucketLifecycleConfigurationHandler
2021-10-14 01:35:33 +05:00
Konstantin Lebedev
458145425e
S3 MultipartUpload pass contentType to meta
2021-10-12 17:14:54 +05:00
Konstantin Lebedev
be4b3ed509
AclHandlers
2021-10-11 15:03:56 +05:00
Chris Lu
84fdda85e7
go fmt
2021-09-19 12:06:15 -07:00
Chris Lu
5abdc0be77
s3: avoid overwriting object with ACL/LegalHold/Retension/LockConfiguration requests
2021-09-19 03:24:47 -07:00
Chris Lu
59dd271734
more glog
2021-09-19 00:29:51 -07:00
Chris Lu
4cbba2b1c6
add more glog to s3
2021-09-19 00:28:22 -07:00
Chris Lu
71175461ef
add glog for s3 handlers
2021-09-19 00:18:59 -07:00
Chris Lu
e5fc35ed0c
change server address from string to a type
2021-09-12 22:47:52 -07:00
Chris Lu
f62df7f695
fix wrong srcPath checking
2021-09-05 23:37:40 -07:00
Chris Lu
491f7636f8
s3: CopyObject return http Status 400 Bad Request for non-existing source
...
fix https://github.com/chrislusf/seaweedfs/issues/2306
2021-09-03 17:38:39 -07:00
Chris Lu
5a0f92423e
use grpc and jwt
2021-08-12 21:40:33 -07:00
Bl1tz23
1c94b3d013
merge master, resolve conflicts
2021-08-10 13:45:24 +03:00
Bl1tz23
e6e57db530
Add liveness\readiness probe for s3 api handler on /status path
2021-08-10 13:42:46 +03:00
Chris Lu
6b743dbbf9
refactor client subscribe metadata
2021-08-04 16:25:46 -07:00
Chris Lu
7359193e97
go fmt
2021-07-21 14:38:12 -07:00
Chris Lu
f0042f62dd
readable logs
2021-07-15 12:17:48 -07:00
Chris Lu
ecce300964
s3 config read via grpc
2021-07-09 02:48:03 -07:00
Chris Lu
a024254ad7
logging
2021-07-03 14:51:01 -07:00
Chris Lu
d39b2689a5
S3 authorization: StreamingSigned enforces access control
...
fix https://github.com/chrislusf/seaweedfs/issues/2180
2021-07-03 14:50:53 -07:00
lyg
1a4db87e19
fix s3 metadata error with multipart upload
2021-07-02 11:00:42 +08:00
Chris Lu
c6d4c16079
S3: add metadata with multipart upload
...
fix https://github.com/chrislusf/seaweedfs/issues/2173
2021-07-01 19:12:11 -07:00
Chris Lu
b71c3cfba4
avoid possible empty object
2021-06-10 22:17:53 -07:00
Chris Lu
310e31424e
adjust the error output
...
fix https://github.com/chrislusf/seaweedfs/issues/2123
2021-06-10 21:55:13 -07:00
Chris Lu
8b382a8209
refactor
2021-06-10 21:50:21 -07:00
Chris Lu
33b87244ef
refactoring
2021-06-10 16:54:36 -07:00
Chris Lu
2b60e2abb1
only disallow streaming signed when no auth enabled
...
fix https://github.com/chrislusf/seaweedfs/issues/2101
2021-05-31 01:03:04 -07:00
Konstantin Lebedev
b612d5aebd
s3 test get w/ If-Match: bogus ETag
2021-05-24 16:59:44 +05:00
Konstantin Lebedev
69c768870b
- object write cache control
...
- object write expires
2021-05-24 15:43:55 +05:00
Konstantin Lebedev
84dce32a57
Merge branch 'master' into head_check_all_chunks
2021-05-24 12:28:19 +05:00
Chris Lu
431684798b
s3: add errors if requests are signed by no authentication is setup
...
fix https://github.com/chrislusf/seaweedfs/issues/2075
2021-05-21 14:08:47 -07:00
Konstantin Lebedev
03d1199d5f
Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader"
...
This reverts commit ac71117e
2021-05-20 11:45:21 +05:00
Chris Lu
26a55bbb5c
Adjust error message when bucket name conflicts with existing collections
...
fix https://github.com/chrislusf/seaweedfs/issues/2069
2021-05-12 22:30:39 -07:00
Chris Lu
ac71117ee6
revert PR #1903 avoid http error: superfluous response.WriteHeader
2021-05-05 15:11:39 -07:00
Chris Lu
e87e6ef33c
s3: return 404 if bucket does not exist
...
fix https://github.com/chrislusf/seaweedfs/issues/2039
2021-05-02 21:30:37 -07:00
Konstantin Lebedev
a48785c7df
auth use bucket wild cards
2021-04-27 21:45:40 +05:00
Chris Lu
f5de42fae3
Merge pull request #1975 from kmlebedev/iam_handlers
...
IAM handlers
2021-04-12 12:07:45 -07:00
Merlin Gaillard
4d4acc715e
s3api: handle 304 response code from filer
2021-04-09 12:13:19 +02:00
Konstantin Lebedev
ba175f81b5
add auth aws signV4
2021-04-08 17:40:47 +05:00
Chris Lu
b1a86cf808
s3: copy object to itself
...
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Konstantin Lebedev
bf94149920
add line
2021-03-17 23:41:34 +05:00
Konstantin Lebedev
c5705e7a35
dir + slash key does not exist
...
https://github.com/chrislusf/seaweedfs/issues/1917
2021-03-17 23:40:42 +05:00
Konstantin Lebedev
eb54993a4e
Merge branch 'upstreamMaster' into check_chunkviews_mr
...
# Conflicts:
# weed/filer/filechunk_manifest.go
# weed/filer/stream.go
# weed/replication/repl_util/replication_util.go
# weed/util/fasthttp_util.go
2021-03-16 15:29:49 +05:00
wuh-fnst
050a7f9599
s3: count correction for the number of files in the dir
2021-03-16 09:22:03 +08:00
Konstantin Lebedev
3a3699867b
Status PartialContent for Content-Range response
2021-03-15 23:30:22 +05:00
Chris Lu
d084334ffd
Merge pull request #1883 from kmlebedev/passDelNonEmptyFolder
...
s3 delete-objects pass OK for fail to delete non-empty folder
2021-03-11 11:01:44 -08:00
Chris Lu
79280e1513
escape special characters
...
fix https://github.com/chrislusf/seaweedfs/issues/1884
2021-03-11 09:49:40 -08:00
wuheng
828fbf3fb7
s3: "isLast" returns true when the file does not exist
2021-03-11 15:20:50 +08:00
Chris Lu
ca546f47e1
s3: escape object key if containing special characters
...
fix https://github.com/chrislusf/seaweedfs/issues/1884
2021-03-10 13:19:28 -08:00
Konstantin Lebedev
bb3af2d70b
no directoriesWithDeletion
2021-03-10 19:52:41 +05:00
Konstantin Lebedev
15b018da34
s3 delete-objects pass OK for fail to delete non-empty folder
2021-03-10 19:41:35 +05:00
Chris Lu
387c6f4218
Revert "delete the folder object in multi_object_delete"
...
This reverts commit 2e89c8c9ae
.
2021-03-09 10:07:27 -08:00
Chris Lu
2e89c8c9ae
delete the folder object in multi_object_delete
...
fix https://github.com/chrislusf/seaweedfs/issues/1857
2021-03-04 14:03:40 -08:00
Chris Lu
5511722420
s3: list permission is needed to list my buckets
...
fix https://github.com/chrislusf/seaweedfs/issues/1837
2021-03-01 12:41:55 -08:00
bingoohuang
352ac2f271
Merge remote-tracking branch 'origin/master'
2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34
use backticks instead of double quotes to avoid escaped additionally in regex
2021-02-18 14:05:28 +08:00
Chris Lu
8b169aa47c
s3: Critical - S3 Delete Multiple Objects, delete the bucket
...
fix https://github.com/chrislusf/seaweedfs/issues/1806
2021-02-15 13:38:29 -08:00
Chris Lu
487e435679
adjust http max idle connections per host
...
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
Chris Lu
a0e84c4fbc
go fmt
2021-02-10 23:41:05 -08:00
Chris Lu
5353e38469
S3: should return 204 on DELETE to nonexistent file
...
fix https://github.com/chrislusf/seaweedfs/issues/1776
2021-02-03 00:35:44 -08:00
Chris Lu
609daaf387
s3: DeleteMultipleObjectsHandler clean up leftover empty folders
...
fix https://github.com/chrislusf/seaweedfs/issues/1772
2021-02-01 10:49:17 -08:00
Chris Lu
990fa69bfe
add back AdjustedUrl() related code
2021-01-28 14:36:29 -08:00
Chris Lu
9292796ec2
fmt
2021-01-28 14:29:14 -08:00
Chris Lu
c2bf1a88ac
delete from the deepest directory first when checking empty folders
2021-01-28 14:28:40 -08:00
Chris Lu
e9d8201925
s3: batch purge empty folders
2021-01-28 13:20:06 -08:00
Chris Lu
00707ec00f
mount: outsideContainerClusterMode proxy through filer
...
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
6ca10725b8
Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
...
This reverts commit 096e088d7b
.
2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b
mount: when outside cluster network, use filer as proxy to access volume servers
2021-01-24 01:41:38 -08:00
Chris Lu
ccbdb38c89
s3: fix for listing objects if more than 1000 in the folder
...
many existing tools have max key set to 1000
2021-01-24 00:27:52 -08:00
Chris Lu
389426bbb7
s3: listing may repeat on the edge
2021-01-17 22:59:31 -08:00
Chris Lu
8a7302ce37
s3: KeyCount is zero
...
fix https://github.com/chrislusf/seaweedfs/issues/1741
2021-01-11 03:50:45 -08:00
Chris Lu
e327385644
go fmt
2021-01-06 04:22:00 -08:00
Chris Lu
6d2c979999
s3: fix regression
...
fix https://github.com/chrislusf/seaweedfs/issues/1707
2020-12-27 21:09:45 -08:00
Chris Lu
561a2ff0bc
s3: add option for "alllowEmptyFolder"
2020-12-27 10:50:27 -08:00
Chris Lu
aa020ee7e7
s3: restore V2 auth type
...
The added headers were included in calculating the signatures, failing the comparison
2020-12-26 22:01:16 -08:00
Chris Lu
090f85be4b
s3: support config action Admin:bucket
2020-12-25 00:38:56 -08:00
Chris Lu
8e48a235e2
s3: avoid looping if the directory is empty
...
fix https://github.com/chrislusf/seaweedfs/issues/1701
2020-12-24 11:34:52 -08:00
Chris Lu
1620de08ae
added more logs
2020-12-24 01:52:06 -08:00
Chris Lu
a09cd28986
add debug messages
2020-12-24 01:42:15 -08:00
Chris Lu
3be3635799
just log error when checking empty dir failed
2020-12-23 14:36:48 -08:00
Chris Lu
0ca9d89589
s3: break loop if error
...
fix #1701
2020-12-23 14:34:59 -08:00
Chris Lu
4f31c1bb94
go fmt
2020-12-22 02:34:08 -08:00
Chris Lu
5c465293e9
correctly determine whether a folder is empty
...
avoid edge cases that deleting the folder if previous 32 directories are all empty
early terminate if one file is found
2020-12-12 16:19:29 -08:00
Chris Lu
f930c713fc
more efficient recursion
2020-12-12 13:25:19 -08:00
Chris Lu
03637d6f57
s3: move "delete-directory-if-empty" to read time
...
move "delete-directory-if-empty" to read time instead of entry deletion time
the listing speed for a s3 bucket folder will slow down if it has many sub folders
related to 0d345ac97d
fix https://github.com/chrislusf/seaweedfs/issues/1647
fix https://github.com/chrislusf/seaweedfs/issues/1670
2020-12-12 03:38:34 -08:00
Chris Lu
26731694f8
s3: use static configuration by default
...
So that users can still use the previous configuration files.
If leave it empty, s3 will try to use the version from filer
2020-12-10 00:59:04 -08:00
Chris Lu
765b3ef1be
save /etc/iam/identity.json inside filer store
2020-12-10 00:15:22 -08:00
Chris Lu
02a160c3fd
s3: recursively iterate all sub folders
...
fix https://github.com/chrislusf/seaweedfs/issues/1656
2020-12-09 16:47:34 -08:00
Chris Lu
fce8803087
break import cycle
2020-12-07 00:29:17 -08:00
Chris Lu
eed87791b7
s3: subscribe to s3.configure changes
2020-12-07 00:10:29 -08:00
Chris Lu
01e2da5782
refactoring
2020-12-06 23:16:20 -08:00
Konstantin Lebedev
a26f1b2040
new pkg s3iam
2020-11-26 01:30:11 +05:00
Konstantin Lebedev
6206737df2
s3 configure
2020-11-25 21:02:31 +05:00
Konstantin Lebedev
27e73de797
Merge branch 'upstream_master' into store_s3cred
...
# Conflicts:
# weed/s3api/filer_util.go
2020-11-19 18:16:44 +05:00
Chris Lu
781585b195
remove unnecessary checking
2020-11-17 17:00:06 -08:00
Chris Lu
7c9e592c2d
s3: avoid nil resp when having error
...
fix https://github.com/chrislusf/seaweedfs/issues/1622
2020-11-17 11:23:17 -08:00
Chris Lu
0ea5c087ce
go fmt
2020-11-15 16:59:28 -08:00
ruitao.liu
a9990a1dc6
adjust check bucket if exist or has access.
2020-11-13 17:13:20 +08:00
Chris Lu
e6333da65a
enable admin to access all buckets
2020-11-12 13:57:54 -08:00
Chris Lu
7094492428
fix ErrNotFound
2020-11-12 13:30:46 -08:00
Chris Lu
c6a0704019
rename function
2020-11-12 13:30:08 -08:00
Chris Lu
ed3b0f81c6
Merge pull request #1613 from taozix/master
...
check permission for bucket delete/head.
2020-11-12 13:25:28 -08:00
ruitao.liu
c4f0fd6e1b
skip if entry.Extended map is nil.
2020-11-12 17:59:31 +08:00
ruitao.liu
ab966410d2
return NoSuchBucket instead of InternalError delete non-existed bucket.
2020-11-12 16:44:16 +08:00
ruitao.liu
e06676f007
check permission for bucket delete/head.
2020-11-12 16:15:59 +08:00
Chris Lu
0fbc27b640
Merge pull request #1612 from taozix/master
...
add owner attr for bucket.
2020-11-11 22:19:47 -08:00
ruitao.liu
d7cc0498e0
check if bucket already exists.
2020-11-12 14:11:03 +08:00
ruitao.liu
dbba8cb57e
fix if nil map in entry.
2020-11-12 11:50:19 +08:00
Chris Lu
86cdb2a3e5
adjust logs
2020-11-11 02:01:24 -08:00
ruitao.liu
5b636b3242
Add bucket owner attr.
2020-11-11 16:20:59 +08:00
Konstantin Lebedev
6cd6ff6962
avoid blank response for 404
2020-11-04 22:54:47 +05:00
Konstantin Lebedev
e1190b3224
load S3 config from filer
...
https://github.com/chrislusf/seaweedfs/issues/1500
2020-11-03 21:45:56 +05:00
ruitao.liu
22a9ea0512
adjust s3 header file.
2020-10-29 16:05:40 +08:00
ruitao.liu
b917be7955
S3 bucket list, response with uploaded storageclass.
2020-10-29 14:57:19 +08:00
Chris Lu
6da87720eb
shorter tag prefix
2020-10-27 15:33:26 -07:00
ruitao.liu
8766ca1b95
rename s3 tag prefix.
2020-10-27 17:33:24 +08:00
ruitao.liu
19026ae55d
return x-amz-tag-count header when GET object.
2020-10-27 16:49:31 +08:00
Chris Lu
da36abf033
go fmt
2020-10-24 20:12:04 -07:00
Konstantin Lebedev
05b5f12f2e
multiplate DomainNames through comma
2020-10-21 20:48:51 +05:00
Chris Lu
b3aa2fab9a
s3: report error to s3 when updating an object but it is already a directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1545
2020-10-20 10:25:16 -07:00
Chris Lu
ace0ea3d28
s3: avoid duplicated bucket
2020-10-15 10:52:20 -07:00
Chris Lu
723ae11db4
refactoring in order to adjust volume server url later
2020-10-11 20:15:10 -07:00
Chris Lu
9e7a2772b1
s3: only admin can list all buckets
2020-10-08 10:12:09 -07:00
Chris Lu
62d3d3aea0
go fmt
2020-10-07 23:30:54 -07:00
Chris Lu
e91b9c85a2
s3: Added support for "List" action in weed s3 -config=... in the config file.
...
fix https://github.com/chrislusf/seaweedfs/issues/1511
2020-10-07 23:22:35 -07:00
Chris Lu
f781cce500
s3: support object tagging
...
* GetObjectTagging
* PutObjectTagging
* DeleteObjectTagging
2020-10-02 22:21:51 -07:00
Konstantin Lebedev
68463e92c1
add status code in S3RequestCounter
2020-10-01 00:59:39 +05:00
limd
402aef8f30
s3:
...
1.fix spark reading S3 directory wildcard problem
2.fix the problem of the spark history service writing S3 directory
reference git revsion number: b41b7ea4d0
2020-09-25 14:37:02 +08:00
Chris Lu
043b063136
Merge pull request #1482 from hilimd/master
...
Fix: s3 delete object
2020-09-24 18:21:34 -07:00
Chris Lu
dbf5327b98
s3: handle response-content-encoding
...
fix https://github.com/chrislusf/seaweedfs/issues/1487
2020-09-24 18:09:52 -07:00
limd
d506080c36
rollback
2020-09-23 14:29:53 +08:00
limd
8f9f29b773
fixed the problem of empty directory when S3 deleted the directory
2020-09-23 13:33:13 +08:00
limd
1892677b22
fixed the problem of empty directory when S3 deleted the directory
2020-09-23 13:15:06 +08:00
limd
a99f63cb4d
1.add S3 copy directory function
...
2.fixed the problem of empty directory when S3 deleted the directory
2020-09-23 11:26:01 +08:00
Chris Lu
9cdbfc1a49
refactor
2020-09-21 17:34:38 -07:00
Chris Lu
b9c1f3e9de
s3: fixes for list multipart upload
2020-09-21 10:51:24 -07:00
Chris Lu
9a3b564508
adjust printout
2020-09-20 16:01:56 -07:00
Chris Lu
62563a895a
refactoring
2020-09-20 16:00:01 -07:00
Chris Lu
d013d09a9b
adjust logging
2020-09-20 15:38:59 -07:00
Chris Lu
dc4d2145ff
open up
2020-09-19 20:23:12 -07:00
Chris Lu
29abe980df
s3: add support for PostPolicy
...
fix https://github.com/chrislusf/seaweedfs/issues/1426
2020-09-19 20:14:19 -07:00
Chris Lu
5b40a2690a
refactoring
2020-09-19 14:09:58 -07:00
Chris Lu
2cbd1cf121
fix compilation
2020-09-18 00:15:54 -07:00
Chris Lu
23e9ede068
s3: collect metrics
2020-09-18 00:09:04 -07:00
Chris Lu
3984c3962f
add comment
2020-09-11 15:07:19 -07:00
Chris Lu
baa6bdf4d4
s3: listMultipartUploads fix output format
2020-09-11 15:04:01 -07:00