mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 05:07:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,335 Commits 32 Branches 296 Tags 152 MiB
dc9568fc0d
Commit Graph

62 Commits

Author SHA1 Message Date
chrislu
8f79bb3987 Merge branch 'master' into mq-subscribe 2024-03-07 10:42:29 -08:00
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink (#5352) 
* fix: install cronie

* chore: refactor configure S3Sink

* chore: refactor cinfig

* add filer-backup compose file

* fix: X-Amz-Meta-Mtime and resolve with comments

* fix: attr mtime

* fix: MaxUploadPartst is reduced to the maximum allowable

* fix: env and force set max MaxUploadParts

* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
 2024-03-07 08:35:51 -08:00
chrislu
0aed16a9c4 add make brokers 2024-01-31 21:31:10 -08:00
Konstantin Lebedev
1169f94310
Fix filer sync set offset (#5197) 
* fix: compose 2mount with sync

* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
 2024-01-12 10:57:18 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue (#5098) 
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
 2023-12-11 12:05:54 -08:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
Konstantin Lebedev
a40888d343
fix: make dev_tls with volume https (#4866) 2023-09-25 06:47:38 -07:00
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
Ryan Russell
949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705) 
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-10-16 14:02:33 -07:00
Konstantin Lebedev
b9933d5589
master server graceful stop (#3797) 2022-10-06 09:30:30 -07:00
Ryan Russell
b51880ed3b
docs(docker): readability fixes (#3684) 
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-09-15 01:27:35 -07:00
Patrick Schmidt
a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562) 
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
 2022-08-31 09:27:53 -07:00
Konstantin Lebedev
bcbdc4cb37 use const multipart uploads folder 
avoid error bucket NotEmpty if multipart uploads folder exist
 2022-06-29 16:21:16 +05:00
Chris Lu
e22c2c8012
Merge pull request #3188 from kmlebedev/fix_make_dev 
fix ip.bind cmd for docker compose
 2022-06-29 01:36:59 -07:00
Konstantin Lebedev
5b388ed6c1 initial advancedtls 2022-06-23 23:32:15 +05:00
Konstantin Lebedev
a9f6c4b43d fix ip.bind cmd for docker compose 2022-06-15 16:49:53 +05:00
Konstantin Lebedev
b86628f85d fix show empty dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-06-01 20:10:52 +05:00
Konstantin Lebedev
a14728d508 docker compose nextcloud rm mysql 2022-05-23 20:11:32 +05:00
Konstantin Lebedev
a34438384e docker compose file for nextcloud testing 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-05-23 18:53:35 +05:00
Konstantin Lebedev
d12a423aa0 refactor 
add some tests

https://github.com/chrislusf/seaweedfs/pull/2996
 2022-05-12 12:23:02 +05:00
chrislu
22fbce8fd5 add docker compose file to sync 2 mounts 2022-05-06 03:53:36 -07:00
chrislu
b2a43dd044 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-06 01:51:28 -07:00
chrislu
362dec87a3 add tests 2022-05-06 01:50:34 -07:00
Konstantin Lebedev
8342f651f3 fix scanNamed 2022-05-03 15:18:28 +05:00
Konstantin Lebedev
d8925b4e83 Merge branch 'new_master' into ydb 
# Conflicts:
#	go.mod
#	go.sum
 2022-05-03 00:13:57 +05:00
Konstantin Lebedev
319d300d48 fix createDB 2022-05-02 22:23:07 +05:00
Konstantin Lebedev
275c3bb19c ydb initial 
https://github.com/chrislusf/seaweedfs/issues/2942
 2022-05-01 17:28:58 +05:00
chrislu
192983b464 s3 backend support customizing storage class 2022-04-30 17:36:40 -07:00
Konstantin Lebedev
357aa818fe add raft shell cmds 2022-04-06 15:23:53 +05:00
Konstantin Lebedev
b7cdde14ae auto bootstraping and update peers 2022-04-05 13:17:53 +05:00
Konstantin Lebedev
1a1e5778c3 fix cluster status 2022-04-04 18:52:08 +05:00
Konstantin Lebedev
14dd971890 hashicorp raft with state machine 2022-04-04 17:51:51 +05:00
Konstantin Lebedev
c514710b7b initial add hashicorp raft 2022-04-04 13:50:56 +05:00
Konstantin Lebedev
b6a2b43efe allowDeleteBucketNotEmpty 2022-03-30 22:46:13 +05:00
Konstantin Lebedev
5b90a39954 fix s3 tests: 
multipart_upload_incorrect_etag
multipart_resend_first_finishes_last
 2022-03-29 19:21:09 +05:00
chrislu
24541fc7ec minor 2022-03-21 00:43:47 -07:00
kmlebedev
4f98553ba9 audit log SignatureVersion 2021-12-10 19:40:32 +05:00
Konstantin Lebedev
10678cde81 audit log config 2021-12-07 18:20:52 +05:00
Konstantin Lebedev
9d6ffa0ea1 GetBucketLifecycleConfigurationHandler 2021-10-14 01:35:33 +05:00
Konstantin Lebedev
be4b3ed509 AclHandlers 2021-10-11 15:03:56 +05:00
Konstantin Lebedev
b612d5aebd s3 test get w/ If-Match: bogus ETag 2021-05-24 16:59:44 +05:00
Konstantin Lebedev
69c768870b - object write cache control 
- object write expires
 2021-05-24 15:43:55 +05:00
Chris Lu
e3f47ddb76 debug 2021-05-13 00:32:46 -07:00
Chris Lu
f51bf61a16 remove wrong parameter 2021-04-30 22:51:30 -07:00
Konstantin Lebedev
995ae91007 add DeleteUserPolicy 2021-04-08 11:16:36 +05:00
Konstantin Lebedev
6b3cc94ac4 minio gateway s3 docker compose file 2021-03-29 14:59:59 +05:00
Chris Lu
b1a86cf808 s3: copy object to itself 
fix https://github.com/chrislusf/seaweedfs/issues/1922
 2021-03-19 01:31:49 -07:00
Chris Lu
2d567e00aa add mount memory profiling 2021-03-12 02:54:08 -08:00
Konstantin Lebedev
d66645ecda add 12 tests 
https://github.com/chrislusf/seaweedfs/issues/1810
 2021-03-12 12:40:33 +05:00
Konstantin Lebedev
348e21a08c add comments 2021-03-10 14:42:39 +05:00