mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-24 02:59:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,060 Commits 32 Branches 296 Tags 147 MiB
eac37e41ef
Commit Graph

217 Commits

Author SHA1 Message Date
chrislu
8211b29689 fix s3tests.conf file name 2024-06-24 17:15:16 -07:00
chrislu
dd1571b95b add some delay when starting filers, to simulate real production 2024-05-05 14:18:11 -07:00
chrislu
30f8600a24 working 2024-05-03 21:27:06 -07:00
chrislu
cc1c69f312 Merge branch 'master' into mq-subscribe 2024-04-08 11:03:14 -07:00
Konstantin Lebedev
f9ec92dd83
[tests] add action s3 tests (#5427) 2024-03-29 07:57:27 -07:00
chrislu
707ff536a3 Merge branch 'master' into mq-subscribe 2024-03-25 11:21:19 -07:00
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage (#5403) 2024-03-24 08:36:21 -07:00
chrislu
aba934f0b5 adjust log level 2024-03-16 23:15:06 -07:00
chrislu
8f79bb3987 Merge branch 'master' into mq-subscribe 2024-03-07 10:42:29 -08:00
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink (#5352) 
* fix: install cronie

* chore: refactor configure S3Sink

* chore: refactor cinfig

* add filer-backup compose file

* fix: X-Amz-Meta-Mtime and resolve with comments

* fix: attr mtime

* fix: MaxUploadPartst is reduced to the maximum allowable

* fix: env and force set max MaxUploadParts

* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
 2024-03-07 08:35:51 -08:00
chrislu
a52ce076fc Merge branch 'master' into mq-subscribe 2024-02-25 08:00:37 -08:00
Chris Lu
2ad363d984 upgrade go, rocksdb 2024-02-07 00:41:03 -08:00
Chris Lu
c471265837 build with pub sub clients for testing 2024-02-05 16:47:11 -08:00
chrislu
0aed16a9c4 add make brokers 2024-01-31 21:31:10 -08:00
Konstantin Lebedev
1169f94310
Fix filer sync set offset (#5197) 
* fix: compose 2mount with sync

* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
 2024-01-12 10:57:18 -08:00
chrislu
53188a1191 fix rocksdb build 2024-01-01 22:42:06 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue (#5098) 
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
 2023-12-11 12:05:54 -08:00
chrislu
556a242acf remove unused file 2023-11-30 08:20:01 -08:00
Konstantin Lebedev
ce9b822a8e
golang up version to 1.21.3 (#4911) 2023-10-13 05:31:43 -07:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
Konstantin Lebedev
a40888d343
fix: make dev_tls with volume https (#4866) 2023-09-25 06:47:38 -07:00
chrislu
1494221b08 reduce binary size 2023-09-21 11:58:03 -07:00
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
chrislu
b0da8788a1 upgrade rocksdb versions 2023-05-29 23:14:45 -07:00
chrislu
1f7934a89c enable gocdk in the generated local image 
fix https://github.com/seaweedfs/seaweedfs/issues/4256
 2023-02-27 19:20:02 -08:00
Chris Lu
bac45912e3
go 1.20 (#4192) 2023-02-08 16:41:23 -08:00
CommanderRoot
c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 (#4094) 2023-01-01 05:07:53 -08:00
Ryan Russell
949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705) 
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-10-16 14:02:33 -07:00
Konstantin Lebedev
b9933d5589
master server graceful stop (#3797) 2022-10-06 09:30:30 -07:00
Ryan Russell
b51880ed3b
docs(docker): readability fixes (#3684) 
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-09-15 01:27:35 -07:00
chrislu
5afc20dbd3 add tmux for dev 2022-09-01 14:47:45 -07:00
chrislu
04300dc2ac build local rocksdb 2022-09-01 09:44:48 -07:00
chrislu
466bbe4cdd dockerfile fix directory 2022-09-01 09:43:41 -07:00
chrislu
9c76896e2d fix building rocksdb image 2022-09-01 00:20:23 -07:00
Patrick Schmidt
a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562) 
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
 2022-08-31 09:27:53 -07:00
Konstantin Lebedev
913a5d0dd4
add target dev_race to MakeFile (#3522) 2022-08-26 01:28:03 -07:00
Konstantin Lebedev
d8c54b527e
avoid Dockerfile Entrypoint: "file not found" (#3430) 
https://www.koskila.net/how-to-fix-exec-user-process-caused-no-such-file-or-directory-in-docker-on-windows/
 2022-08-11 15:07:44 -07:00
chrislu
8a61c124c7 go 1.19 2022-08-02 12:17:31 -07:00
chrislu
5fbe437ee9 update rocksdb version to 7.4.4 2022-07-31 19:47:27 -07:00
chrislu
1d05e7fcbd site move 2022-07-29 15:47:14 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev
bcbdc4cb37 use const multipart uploads folder 
avoid error bucket NotEmpty if multipart uploads folder exist
 2022-06-29 16:21:16 +05:00
Chris Lu
e22c2c8012
Merge pull request #3188 from kmlebedev/fix_make_dev 
fix ip.bind cmd for docker compose
 2022-06-29 01:36:59 -07:00
Konstantin Lebedev
5b388ed6c1 initial advancedtls 2022-06-23 23:32:15 +05:00
Konstantin Lebedev
a9f6c4b43d fix ip.bind cmd for docker compose 2022-06-15 16:49:53 +05:00
Konstantin Lebedev
fb57aa431c stats master_replica_placement_mismatch 2022-06-10 15:30:40 +05:00
Chris Lu
f43ec9f363
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud 2022-06-07 08:56:55 -07:00
Kaiwalya Joshi
47d335cf8c
feat: Send commands to weed shell from the docker image. 
Add the ability to send commands to weed shell from the docker image.

Allows an operator to perform maintenance commands like so:
```
docker run \
  --rm \
  -e SHELL_FILER=localhost:8888 \
  -e SHELL_MASTER=localhost:9333 \
  chrislusf/seaweedfs:local \
  "shell" \
  "fs.configure -locationPrefix=/buckets/foo -volumeGrowthCount=3 -replication=002 -apply"
```
 2022-06-01 15:47:10 -07:00
Konstantin Lebedev
b86628f85d fix show empty dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-06-01 20:10:52 +05:00
Konstantin Lebedev
a14728d508 docker compose nextcloud rm mysql 2022-05-23 20:11:32 +05:00