mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-25 10:17:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
10,090 Commits 32 Branches 296 Tags 153 MiB
2ad363d984
Commit Graph

6453 Commits

Author SHA1 Message Date
Konstantin Lebedev
df4ded758e
fix: avoid deleting more than one replica (#4873) 
https://github.com/seaweedfs/seaweedfs/issues/4647

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-26 00:20:48 -07:00
chrislu
19505c1cf4 describe a topic 2023-09-25 20:46:00 -07:00
Tobias Gurtzick
78dbac7702
fix(ec): volumes created by foreign collection due to bug in ec balance (#4864) 
* fix(ec): ignore 0 byte data files

refers to parts of #4861

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

* fix(ec): ignore volumes not from the current collection during balance

fixes #4861

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>

---------

Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
 2023-09-25 10:35:43 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"" 
This reverts commit 8cb42c39
 2023-09-25 09:35:16 -07:00
chrislu
3d07895518 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-25 09:16:35 -07:00
chrislu
a6e35e4f68 fix compilation 2023-09-25 09:16:33 -07:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
chrislu
c9177c92e5 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-25 08:20:16 -07:00
chrislu
81e57bd45e fix test 2023-09-25 08:20:03 -07:00
Lars Lehtonen
28a3a31b27
weed/replication/sub: fix dropped error (#4865) 2023-09-25 07:33:18 -07:00
chrislu
23f334dd13 comment 2023-09-24 23:36:43 -07:00
chrislu
c7e05e4e71 ensure latest stats are reported 2023-09-24 23:34:31 -07:00
chrislu
dff2ce5d2f read existing broker stats 2023-09-24 23:10:33 -07:00
chrislu
1492bf7552 fix listing topics 2023-09-24 23:05:41 -07:00
chrislu
3cf9b8d621 create topic and report topic 2023-09-24 22:00:43 -07:00
chrislu
b3f94feede do create topic 2023-09-24 22:00:43 -07:00
chrislu
d74348048a implement create topic 2023-09-24 22:00:43 -07:00
chrislu
0361c321b4 add CreateTopic API 2023-09-24 22:00:43 -07:00
chrislu
0f8168c0c9 3.57 2023-09-24 21:29:31 -07:00
chrislu
1df4223195 minor 2023-09-24 09:54:57 -07:00
chrislu
10ac25f857 avoid deleting the entry too early 2023-09-23 12:43:00 -07:00
chrislu
c43238b30a fix waiting condition 2023-09-23 12:28:02 -07:00
chrislu
322af1fc04 fix compilation 2023-09-23 12:13:32 -07:00
chrislu
0414b68a4f if shared lock, still need to wait for exclusive lock to release 2023-09-23 12:08:23 -07:00
chrislu
9ba4639a27 adjust for better logs 2023-09-23 11:53:44 -07:00
chrislu
186b1f029b read source uses shared lock 2023-09-23 10:33:44 -07:00
chrislu
2df70ce95f adjust logs 2023-09-21 23:51:32 -07:00
chrislu
3e23421608 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-21 20:08:30 -07:00
chrislu
49a470bcdf reduce logs 2023-09-21 20:08:20 -07:00
Konstantin Lebedev
750a0ba1b2
[iam] Replace action read/write to readAcp/writeAcp for handlers with acl (#4858) 
Replace action read/write to readAcp/writeAcp for handlers with acl query
 https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 14:07:04 -07:00
chrislu
8b2c39f2c0 reduce binary size 2023-09-21 13:22:08 -07:00
chrislu
b388957340 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-21 11:08:30 -07:00
chrislu
31fc165715 mount switch to ordered lock requests 2023-09-21 11:08:26 -07:00
chrislu
7669852241 adjust logs 2023-09-21 11:04:12 -07:00
chrislu
e3b1bacf3f add lock table to ensure ordered locks 2023-09-21 10:24:34 -07:00
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
Konstantin Lebedev
d8b424d123
[s3] optimization iam lookup for reducing algorithm complexity (#4857) 
optimization iam lookup for reducing algorithm complexity
https://github.com/seaweedfs/seaweedfs/issues/4519

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-09-21 08:19:11 -07:00
chrislu
411bdda08d fix compilation 2023-09-19 16:22:41 -07:00
chrislu
d531b7cbbf merge some previous missing changes 2023-09-19 14:10:54 -07:00
chrislu
b18112ae10 fix compilation bugs during merge 2023-09-19 14:08:17 -07:00
chrislu
6c6655de9d wip 2023-09-19 14:03:42 -07:00
chrislu
cf28108ea4 fix grpd dial option 2023-09-19 14:02:51 -07:00
chrislu
27af11f1e8 Revert "Revert "Merge branch 'master' into sub"" 
This reverts commit 0bb97709d4.
 2023-09-18 18:47:34 -07:00
chrislu
0bb97709d4 Revert "Merge branch 'master' into sub" 
This reverts commit 4d414f54a2, reversing
changes made to 4827425146.
 2023-09-18 16:13:20 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs" 
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
 2023-09-18 16:12:50 -07:00
chrislu
2e5aa06026 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-18 16:06:41 -07:00
chrislu
4d414f54a2 Merge branch 'master' into sub 2023-09-18 16:04:59 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850) 
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2023-09-18 14:43:05 -07:00
chrislu
b64fc44e8d Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-18 09:01:17 -07:00
chrislu
340aea9b34 add broker leader 2023-09-18 09:01:14 -07:00
chrislu
0d390ec882 fix nil 2023-09-18 08:52:35 -07:00
Lars Lehtonen
d83107efe5 fix: weed/storage: dropped error 2023-09-18 08:18:57 -07:00
zemul
91cbaef02b fix: mount proxyByFiler write error: wrong jwt 2023-09-18 07:44:50 -07:00
chrislu
4e17d399a6 refactor 2023-09-17 00:04:20 -07:00
chrislu
7a49d3541a avoid nil 2023-09-17 00:04:13 -07:00
chrislu
a65088d739 broker report stats to balancer 2023-09-16 23:00:51 -07:00
chrislu
a1cb49d078 adjust log 2023-09-16 23:00:28 -07:00
chrislu
b590a68e75 revert to non-streaming mode to assign file id 
fix https://github.com/seaweedfs/seaweedfs/issues/4838#issuecomment-1722290685
 2023-09-16 16:13:08 -07:00
chrislu
4650c4c65f Revert "turn on streaming assign file id" 
This reverts commit 733db2bc88.
 2023-09-16 16:09:20 -07:00
chrislu
fa62926347 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-16 15:06:22 -07:00
chrislu
89a1fd1751 Squashed commit of the following: 
commit 4827425146
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Sep 16 15:05:38 2023 -0700

    balancer works

commit 3b50139f68
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:22:32 2023 -0700

    comments

commit 7f685ce7ba
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 15 22:20:05 2023 -0700

    adjust APIs

commit 436d99443b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 14 23:49:05 2023 -0700

    receive broker stats

commit b771fefa37
Merge: 0a851ec00 890881037
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 13 00:03:47 2023 -0700

    Merge branch 'master' into sub

commit 0a851ec00b
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Sep 10 22:01:25 2023 -0700

    Create balancer.go

commit 39941edc0b
Author: chrislu <chris.lu@gmail.com>
Date:   Thu Sep 7 23:55:19 2023 -0700

    add publisher shutdown

commit 875f562779
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:16:41 2023 -0700

    server side send response at least once per second

commit 984b6c54cf
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 23:15:29 2023 -0700

    ack interval 128

commit 2492a45499
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Sep 6 22:39:46 2023 -0700

    ack interval

commit ba67e6ca29
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:50 2023 -0700

    api for sub

commit 9e4f985698
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Sep 4 21:43:30 2023 -0700

    publish, benchmark

commit cb470d44df
Author: chrislu <chris.lu@gmail.com>
Date:   Fri Sep 1 00:36:51 2023 -0700

    can pub and sub

commit 1eb2da46d5
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:02:12 2023 -0700

    connect and publish

commit 504ae8383a
Author: chrislu <chris.lu@gmail.com>
Date:   Mon Aug 28 09:01:25 2023 -0700

    protoc version

commit dbcba75271
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:59:04 2023 -0700

    rename to lookup

commit c9caf33119
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 18:33:46 2023 -0700

    move functions

commit 4d6c18d86f
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 17:50:59 2023 -0700

    pub sub initial tests

commit 4eb8e8624d
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:14:39 2023 -0700

    rename

commit 1990456670
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 27 13:13:14 2023 -0700

    sub

commit 905911853d
Author: chrislu <chris.lu@gmail.com>
Date:   Sat Aug 26 13:39:21 2023 -0700

    adjust proto
 2023-09-16 15:06:16 -07:00
chrislu
4827425146 balancer works 2023-09-16 15:05:38 -07:00
chrislu
3b50139f68 comments 2023-09-15 22:22:32 -07:00
chrislu
7f685ce7ba adjust APIs 2023-09-15 22:20:05 -07:00
chrislu
436d99443b receive broker stats 2023-09-14 23:49:05 -07:00
wang wusong
9e5b1eef9b avoid passe lock by value 2023-09-14 06:51:12 -07:00
chrislu
b771fefa37 Merge branch 'master' into sub 2023-09-13 00:03:47 -07:00
annluda
8908810376 Fix s3 api object list with params max-keys and prefix 2023-09-11 21:35:15 -07:00
chrislu
5407b86766 3.56 2023-09-10 22:05:41 -07:00
chrislu
07c2d75ac7 ensure single threaded access 2023-09-10 22:03:44 -07:00
chrislu
0a851ec00b Create balancer.go 2023-09-10 22:01:25 -07:00
chrislu
39941edc0b add publisher shutdown 2023-09-07 23:55:19 -07:00
chrislu
875f562779 server side send response at least once per second 2023-09-06 23:16:41 -07:00
chrislu
984b6c54cf ack interval 128 2023-09-06 23:15:29 -07:00
chrislu
2492a45499 ack interval 2023-09-06 22:39:46 -07:00
Lars Lehtonen
f24c7e803f weed/storage/erasure_coding: Close() after error handling 2023-09-06 07:35:54 -07:00
Lars Lehtonen
d762b12e11 weed/storage/erasure_coding: fix dropped test error 2023-09-06 07:35:54 -07:00
Lars Lehtonen
a879e1bbb5 weed/storage/erasure_coding: remove unused err from encodeDatFile() signature 2023-09-06 07:35:54 -07:00
Patrick Schmidt
98dcec0ee2 Clean up old signature hash pools 2023-09-05 10:33:27 -07:00
Patrick Schmidt
cdd817edf9 Improve S3 request signing performance 
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
 2023-09-05 10:33:27 -07:00
chrislu
ba67e6ca29 api for sub 2023-09-04 21:43:50 -07:00
chrislu
9e4f985698 publish, benchmark 2023-09-04 21:43:30 -07:00
Lars Lehtonen
e6385e5e8b
weed/util: fix dropped errors (#4803) 2023-09-01 13:10:45 -07:00
chrislu
cb470d44df can pub and sub 2023-09-01 00:36:51 -07:00
chrislu
1eb2da46d5 connect and publish 2023-08-28 09:02:12 -07:00
chrislu
504ae8383a protoc version 2023-08-28 09:01:25 -07:00
chrislu
dbcba75271 rename to lookup 2023-08-27 18:59:04 -07:00
chrislu
c9caf33119 move functions 2023-08-27 18:33:46 -07:00
chrislu
4d6c18d86f pub sub initial tests 2023-08-27 17:50:59 -07:00
chrislu
4eb8e8624d rename 2023-08-27 13:14:39 -07:00
chrislu
1990456670 sub 2023-08-27 13:13:14 -07:00
chrislu
905911853d adjust proto 2023-08-26 13:39:21 -07:00
Nico D'Cotta
796b7508f3
Implement SRV lookups for filer (#4767) 2023-08-24 07:08:56 -07:00
chrislu
733db2bc88 turn on streaming assign file id 2023-08-23 00:34:21 -07:00
chrislu
99f037b958 streaming assign file ids 2023-08-23 00:31:33 -07:00
chrislu
94b7e2a37c add stream assign server side implementation 2023-08-22 09:59:04 -07:00
chrislu
ccedad5196 refactor files 2023-08-22 09:54:06 -07:00
chrislu
010c5e91e3 add stream assign proto 2023-08-22 09:53:54 -07:00
chrislu
a1149f060b upgrade jwt 2023-08-22 08:26:31 -07:00
chrislu
c49caff5a1 change from deprecated jwt.StandardClaims to new jwt.RegisteredClaims 2023-08-21 23:42:39 -07:00
chrislu
c45665eb97 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-08-21 09:47:36 -07:00
SmsS4
f61490966f
Add time to first byte metric for s3 (#4768) 
* Add time to first byte metric for s3

* Change second to millisecond
 2023-08-21 00:42:39 -07:00
chrislu
01d70c21f3 Squashed commit of the following: 
commit 32f4b1a13057d56b6de487cdb80ff7c205af01a6
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 22:52:19 2023 -0700

    fix compilation

commit e77ad33b7ca0423138fbae26a4433b60923a9588
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 22:46:44 2023 -0700

    pub

commit f431f30cc7ca277ca299e3cd118c05537fb9f5c3
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 13:27:39 2023 -0700

    fix generic type

commit 4e9dcb18293fd1e3e306e2dceb995dfd67a35e1d
Merge: 30f942580 16e3f2d52
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 12:47:14 2023 -0700

    Merge branch 'master' into pubsub

commit 30f942580ad1bb32ae94aade2e3a21ec3ab63e21
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 11:10:58 2023 -0700

    wip

commit f8b00980bc2f3879bb43decffd9a08d842f196f2
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:14:35 2023 -0700

    add design document

commit 08d2bebe42a26ebc39f1542f54d99e73620727dd
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:14:06 2023 -0700

    minor

commit bcfa7982b262a40fcdce6fc6613fad2ce07c13da
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:13:49 2023 -0700

    rename
 2023-08-20 22:53:05 -07:00
chrislu
16e3f2d528 fix log buffer test 2023-08-20 12:46:15 -07:00
chrislu
3e9c32a3f0 refactoring to use atomic bool 
fix compilation
 2023-08-20 12:13:06 -07:00
SmsS4
1e72579ac7
Fix s3.clean.uploads (#4766) 2023-08-19 06:30:18 -07:00
chrislu
6792db21dc ErrInvalidPart when completed parts has extra parts 
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
 2023-08-17 00:51:42 -07:00
chrislu
f5c4b642bb re-ordering 2023-08-16 23:54:23 -07:00
chrislu
3852307e94 renaming 2023-08-16 23:47:43 -07:00
chrislu
6c7fa567d4 add separate cache directory for write buffers 2023-08-16 23:39:21 -07:00
SmsS4
ea1b259c12
Update weed fix description (#4745) 2023-08-13 03:50:53 -07:00
Nikita Mochalov
4b1ba7f5b2
Save disk space metrics immediately (#4740) 2023-08-10 06:52:32 -07:00
Nikita Mochalov
e6a49dc533
Fix resource leaks (#4737) 
* Fix division by zero

* Fix file handle leak

* Fix file handle leak

* Fix file handle leak

* Fix goroutine leak
 2023-08-09 15:30:36 -07:00
chrislu
3365468d0d added an error message 2023-08-08 20:35:21 -07:00
wusong
3ec977cd6c
minor (#4723) 2023-08-01 02:34:59 -07:00
chrislu
c09100f928 fix building mongo index 2023-07-24 11:09:51 -07:00
chrislu
7eafa3420b 3.55 2023-07-23 23:11:37 -07:00
SmoothDenis
51bcc219ea
s3api should return 500 code from filer (#4699) 2023-07-22 07:22:38 -07:00
wusong
33b9192536
Fix large file chunks deleted by mistake (#4678) 2023-07-17 07:52:14 -07:00
chrislu
358b3a4894 3.54 2023-07-16 22:47:57 -07:00
chrislu
6f588b5b18 fix refactoring mistake 
fix https://github.com/seaweedfs/seaweedfs/issues/4639
 2023-07-16 11:22:48 -07:00
mervynzhang
df400e6c71
Concurrency works better (#4663) 
Co-authored-by: mervyn.zhang <mervyn.zhang@sap.com>
 2023-07-12 23:04:54 -07:00
wusong
0cb9ddd8ec
Fix data loss: add lock for metacache (#4664) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-07-11 22:23:32 -07:00
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows (#4666) 
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
 2023-07-11 09:05:14 -07:00
Henco Appel
680c2e76f1
Fix the segfault from not setting s3 https port (#4667) 2023-07-11 08:03:20 -07:00
Konstantin Lebedev
2c076ebd96
avoid logging nil error (#4659) 
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-07-10 23:38:52 -07:00
LHHDZ
3730772e05
refresh max volume count after adjust (#4662) 
Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2023-07-10 23:37:33 -07:00
chrislu
c278bac263 avoid nil needle map 
fix https://github.com/seaweedfs/seaweedfs/issues/4640
 2023-07-07 22:16:58 -07:00
chrislu
0606b59a3e mount: create mount root on filer 
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/127
 2023-07-07 09:30:32 -07:00
chrislu
14546f0ad8 minor 2023-07-06 09:12:28 -07:00
chrislu
ab30f489c3 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-07-06 00:33:02 -07:00
chrislu
31b2751aff clone volume locations in case they are changed 
fix https://github.com/seaweedfs/seaweedfs/issues/4642
 2023-07-06 00:32:58 -07:00
wusong
61553beba2
Fix DataBackend nil pointer (#4641) 2023-07-04 23:22:10 -07:00
chrislu
a315490f7d proxy to master uses http address 
fix https://github.com/seaweedfs/seaweedfs/issues/4607
 2023-07-04 11:45:21 -07:00
chrislu
2aa59ab37c fix copying level db files during commi 
fix https://github.com/seaweedfs/seaweedfs/issues/4635
 2023-07-04 11:28:12 -07:00
wusong
c44a6552d2
Fix buff use (#4628) 2023-07-03 08:33:34 -07:00
Amir H. Yeganemehr
f0d3c874b6
Unix Socket listener for S3 server (#4621) 
Add localSocket option to s3 server
 2023-06-26 16:22:45 -07:00
chrislu
2c4c2f0994 3.53 2023-06-25 22:51:14 -07:00
chrislu
f193942adc lock on append metadata operation 2023-06-25 22:07:23 -07:00
chrislu
61c42f9991 adjust lock APIs 2023-06-25 20:30:20 -07:00
chrislu
aafb874ad2 lock until cancelled 2023-06-25 19:31:25 -07:00
chrislu
c4f44d56f7 add distributed lock client 2023-06-25 17:38:34 -07:00
chrislu
f8aa5ea844 adjust filer.proto 2023-06-25 17:38:19 -07:00
chrislu
13a8593ab9 adjust errors 2023-06-25 17:37:54 -07:00
chrislu
06471dac9d init lock ring 2023-06-25 15:28:16 -07:00
chrislu
5149b3d07b filer can proxy to peer filer holding the lock 2023-06-25 15:28:01 -07:00
chrislu
868f7875d7 refactor 2023-06-25 14:30:58 -07:00
chrislu
26eff062c8 refactor 2023-06-25 14:17:58 -07:00
chrislu
464a71a373 add distributed lock manager 2023-06-25 14:14:40 -07:00
chrislu
ee4f7cd636 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-06-25 00:58:24 -07:00
chrislu
3fd659df2a add distributed lock manager 2023-06-25 00:58:21 -07:00
Damiano Albani
0d9ba8c612
Support templating name of files stored via Rclone backend (#4606) 2023-06-24 07:08:24 -07:00
chrislu
a82c44972b refactor 2023-06-20 00:12:59 -07:00
chrislu
14a19170e2 refactor 2023-06-19 23:51:04 -07:00
chrislu
84ced2abf5 add LockRing 2023-06-19 23:48:33 -07:00
chrislu
fe1716ccdf remove deprecated tests 2023-06-19 18:43:51 -07:00
chrislu
1ce725728d minor 2023-06-19 18:43:41 -07:00
chrislu
11549706ef refactor 2023-06-19 18:21:07 -07:00
chrislu
8ec1bc2c99 remove unused cluster node leader 2023-06-19 18:19:13 -07:00
Bai Jie
3b88ab42aa
remove duplicate fileCount query (#4588) 2023-06-18 00:14:14 -07:00
Bai Jie
44b9d72ef0
doIsEmpty() return error if v.DataBackend is nil (#4587) 2023-06-18 00:13:40 -07:00
chrislu
94fbf02ea2 generate jwt token when looking up needle id 
fix https://github.com/seaweedfs/seaweedfs/issues/4577
 2023-06-17 00:11:23 -07:00
柏杰
0b0fb9b9e4
avoid data race read volume.IsEmpty (#4574) 
* avoid data race read volume.IsEmpty

-   avoid phantom read isEmpty for onlyEmpty
-   use `v.DataBackend.GetStat()` in v.dataFileAccessLock scope

* add Destroy(onlyEmpty: true) test

* add Destroy(onlyEmpty: false) test

* remove unused `IsEmpty()`

* change literal `8` to `SuperBlockSize`
 2023-06-14 14:39:58 -07:00
Konstantin Lebedev
1e22d5caf2
fix get file stats for IsEmpty (#4576) 2023-06-14 01:43:30 -07:00
Konstantin Lebedev
4dd890d4a2
optional https port for s3 (#4482) 
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-13 23:58:49 -07:00
chrislu
3fbf4f6189 Revert "Initial patch for redis version 7 support (#4572)" 
This reverts commit a25bca0692.
 2023-06-12 22:27:38 -07:00
Konstantin Lebedev
4527ead295
fix from comment delete volume is empty (#4573) 
* fix from coments
https://github.com/seaweedfs/seaweedfs/pull/4561

* fix tests

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-12 22:22:46 -07:00
Aaron Gipson
a25bca0692
Initial patch for redis version 7 support (#4572) 2023-06-12 13:30:03 -07:00
Konstantin Lebedev
25535e9c36
Delete volume is empty (#4561) 
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559

* fix IsEmpty

* fix test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-12 10:42:44 -07:00
柏杰
68feffa4e6
avoid overwriting non-nil uploadReaderToChunks.uploadErr with nil value (#4560) 2023-06-10 23:15:27 -07:00
Konstantin Lebedev
5ee04d20fa
Healthz check for deadlocks (#4558) 2023-06-09 09:42:48 -07:00
wusong
d6af3a8eb6
fix 404 not found (#4536) 
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-06-06 09:14:49 -07:00
Konstantin Lebedev
5caff2c0f0
fix: deletes newly created volumes (#4551) 
https://github.com/seaweedfs/seaweedfs/issues/4531

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-06 09:08:27 -07:00
wusong
9bdbf9c880
revert #4491 (#4550) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-06-06 00:17:51 -07:00
chrislu
0bffb3770f support TTL when creating folders 
related to https://github.com/seaweedfs/seaweedfs/discussions/4549
 2023-06-05 23:29:25 -07:00
wusong
26f15d0079
Fix no more writable volumes by delay judgment (#4548) 
* fix nomore writables volumes while disk free space is sufficient by time delay

* reset

---------

Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-06-05 10:17:21 -07:00
chrislu
fb4b61036c 3.52 2023-06-04 22:34:26 -07:00
Konstantin Lebedev
a0931be0c0
S3 TLS credentials Refreshing (#4506) 
* S3 TLS credentials Refreshing

* fix: logging

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-04 14:27:56 -07:00
wusong
5aec6da8a3
fix deadlock for filer upload (#4527) 
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-06-03 13:38:27 -07:00
chrislu
e23f3d6eca set df.File to nil after it is closed 
possibly fix https://github.com/seaweedfs/seaweedfs/issues/4530
 2023-06-02 17:16:45 -07:00
chrislu
ca7cc61319 reduce the window size between unregistering a volume server and creating volumes on that server 
fix https://github.com/seaweedfs/seaweedfs/issues/4467
 2023-05-30 08:20:00 -07:00
chrislu
68c3cf767c Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-05-29 16:45:28 -07:00
chrislu
d1d1757520 adjust error message 2023-05-29 16:45:25 -07:00
wusong
e7f4418855
add return for error handle (#4502) 2023-05-24 07:45:06 -07:00
chrislu
4310e1fac4 3.51 2023-05-21 23:43:39 -07:00
wusong
8fffe3e822
fix no more writables volumes while disk free space is sufficient (#4491) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-05-21 22:18:50 -07:00
mervynzhang
1ebb549f77
support swift (#4480) 2023-05-19 06:39:25 -07:00
Mesar Hameed
a90b777ff4
Feat: etcd filer store keys should have customizable prefix (#4484) 
An etcd cluster is not necessarily only dedicated to seaweedfs.
This security enhancement adds a customizable key_prefix option to the etcd filer store.
This will allow an etcd cluster administrator to limit the seaweedfs etcd user to only read/write a subset of keys under the
key_prefix, instead of all keys on the etcd cluster.
 2023-05-18 23:08:56 -07:00
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix (#4465) 
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
 2023-05-16 09:39:43 -07:00
chrislu
b7f011f777 fix test 2023-05-15 17:46:56 -07:00
Mesar Hameed
65484e80ef
Feat: support username/password authentication for etcd filer store s… (#4477) 
Feat: support username/password authentication for etcd filer store seaweedfs/seaweedfs#4262

Co-authored-by: Mesar Hameed <mesar.hameed@gmail.com>
 2023-05-15 17:45:14 -07:00
chrislu
9204ee2d2d 3.50 2023-05-11 08:03:11 -07:00
chrislu
d999f1f0e2 update raft version 
fix #4460
 2023-05-09 22:54:23 -07:00
chrislu
25b237b83c reset wait time on a successful retry 2023-05-07 23:39:33 -07:00
chrislu
59f55c1a66 3.49 2023-05-07 14:00:19 -07:00
chrislu
b2a3860290 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-05-07 13:33:47 -07:00
chrislu
4511edc871 update raft 2023-05-07 13:33:44 -07:00
themarkchen
7592d013fe
fix shell volume.balance bug (#4447) 2023-05-04 07:35:37 -07:00
chrislu
42766bccc8 3.48 2023-04-30 22:20:18 -07:00
zemul
e9fda774f4
[Filer] post add param:saveInside (#4434) 
* fix:mount deadlock

* feat: filer http upload to metadata

* feat: /etc save inside

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2023-04-29 08:31:05 -07:00