chrislu
bf13f3ced7
add subscriber coordinator
2023-10-02 01:02:27 -07:00
chrislu
734178093e
refactor TopicPartition struct
2023-10-02 01:01:45 -07:00
chrislu
2a578b9033
refactor
2023-10-01 11:59:19 -07:00
chrislu
321fda552c
comment
2023-10-01 11:36:27 -07:00
chrislu
b02fdeabff
RetryForever => RetryUntil
2023-10-01 11:33:56 -07:00
chrislu
ebd6f96d35
adjust mq.proto
2023-09-30 21:47:57 -07:00
chrislu
02cb799481
add a test framework for grpc
2023-09-30 21:36:58 -07:00
chrislu
c4e6545197
fix compilation
2023-09-30 21:26:09 -07:00
chrislu
358cba43ef
update proto generated files
2023-09-30 13:19:25 -07:00
chrislu
597ab1e4bf
add design for mq balancer
2023-09-30 13:19:05 -07:00
chrislu
d7c211fe8c
adjust mq proto
2023-09-30 13:18:49 -07:00
chrislu
40888b5612
align 64bit atomic operation
...
fix https://github.com/seaweedfs/seaweedfs/issues/4878
2023-09-28 07:58:43 -07:00
Konstantin Lebedev
dd580190b4
fix: avoid deleting one replica without sync ( #4875 )
...
* fix: avoid deleting one replica without sync
https://github.com/seaweedfs/seaweedfs/issues/4647
* Update weed/shell/command_volume_fix_replication.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* fix: revert this existing do option to positive
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-27 23:12:10 -07:00
Konstantin Lebedev
44906f1f3b
fix: avoid error file name too long when writing a file ( #4876 )
2023-09-27 05:40:51 -07:00
chrislu
9d589b48e6
rename function
2023-09-26 15:17:33 -07:00
Konstantin Lebedev
df4ded758e
fix: avoid deleting more than one replica ( #4873 )
...
https://github.com/seaweedfs/seaweedfs/issues/4647
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-09-26 00:20:48 -07:00
chrislu
19505c1cf4
describe a topic
2023-09-25 20:46:00 -07:00
Tobias Gurtzick
78dbac7702
fix(ec): volumes created by foreign collection due to bug in ec balance ( #4864 )
...
* fix(ec): ignore 0 byte data files
refers to parts of #4861
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
* fix(ec): ignore volumes not from the current collection during balance
fixes #4861
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
---------
Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
2023-09-25 10:35:43 -07:00
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
...
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
3d07895518
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-25 09:16:35 -07:00
chrislu
a6e35e4f68
fix compilation
2023-09-25 09:16:33 -07:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration ( #4859 )
...
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface
https://github.com/seaweedfs/seaweedfs/issues/4519
* fix: test bucket acl default and
adjust the variable names
* fix: s3 api config test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-25 08:34:12 -07:00
chrislu
c9177c92e5
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-25 08:20:16 -07:00
chrislu
81e57bd45e
fix test
2023-09-25 08:20:03 -07:00
Lars Lehtonen
28a3a31b27
weed/replication/sub: fix dropped error ( #4865 )
2023-09-25 07:33:18 -07:00
chrislu
23f334dd13
comment
2023-09-24 23:36:43 -07:00
chrislu
c7e05e4e71
ensure latest stats are reported
2023-09-24 23:34:31 -07:00
chrislu
dff2ce5d2f
read existing broker stats
2023-09-24 23:10:33 -07:00
chrislu
1492bf7552
fix listing topics
2023-09-24 23:05:41 -07:00
chrislu
3cf9b8d621
create topic and report topic
2023-09-24 22:00:43 -07:00
chrislu
b3f94feede
do create topic
2023-09-24 22:00:43 -07:00
chrislu
d74348048a
implement create topic
2023-09-24 22:00:43 -07:00
chrislu
0361c321b4
add CreateTopic API
2023-09-24 22:00:43 -07:00
chrislu
0f8168c0c9
3.57
2023-09-24 21:29:31 -07:00
chrislu
1df4223195
minor
2023-09-24 09:54:57 -07:00
chrislu
10ac25f857
avoid deleting the entry too early
2023-09-23 12:43:00 -07:00
chrislu
c43238b30a
fix waiting condition
2023-09-23 12:28:02 -07:00
chrislu
322af1fc04
fix compilation
2023-09-23 12:13:32 -07:00
chrislu
0414b68a4f
if shared lock, still need to wait for exclusive lock to release
2023-09-23 12:08:23 -07:00
chrislu
9ba4639a27
adjust for better logs
2023-09-23 11:53:44 -07:00
chrislu
186b1f029b
read source uses shared lock
2023-09-23 10:33:44 -07:00
chrislu
2df70ce95f
adjust logs
2023-09-21 23:51:32 -07:00
chrislu
3e23421608
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-21 20:08:30 -07:00
chrislu
49a470bcdf
reduce logs
2023-09-21 20:08:20 -07:00
Konstantin Lebedev
750a0ba1b2
[iam] Replace action read/write to readAcp/writeAcp for handlers with acl ( #4858 )
...
Replace action read/write to readAcp/writeAcp for handlers with acl query
https://github.com/seaweedfs/seaweedfs/issues/4519
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-09-21 14:07:04 -07:00
chrislu
8b2c39f2c0
reduce binary size
2023-09-21 13:22:08 -07:00
chrislu
b388957340
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-21 11:08:30 -07:00
chrislu
31fc165715
mount switch to ordered lock requests
2023-09-21 11:08:26 -07:00
chrislu
7669852241
adjust logs
2023-09-21 11:04:12 -07:00
chrislu
e3b1bacf3f
add lock table to ensure ordered locks
2023-09-21 10:24:34 -07:00
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. ( #4856 )
...
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private
https://github.com/seaweedfs/seaweedfs/issues/4519
* Update weed/s3api/s3api_bucket_handlers.go
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-21 08:20:05 -07:00
Konstantin Lebedev
d8b424d123
[s3] optimization iam lookup for reducing algorithm complexity ( #4857 )
...
optimization iam lookup for reducing algorithm complexity
https://github.com/seaweedfs/seaweedfs/issues/4519
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-09-21 08:19:11 -07:00
chrislu
411bdda08d
fix compilation
2023-09-19 16:22:41 -07:00
chrislu
d531b7cbbf
merge some previous missing changes
2023-09-19 14:10:54 -07:00
chrislu
b18112ae10
fix compilation bugs during merge
2023-09-19 14:08:17 -07:00
chrislu
6c6655de9d
wip
2023-09-19 14:03:42 -07:00
chrislu
cf28108ea4
fix grpd dial option
2023-09-19 14:02:51 -07:00
chrislu
27af11f1e8
Revert "Revert "Merge branch 'master' into sub""
...
This reverts commit 0bb97709d4
2023-09-18 18:47:34 -07:00
chrislu
0bb97709d4
Revert "Merge branch 'master' into sub"
...
This reverts commit 4d414f54a24827425146
2023-09-18 16:13:20 -07:00
chrislu
8cb42c39ad
Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs "
...
This reverts commit 2e5aa060264d414f54a2
2023-09-18 16:12:50 -07:00
chrislu
2e5aa06026
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-18 16:06:41 -07:00
chrislu
4d414f54a2
Merge branch 'master' into sub
2023-09-18 16:04:59 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ( #4850 )
...
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* API changes
* go mod
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2023-09-18 14:43:05 -07:00
chrislu
b64fc44e8d
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-18 09:01:17 -07:00
chrislu
340aea9b34
add broker leader
2023-09-18 09:01:14 -07:00
chrislu
0d390ec882
fix nil
2023-09-18 08:52:35 -07:00
Lars Lehtonen
d83107efe5
fix: weed/storage: dropped error
2023-09-18 08:18:57 -07:00
zemul
91cbaef02b
fix: mount proxyByFiler write error: wrong jwt
2023-09-18 07:44:50 -07:00
chrislu
4e17d399a6
refactor
2023-09-17 00:04:20 -07:00
chrislu
7a49d3541a
avoid nil
2023-09-17 00:04:13 -07:00
chrislu
a65088d739
broker report stats to balancer
2023-09-16 23:00:51 -07:00
chrislu
a1cb49d078
adjust log
2023-09-16 23:00:28 -07:00
chrislu
b590a68e75
revert to non-streaming mode to assign file id
...
fix https://github.com/seaweedfs/seaweedfs/issues/4838#issuecomment-1722290685
2023-09-16 16:13:08 -07:00
chrislu
4650c4c65f
Revert "turn on streaming assign file id"
...
This reverts commit 733db2bc88
2023-09-16 16:09:20 -07:00
chrislu
fa62926347
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-16 15:06:22 -07:00
chrislu
89a1fd1751
Squashed commit of the following:
...
commit 48274251463b50139f687f685ce7ba436d99443bb771fefa370a851ec008908810370a851ec00b39941edc0b875f562779984b6c54cf2492a45499ba67e6ca299e4f985698cb470d44df1eb2da46d5504ae8383adbcba75271c9caf331194d6c18d86f4eb8e8624d1990456670905911853d
2023-09-16 15:06:16 -07:00
chrislu
4827425146
balancer works
2023-09-16 15:05:38 -07:00
chrislu
3b50139f68
comments
2023-09-15 22:22:32 -07:00
chrislu
7f685ce7ba
adjust APIs
2023-09-15 22:20:05 -07:00
chrislu
436d99443b
receive broker stats
2023-09-14 23:49:05 -07:00
wang wusong
9e5b1eef9b
avoid passe lock by value
2023-09-14 06:51:12 -07:00
chrislu
b771fefa37
Merge branch 'master' into sub
2023-09-13 00:03:47 -07:00
annluda
8908810376
Fix s3 api object list with params max-keys and prefix
2023-09-11 21:35:15 -07:00
chrislu
5407b86766
3.56
2023-09-10 22:05:41 -07:00
chrislu
07c2d75ac7
ensure single threaded access
2023-09-10 22:03:44 -07:00
chrislu
0a851ec00b
Create balancer.go
2023-09-10 22:01:25 -07:00
chrislu
39941edc0b
add publisher shutdown
2023-09-07 23:55:19 -07:00
chrislu
875f562779
server side send response at least once per second
2023-09-06 23:16:41 -07:00
chrislu
984b6c54cf
ack interval 128
2023-09-06 23:15:29 -07:00
chrislu
2492a45499
ack interval
2023-09-06 22:39:46 -07:00
Lars Lehtonen
f24c7e803f
weed/storage/erasure_coding: Close() after error handling
2023-09-06 07:35:54 -07:00
Lars Lehtonen
d762b12e11
weed/storage/erasure_coding: fix dropped test error
2023-09-06 07:35:54 -07:00
Lars Lehtonen
a879e1bbb5
weed/storage/erasure_coding: remove unused err from encodeDatFile() signature
2023-09-06 07:35:54 -07:00
Patrick Schmidt
98dcec0ee2
Clean up old signature hash pools
2023-09-05 10:33:27 -07:00
Patrick Schmidt
cdd817edf9
Improve S3 request signing performance
...
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
2023-09-05 10:33:27 -07:00
chrislu
ba67e6ca29
api for sub
2023-09-04 21:43:50 -07:00
chrislu
9e4f985698
publish, benchmark
2023-09-04 21:43:30 -07:00
Lars Lehtonen
e6385e5e8b
weed/util: fix dropped errors ( #4803 )
2023-09-01 13:10:45 -07:00
chrislu
cb470d44df
can pub and sub
2023-09-01 00:36:51 -07:00
chrislu
1eb2da46d5
connect and publish
2023-08-28 09:02:12 -07:00
chrislu
504ae8383a
protoc version
2023-08-28 09:01:25 -07:00
chrislu
dbcba75271
rename to lookup
2023-08-27 18:59:04 -07:00
chrislu
c9caf33119
move functions
2023-08-27 18:33:46 -07:00
chrislu
4d6c18d86f
pub sub initial tests
2023-08-27 17:50:59 -07:00
chrislu
4eb8e8624d
rename
2023-08-27 13:14:39 -07:00
chrislu
1990456670
sub
2023-08-27 13:13:14 -07:00
chrislu
905911853d
adjust proto
2023-08-26 13:39:21 -07:00
Nico D'Cotta
796b7508f3
Implement SRV lookups for filer ( #4767 )
2023-08-24 07:08:56 -07:00
chrislu
733db2bc88
turn on streaming assign file id
2023-08-23 00:34:21 -07:00
chrislu
99f037b958
streaming assign file ids
2023-08-23 00:31:33 -07:00
chrislu
94b7e2a37c
add stream assign server side implementation
2023-08-22 09:59:04 -07:00
chrislu
ccedad5196
refactor files
2023-08-22 09:54:06 -07:00
chrislu
010c5e91e3
add stream assign proto
2023-08-22 09:53:54 -07:00
chrislu
a1149f060b
upgrade jwt
2023-08-22 08:26:31 -07:00
chrislu
c49caff5a1
change from deprecated jwt.StandardClaims to new jwt.RegisteredClaims
2023-08-21 23:42:39 -07:00
chrislu
c45665eb97
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-08-21 09:47:36 -07:00
SmsS4
f61490966f
Add time to first byte metric for s3 ( #4768 )
...
* Add time to first byte metric for s3
* Change second to millisecond
2023-08-21 00:42:39 -07:00
chrislu
01d70c21f3
Squashed commit of the following:
...
commit 32f4b1a13057d56b6de487cdb80ff7c205af01a6
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 22:52:19 2023 -0700
fix compilation
commit e77ad33b7ca0423138fbae26a4433b60923a9588
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 22:46:44 2023 -0700
pub
commit f431f30cc7ca277ca299e3cd118c05537fb9f5c3
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 13:27:39 2023 -0700
fix generic type
commit 4e9dcb18293fd1e3e306e2dceb995dfd67a35e1d
Merge: 30f942580 16e3f2d52
2023-08-20 22:53:05 -07:00
chrislu
16e3f2d528
fix log buffer test
2023-08-20 12:46:15 -07:00
chrislu
3e9c32a3f0
refactoring to use atomic bool
...
fix compilation
2023-08-20 12:13:06 -07:00
SmsS4
1e72579ac7
Fix s3.clean.uploads ( #4766 )
2023-08-19 06:30:18 -07:00
chrislu
6792db21dc
ErrInvalidPart when completed parts has extra parts
...
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
2023-08-17 00:51:42 -07:00
chrislu
f5c4b642bb
re-ordering
2023-08-16 23:54:23 -07:00
chrislu
3852307e94
renaming
2023-08-16 23:47:43 -07:00
chrislu
6c7fa567d4
add separate cache directory for write buffers
2023-08-16 23:39:21 -07:00
SmsS4
ea1b259c12
Update weed fix description ( #4745 )
2023-08-13 03:50:53 -07:00
Nikita Mochalov
4b1ba7f5b2
Save disk space metrics immediately ( #4740 )
2023-08-10 06:52:32 -07:00
Nikita Mochalov
e6a49dc533
Fix resource leaks ( #4737 )
...
* Fix division by zero
* Fix file handle leak
* Fix file handle leak
* Fix file handle leak
* Fix goroutine leak
2023-08-09 15:30:36 -07:00
chrislu
3365468d0d
added an error message
2023-08-08 20:35:21 -07:00
wusong
3ec977cd6c
minor ( #4723 )
2023-08-01 02:34:59 -07:00
chrislu
c09100f928
fix building mongo index
2023-07-24 11:09:51 -07:00
chrislu
7eafa3420b
3.55
2023-07-23 23:11:37 -07:00
SmoothDenis
51bcc219ea
s3api should return 500 code from filer ( #4699 )
2023-07-22 07:22:38 -07:00
wusong
33b9192536
Fix large file chunks deleted by mistake ( #4678 )
2023-07-17 07:52:14 -07:00
chrislu
358b3a4894
3.54
2023-07-16 22:47:57 -07:00
chrislu
6f588b5b18
fix refactoring mistake
...
fix https://github.com/seaweedfs/seaweedfs/issues/4639
2023-07-16 11:22:48 -07:00
mervynzhang
df400e6c71
Concurrency works better ( #4663 )
...
Co-authored-by: mervyn.zhang <mervyn.zhang@sap.com>
2023-07-12 23:04:54 -07:00
wusong
0cb9ddd8ec
Fix data loss: add lock for metacache ( #4664 )
...
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-07-11 22:23:32 -07:00
Henco Appel
89542db2d0
Fix S3 API multipart upload on Windows ( #4666 )
...
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.
Co-authored-by: Hendrik Appel <happel@europe.altair.com>
2023-07-11 09:05:14 -07:00
Henco Appel
680c2e76f1
Fix the segfault from not setting s3 https port ( #4667 )
2023-07-11 08:03:20 -07:00
Konstantin Lebedev
2c076ebd96
avoid logging nil error ( #4659 )
...
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-07-10 23:38:52 -07:00
LHHDZ
3730772e05
refresh max volume count after adjust ( #4662 )
...
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-07-10 23:37:33 -07:00
chrislu
c278bac263
avoid nil needle map
...
fix https://github.com/seaweedfs/seaweedfs/issues/4640
2023-07-07 22:16:58 -07:00
chrislu
0606b59a3e
mount: create mount root on filer
...
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/127
2023-07-07 09:30:32 -07:00
chrislu
14546f0ad8
minor
2023-07-06 09:12:28 -07:00
chrislu
ab30f489c3
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-07-06 00:33:02 -07:00
chrislu
31b2751aff
clone volume locations in case they are changed
...
fix https://github.com/seaweedfs/seaweedfs/issues/4642
2023-07-06 00:32:58 -07:00
wusong
61553beba2
Fix DataBackend nil pointer ( #4641 )
2023-07-04 23:22:10 -07:00
chrislu
a315490f7d
proxy to master uses http address
...
fix https://github.com/seaweedfs/seaweedfs/issues/4607
2023-07-04 11:45:21 -07:00
chrislu
2aa59ab37c
fix copying level db files during commi
...
fix https://github.com/seaweedfs/seaweedfs/issues/4635
2023-07-04 11:28:12 -07:00
wusong
c44a6552d2
Fix buff use ( #4628 )
2023-07-03 08:33:34 -07:00
Amir H. Yeganemehr
f0d3c874b6
Unix Socket listener for S3 server ( #4621 )
...
Add localSocket option to s3 server
2023-06-26 16:22:45 -07:00
chrislu
2c4c2f0994
3.53
2023-06-25 22:51:14 -07:00
chrislu
f193942adc
lock on append metadata operation
2023-06-25 22:07:23 -07:00
chrislu
61c42f9991
adjust lock APIs
2023-06-25 20:30:20 -07:00
chrislu
aafb874ad2
lock until cancelled
2023-06-25 19:31:25 -07:00
chrislu
c4f44d56f7
add distributed lock client
2023-06-25 17:38:34 -07:00
chrislu
f8aa5ea844
adjust filer.proto
2023-06-25 17:38:19 -07:00
chrislu
13a8593ab9
adjust errors
2023-06-25 17:37:54 -07:00
chrislu
06471dac9d
init lock ring
2023-06-25 15:28:16 -07:00
chrislu
5149b3d07b
filer can proxy to peer filer holding the lock
2023-06-25 15:28:01 -07:00
chrislu
868f7875d7
refactor
2023-06-25 14:30:58 -07:00
chrislu
26eff062c8
refactor
2023-06-25 14:17:58 -07:00
chrislu
464a71a373
add distributed lock manager
2023-06-25 14:14:40 -07:00
chrislu
ee4f7cd636
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-06-25 00:58:24 -07:00
chrislu
3fd659df2a
add distributed lock manager
2023-06-25 00:58:21 -07:00
Damiano Albani
0d9ba8c612
Support templating name of files stored via Rclone backend ( #4606 )
2023-06-24 07:08:24 -07:00
chrislu
a82c44972b
refactor
2023-06-20 00:12:59 -07:00
chrislu
14a19170e2
refactor
2023-06-19 23:51:04 -07:00
chrislu
84ced2abf5
add LockRing
2023-06-19 23:48:33 -07:00
chrislu
fe1716ccdf
remove deprecated tests
2023-06-19 18:43:51 -07:00
chrislu
1ce725728d
minor
2023-06-19 18:43:41 -07:00
chrislu
11549706ef
refactor
2023-06-19 18:21:07 -07:00
chrislu
8ec1bc2c99
remove unused cluster node leader
2023-06-19 18:19:13 -07:00
Bai Jie
3b88ab42aa
remove duplicate fileCount query ( #4588 )
2023-06-18 00:14:14 -07:00
Bai Jie
44b9d72ef0
doIsEmpty() return error if v.DataBackend is nil ( #4587 )
2023-06-18 00:13:40 -07:00
chrislu
94fbf02ea2
generate jwt token when looking up needle id
...
fix https://github.com/seaweedfs/seaweedfs/issues/4577
2023-06-17 00:11:23 -07:00
柏杰
0b0fb9b9e4
avoid data race read volume.IsEmpty ( #4574 )
...
* avoid data race read volume.IsEmpty
- avoid phantom read isEmpty for onlyEmpty
- use `v.DataBackend.GetStat()` in v.dataFileAccessLock scope
* add Destroy(onlyEmpty: true) test
* add Destroy(onlyEmpty: false) test
* remove unused `IsEmpty()`
* change literal `8` to `SuperBlockSize`
2023-06-14 14:39:58 -07:00
Konstantin Lebedev
1e22d5caf2
fix get file stats for IsEmpty ( #4576 )
2023-06-14 01:43:30 -07:00
Konstantin Lebedev
4dd890d4a2
optional https port for s3 ( #4482 )
...
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-13 23:58:49 -07:00
chrislu
3fbf4f6189
Revert "Initial patch for redis version 7 support ( #4572 )"
...
This reverts commit a25bca0692
2023-06-12 22:27:38 -07:00
Konstantin Lebedev
4527ead295
fix from comment delete volume is empty ( #4573 )
...
* fix from coments
https://github.com/seaweedfs/seaweedfs/pull/4561
* fix tests
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 22:22:46 -07:00
Aaron Gipson
a25bca0692
Initial patch for redis version 7 support ( #4572 )
2023-06-12 13:30:03 -07:00
Konstantin Lebedev
25535e9c36
Delete volume is empty ( #4561 )
...
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559
* fix IsEmpty
* fix test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 10:42:44 -07:00
柏杰
68feffa4e6
avoid overwriting non-nil uploadReaderToChunks.uploadErr with nil value ( #4560 )
2023-06-10 23:15:27 -07:00
Konstantin Lebedev
5ee04d20fa
Healthz check for deadlocks ( #4558 )
2023-06-09 09:42:48 -07:00
wusong
d6af3a8eb6
fix 404 not found ( #4536 )
...
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-06 09:14:49 -07:00
Konstantin Lebedev
5caff2c0f0
fix: deletes newly created volumes ( #4551 )
...
https://github.com/seaweedfs/seaweedfs/issues/4531
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-06 09:08:27 -07:00
wusong
9bdbf9c880
revert #4491 ( #4550 )
...
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-06 00:17:51 -07:00
chrislu
0bffb3770f
support TTL when creating folders
...
related to https://github.com/seaweedfs/seaweedfs/discussions/4549
2023-06-05 23:29:25 -07:00
wusong
26f15d0079
Fix no more writable volumes by delay judgment ( #4548 )
...
* fix nomore writables volumes while disk free space is sufficient by time delay
* reset
---------
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-05 10:17:21 -07:00
chrislu
fb4b61036c
3.52
2023-06-04 22:34:26 -07:00
Konstantin Lebedev
a0931be0c0
S3 TLS credentials Refreshing ( #4506 )
...
* S3 TLS credentials Refreshing
* fix: logging
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-04 14:27:56 -07:00
wusong
5aec6da8a3
fix deadlock for filer upload ( #4527 )
...
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-03 13:38:27 -07:00
chrislu
e23f3d6eca
set df.File to nil after it is closed
...
possibly fix https://github.com/seaweedfs/seaweedfs/issues/4530
2023-06-02 17:16:45 -07:00
chrislu
ca7cc61319
reduce the window size between unregistering a volume server and creating volumes on that server
...
fix https://github.com/seaweedfs/seaweedfs/issues/4467
2023-05-30 08:20:00 -07:00
chrislu
68c3cf767c
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-05-29 16:45:28 -07:00
chrislu
d1d1757520
adjust error message
2023-05-29 16:45:25 -07:00
wusong
e7f4418855
add return for error handle ( #4502 )
2023-05-24 07:45:06 -07:00
chrislu
4310e1fac4
3.51
2023-05-21 23:43:39 -07:00
wusong
8fffe3e822
fix no more writables volumes while disk free space is sufficient ( #4491 )
...
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-05-21 22:18:50 -07:00
mervynzhang
1ebb549f77
support swift ( #4480 )
2023-05-19 06:39:25 -07:00
Mesar Hameed
a90b777ff4
Feat: etcd filer store keys should have customizable prefix ( #4484 )
...
An etcd cluster is not necessarily only dedicated to seaweedfs.
This security enhancement adds a customizable key_prefix option to the etcd filer store.
This will allow an etcd cluster administrator to limit the seaweedfs etcd user to only read/write a subset of keys under the
key_prefix, instead of all keys on the etcd cluster.
2023-05-18 23:08:56 -07:00
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix ( #4465 )
...
* Use filerGroup for s3 buckets collection prefix
* Fix templates
* Remove flags
* Remove s3CollectionPrefix
2023-05-16 09:39:43 -07:00
chrislu
b7f011f777
fix test
2023-05-15 17:46:56 -07:00
Mesar Hameed
65484e80ef
Feat: support username/password authentication for etcd filer store s… ( #4477 )
...
Feat: support username/password authentication for etcd filer store seaweedfs/seaweedfs#4262
Co-authored-by: Mesar Hameed <mesar.hameed@gmail.com>
2023-05-15 17:45:14 -07:00
chrislu
9204ee2d2d
3.50
2023-05-11 08:03:11 -07:00
chrislu
d999f1f0e2
update raft version
...
fix #4460
2023-05-09 22:54:23 -07:00
chrislu
25b237b83c
reset wait time on a successful retry
2023-05-07 23:39:33 -07:00
chrislu
59f55c1a66
3.49
2023-05-07 14:00:19 -07:00
chrislu
b2a3860290
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-05-07 13:33:47 -07:00
chrislu
4511edc871
update raft
2023-05-07 13:33:44 -07:00
themarkchen
7592d013fe
fix shell volume.balance bug ( #4447 )
2023-05-04 07:35:37 -07:00
chrislu
42766bccc8
3.48
2023-04-30 22:20:18 -07:00
zemul
e9fda774f4
[Filer] post add param:saveInside ( #4434 )
...
* fix:mount deadlock
* feat: filer http upload to metadata
* feat: /etc save inside
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-29 08:31:05 -07:00
wusong
4867aa03ab
Fix: http rename move dir to subdir ( #4432 )
...
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-04-26 20:54:36 -07:00
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion ( #4430 )
2023-04-25 08:31:14 -07:00
zemul
f352616b7d
Volume range read use sync.pool ( #4422 )
2023-04-24 07:00:48 -07:00
chrislu
18686b7375
3.47
2023-04-23 22:45:41 -07:00
zemul
68f1cc34a5
[mount]Fix copy file range ( #4407 )
...
* fix:mount deadlock
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-23 22:34:38 -07:00
chrislu
e1b92bc701
reset local file content if remote file is updated
...
fix https://github.com/seaweedfs/seaweedfs/issues/4289
2023-04-22 22:32:21 -07:00
chrislu
ff7b6d779e
avoid overwriting variables
...
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-22 11:49:59 -07:00
wusong
2e240704ab
Writables inconsistency ( #4417 )
...
fix: inconsistent read and write permissions between master and volume server
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-21 00:14:41 -07:00
wusong
19245dde50
mount: add retry for read only case ( #4416 )
...
* mount: add retry for read only case
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
* add new util retry function for mount
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
* change error list param
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
---------
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-21 00:09:31 -07:00
SmsS4
4131874fa8
use env for flags wtih dot in name ( #4415 )
2023-04-20 09:33:27 -07:00
Damiano Albani
9f55c7c90e
Rclone storage backend ( #4402 )
...
* Add Rclone storage backend
* Support templating the name of files stored via Rclone
* Enable Rclone accounting
* Remove redundant type conversion
* Provide progress information for Rclone download/upload operations
* Log error when Rclone can't instantiate filesystem
* Remove filename templating functionality for Rclone storage
To (maybe) be later reintroduced as a generic functionality for all
storage backends.
* Remove S3 specific check
* Move Rclone config initialisation to init() method
2023-04-18 14:12:12 -07:00
wusong
a408b46d95
compilation fail ( #4414 )
...
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-18 00:36:55 -07:00
chrislu
0a22eea55d
collect ec shard from multiple locations
...
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-17 22:56:21 -07:00
chrislu
ae9388723f
adjust error message
2023-04-17 22:01:39 -07:00
chrislu
8ecdf958ab
3.46
2023-04-16 21:57:43 -07:00
Chengyu Liu
0f653d2153
update fuse client log which should output the fh of FileHandle. ( #4404 )
2023-04-15 22:15:16 -07:00
zemul
0122e022ea
Mount concurrent read ( #4400 )
...
* fix:mount deadlock
* feat: concurrent read
* fix
* Remove useless code
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-13 22:32:45 -07:00
Konstantin Lebedev
5614ad0000
fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter ( #4399 )
...
* fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter
* fix list with delimiter and start token
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 16:53:49 -07:00
Konstantin Lebedev
88b2bbadd1
try fix s3test test_bucket_listv2_delimiter_prefix ( #4396 )
2023-04-12 08:47:20 -07:00
Konstantin Lebedev
095cdb0c7f
fix test prefix is a directory ( #4393 )
...
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 00:08:13 -07:00
Konstantin Lebedev
44ad07276c
s3 fix get list of dir object key with slash suffix ( #4391 )
...
* s3 fix get list of dir object key with slash suffix
https://github.com/seaweedfs/seaweedfs/issues/3086
* list only entry dir eq prefix
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-11 23:52:35 -07:00
Konstantin Lebedev
130bc3e668
s3 fix get fake dir object key ( #4390 )
2023-04-11 07:36:22 -07:00
Matt
da1d3b5a33
adjusted regex to be from 1 to 255 for the value ( #4377 )
...
* compatibility patch for csi driver
* added namespace to all component parameters
* added namespace to all component parameters
* dereference in range
* added namespace to values.yml defaults
* added namespace to s3 component
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* push on all tags
* push on all tags
* push on all tags
* push on all tags
* push on all tags
* changed helm directory structure
* update charts location
* fixed dereference
* updated permissions
* updated permissions
* match current action schema
* added helm chart liniting CI
* modified chart list changed
* fixed nodejs warning
* standardized a few defaults
* added chart dirs
* lowerd period seconds for volume startup to test chart lint changes
* lowerd period seconds for volume startup to test chart lint changes
* test
* changed back
* adjustment
* debug ls statment
* removed change detection
* remvoed change detection
* always lint the charts
* added missing )
* fixed typo
* added spaces in front of all comments
* pdated values.yaml
* pdated values.yaml
* don't validate maintainers for now
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* - adds helm chart lint on changes
- adds test helm chart install on k8s
* updated helm chart readme.md
* added artifact hub
* added artifact hub
* added ttl validation
* adjusted regex to be from 1 to 255 for the value
* added better error message
* fixed regex
2023-04-10 12:24:38 -07:00
Matt
2d213d87e1
File Path Configuration TTL Validation ( #4376 )
...
* compatibility patch for csi driver
* added namespace to all component parameters
* added namespace to all component parameters
* dereference in range
* added namespace to values.yml defaults
* added namespace to s3 component
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* added helm chart to github pages
* push on all tags
* push on all tags
* push on all tags
* push on all tags
* push on all tags
* changed helm directory structure
* update charts location
* fixed dereference
* updated permissions
* updated permissions
* match current action schema
* added helm chart liniting CI
* modified chart list changed
* fixed nodejs warning
* standardized a few defaults
* added chart dirs
* lowerd period seconds for volume startup to test chart lint changes
* lowerd period seconds for volume startup to test chart lint changes
* test
* changed back
* adjustment
* debug ls statment
* removed change detection
* remvoed change detection
* always lint the charts
* added missing )
* fixed typo
* added spaces in front of all comments
* pdated values.yaml
* pdated values.yaml
* don't validate maintainers for now
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* update helm_ci.yml
* - adds helm chart lint on changes
- adds test helm chart install on k8s
* updated helm chart readme.md
* added artifact hub
* added artifact hub
* added ttl validation
2023-04-04 22:07:17 -07:00
chrislu
efef6e94bf
use UPSERT for postgres style databases
2023-04-04 11:52:36 -07:00
chrislu
422bfaed69
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-04-02 22:53:02 -07:00
chrislu
b9ab65dba5
3.45
2023-04-02 22:52:58 -07:00
wusong
2b82d0f871
style: clerical error ( #4361 )
2023-03-31 07:36:21 -07:00
LHHDZ
8e80ce9476
fix key corrupt when fs.configure copy path trie ( #4353 )
...
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-03-29 11:29:44 -07:00
chrislu
3cc7b90414
print wait in progress
2023-03-26 12:22:48 -07:00
chrislu
f1f14e28bc
adjust name
2023-03-26 12:17:23 -07:00
chrislu
d54798c3ae
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-03-25 22:03:00 -07:00
chrislu
6bfb91b539
update entry Attributes.Mtime = time.Now().Unix()
...
fix https://github.com/seaweedfs/seaweedfs/issues/4337
2023-03-25 22:02:57 -07:00
renweijun
bbd461b2c6
weed mount default EnableACL，Support chmod chown ( #4335 )
2023-03-25 04:47:53 -07:00
chrislu
5db9fcccd4
refactoring
2023-03-21 23:01:49 -07:00
Konstantin Lebedev
de4545c28b
add validate config for raft ( #4332 )
2023-03-21 06:36:13 -07:00
chrislu
adb90bd252
avoid lower casing the command
...
fix https://github.com/seaweedfs/seaweedfs/pull/4321
2023-03-19 21:20:46 -07:00
chrislu
5b43c4bb98
3.44
2023-03-19 20:27:55 -07:00
chrislu
50dc2fe96b
cleaning variables
2023-03-19 18:48:40 -07:00
Konstantin Lebedev
29d1312625
Fs.verify.concurrency ( #4293 )
...
* init
* refactor
* refactor print msg
* param just concurrency
* fix race
* next fix race
* reset variables before each run
* clean
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-03-19 18:38:57 -07:00
chrislu
784daea1fa
fix volume not found if marked as read only
...
fix https://github.com/seaweedfs/seaweedfs/issues/4088
2023-03-19 18:30:13 -07:00
Stewart Miles
264be0d2d4
Retry until a leader is selected. ( #4318 )
...
Fixes regression introduced in
https://github.com/seaweedfs/seaweedfs/pull/4313
Related to #4307
2023-03-16 20:50:38 -07:00
LHHDZ
d695119073
The filer does not set defaultReplication to the defaultReplication o… ( #4315 )
...
The filer does not set defaultReplication to the defaultReplication of the master (it is not necessary, because if the filer is not set, the configuration of the master will be used when calling Assign), otherwise, when the defaultReplication of the master is modified and all master nodes are restarted, the defaultReplication will not take effect (because the filer Keep the previous defaultReplication setting from the master)
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-03-15 22:10:39 -07:00
Stewart Miles
57ab1f8516
Use exponential backoff to query leader. ( #4313 )
...
`topology.Leader()` was using a backoff that typically
resulted in at least a 5s delay when initially starting
a master and raft server. This changes the backoff
algorithm to use exponential backoff starting with 100ms
and waiting up to 20s for leader selection.
Related to #4307
2023-03-15 17:49:46 -07:00
Stewart Miles
dd71f54c6b
Fix -raftHashicorp and -raftBootstrap flag propagation. ( #4309 )
...
`weed server` was not correctly propagating
`-master.raftHashicorp` and `-master.raftBootstrap` flags when
starting the master server.
Related to #4307
2023-03-15 13:03:20 -07:00
Patrick Schmidt
71b33faef0
Skip parent directory creation in mount ( #4310 )
...
A POSIX filesystem does not implicitly create parent directories when
they do not exist. Directories must be explicitly created and
permissions be set.
This also fixes a bug where asynchronous operations would create a
file in the filer before the parent directory was created. If the
file was a symlink or another special type of file the directory
would inherit that type and become unusable in the mounted FS.
2023-03-15 12:37:25 -07:00
chrislu
f5854d13df
fix test
2023-03-14 22:05:16 -07:00
chrislu
0454bb2d88
fix test
...
when size is zero, the "LastModified" is not written
2023-03-14 22:02:49 -07:00
chrislu
2268d38263
fix naming
...
fix https://github.com/seaweedfs/seaweedfs/issues/4305
2023-03-14 08:38:02 -07:00
chrislu
98bc93cf31
fix tests
2023-03-13 10:00:27 -07:00
Konstantin Lebedev
e17429223e
shell script unclean variables ( #4298 )
2023-03-13 07:16:31 -07:00
Thomas Anderson
bd70683871
fix(weed/topology/node.go): typo in error message ( #4292 )
2023-03-08 23:58:03 -08:00
Rohit Chormale
59706c89fb
#4270 set http status code to 409 if dir already exists ( #4287 )
2023-03-08 07:45:08 -08:00
chrislu
7111c08289
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-03-08 01:48:30 -08:00
chrislu
c589e9837c
fix tests
2023-03-08 01:48:28 -08:00
Kevin Liu
244385bf0d
Fix binding metrics to ipv6 ( #4286 )
...
* Fix binding metrics to ipv6
* Update weed/stats/metrics.go
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-03-08 01:04:02 -08:00
zemul
bd6f2716d3
fix_read_tomb_needle_meta ( #4263 )
...
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:26:55 -08:00
zemul
0bf56298d5
fix chunk.ModifiedTsNs ( #4264 )
...
* fix
* fix mtime s > ns
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:24:36 -08:00
chrislu
8241c9a829
adjust ui pagination size
...
fix https://github.com/seaweedfs/seaweedfs/issues/4258
2023-02-28 08:31:26 -08:00
lfhy
1976ca9160
add -disk to filer command ( #4247 )
...
* add -disk to filer command
* add diskType to filer.grpc
* use filer.disk when filerWebDavOptions.disk is empty
* add filer.disk to weed server command.
---------
Co-authored-by: 三千院羽 <3000y@MacBook-Pro.lan>
2023-02-25 09:48:59 -08:00
chrislu
214b7cd286
volume.fix.replication: adjust the retry checking times
2023-02-22 10:47:52 -08:00
LHHDZ
db5515eada
Fix s3api_object_list_handlers returning contents less than the specified limit when more data actually exists ( #4240 )
...
Fix when the stored data is actually enough but s3api_object_list_handlers returns less than the specified limit
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-02-21 23:08:52 -08:00
Zachary Walters
ef2f741823
Updated the deprecated ioutil dependency ( #4239 )
2023-02-21 19:47:33 -08:00
chrislu
3227e4175e
3.43
2023-02-20 16:56:05 -08:00
chrislu
9bb0a9e306
clean comments
2023-02-17 01:31:44 -08:00
chrislu
545d5d0cc3
fix for io.ReaderAt used in filer.remote.sync
...
fix https://github.com/seaweedfs/seaweedfs/issues/4194
2023-02-17 01:30:55 -08:00
wusong
79af6b41d4
[mount] Fix read file-data in entry.content ( #4207 )
2023-02-15 07:18:56 -08:00
chrislu
5869945f16
avoid infinite loop
...
fix https://github.com/seaweedfs/seaweedfs/issues/4195#issuecomment-1426100904
2023-02-10 13:05:04 -08:00
chrislu
e037c71ec3
adjust text
2023-02-10 13:04:29 -08:00
chrislu
67b8c2853a
add line return
2023-02-10 12:53:43 -08:00
chrislu
302155a6fa
fix compilation
2023-02-10 12:32:59 -08:00
chrislu
5520b81551
add some error messages for args
...
related to https://github.com/seaweedfs/seaweedfs/issues/4115
2023-02-10 12:30:53 -08:00
chrislu
dc4ed2cd9b
do not move cloud tier volumes
...
fix https://github.com/seaweedfs/seaweedfs/issues/4195
2023-02-09 22:17:54 -08:00
chrislu
31bb91583f
fix bug when vid not found
...
fix https://github.com/seaweedfs/seaweedfs/issues/4193
2023-02-09 17:30:44 -08:00
chrislu
79d68a81fd
text change
2023-02-07 15:00:41 -08:00
chrislu
999a126a3f
typo
2023-02-07 14:59:47 -08:00
chrislu
1c2364936a
detect invalid bucket name
...
fix https://github.com/seaweedfs/seaweedfs/issues/4143
2023-02-07 14:57:29 -08:00
chrislu
5de93fe442
refactoring
2023-02-07 14:45:20 -08:00
Eric Yang
5083429704
ADHOC: s3 bucket name verification ( #4189 )
...
* ADHOC: s3 bucket name verification
* add test
2023-02-07 14:37:29 -08:00
Neo
d5f77706a8
volume:fix return if Replicate write error ( #4188 )
2023-02-06 22:34:56 -08:00
chrislu
8821d6b161
3.42
2023-02-05 21:28:02 -08:00
Muhammad Hallaj bin Subery
9bd422d2c9
adding support for B2 region ( #4177 )
...
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
chrislu
84e9934bf9
fix filer.remote.sync on a S3 cloud mount
...
fix https://github.com/seaweedfs/seaweedfs/issues/4175
2023-02-01 20:44:00 -08:00
chrislu
0932437a1c
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-02-01 02:38:09 -08:00
chrislu
c47f75eb5b
fix null content type caused exception in hdfs client
...
fix https://github.com/seaweedfs/seaweedfs/issues/4170
2023-02-01 02:38:05 -08:00
wusong
fe573f26a8
memchunk save content twice ( #4174 )
2023-01-31 19:36:59 -08:00
Neo
b9b613a78e
filter system log dir does not make subscribe event ( #4172 )
2023-01-31 18:56:11 -08:00
wusong
665917fdf1
[mount]: delete redundant code ( #4173 )
2023-01-31 18:54:46 -08:00
a
0d085ec175
make directory lister trigger if mime type is httpd/unix-directory ( #4169 )
2023-01-31 06:54:40 -08:00
chrislu
4d71af87f3
3.41
2023-01-29 22:56:01 -08:00
chrislu
ef70415c70
skipping manifest chunk volume id change
2023-01-29 22:55:06 -08:00
chrislu
6e8dbf8868
add fs.meta.changeVolumeId
2023-01-29 22:51:58 -08:00
wusong
de081c0d64
[mount] fix metacache update ( #4161 )
2023-01-29 07:55:27 -08:00
Guo Lei
b63d1e9c6b
fix index ( #4158 )
2023-01-28 06:55:27 -08:00
chrislu
81fdf3651b
grpc connection to filer add sw-client-id header
2023-01-20 01:48:12 -08:00
chrislu
bfe5d910c6
use one readerCache for the whole file
2023-01-16 22:43:02 -08:00
chrislu
4614e85efa
adjust help message
2023-01-15 21:28:36 -08:00
chrislu
c24678153c
use memory chunks only for sequential writes
2023-01-15 21:10:08 -08:00
chrislu
2885ba0e50
3.40
2023-01-15 20:12:05 -08:00
chrislu
12f8251d32
avoid index out of slice range
2023-01-14 13:31:22 -08:00
chrislu
313fd17ae4
allow random writes for mem chunk
2023-01-13 20:32:18 -08:00
Chris Lu
1cd2e64aac
merge chunks during upload ( #4130 )
...
* merge chunks during upload
* fix test
2023-01-12 23:07:36 -08:00
chrislu
2452f3b2f7
sort chunks by offset
2023-01-12 20:29:07 -08:00
chrislu
361d97941b
mysql: Missed escaping column name
...
fix https://github.com/seaweedfs/seaweedfs/issues/4129
2023-01-11 08:46:56 -08:00
chrislu
efcffe4f77
no need to set chunks during flush
2023-01-10 10:01:52 -08:00
chrislu
85f1b70207
reduce group size from 256MB to 64MB
2023-01-10 01:35:44 -08:00
chrislu
86657ed3a8
avoid repeatedly adding manifest chunks
2023-01-10 01:35:10 -08:00
chrislu
28fe578944
avoid possible nil entry
2023-01-10 01:07:34 -08:00
chrislu
47fa5ef979
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-01-10 00:52:50 -08:00
chrislu
9e0d526163
fix compilation
2023-01-10 00:52:05 -08:00
chrislu
7f49c59c14
cluster.ps add filer meta sync progress
2023-01-10 00:51:25 -08:00
chrislu
340e7c3a2e
chunk group remove manifestChunks and reset sections in SetChunks()
2023-01-10 00:49:31 -08:00
chrislu
e650c8397d
ensure section.chunks is always garbage free
2023-01-10 00:48:01 -08:00
chrislu
75bdd4a0d1
refactor
2023-01-10 00:46:46 -08:00
Guo Lei
83e29faa55
use time.NewTicker instead of time.Tick. ( #4119 )
2023-01-09 06:47:08 -08:00
chrislu
51d4a4b28d
3.39
2023-01-08 18:20:22 -08:00
chrislu
0b52f08226
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-01-08 18:03:25 -08:00
chrislu
469c959e6a
add missing options for webdav
2023-01-08 18:03:22 -08:00
monchickey
3e2c9ea73d
Add image cropping. ( #4117 )
2023-01-06 09:28:07 -08:00
chrislu
296fdc296c
mount: faster add chunks
2023-01-06 01:03:29 -08:00
chrislu
2abf817580
fix for stream reader
...
fix https://github.com/seaweedfs/seaweedfs/issues/4112
2023-01-05 11:19:21 -08:00
chrislu
e367444586
add notes for full version
2023-01-04 09:52:25 -08:00
chrislu
5423790b2c
add back previous chunk upload selection algo
2023-01-03 22:50:39 -08:00
chrislu
7bdae5172e
batch delete EC needles
...
fix https://github.com/seaweedfs/seaweedfs/issues/4107
2023-01-03 22:05:26 -08:00
zemul
6b4c033431
add mount log ( #4101 )
...
* filer.backup use replication.source.filer
* add mount log
* Revert "filer.backup use replication.source.filer"
This reverts commit 07bf6f956c
2023-01-03 00:00:45 -08:00
chrislu
48f2edc065
3.38
2023-01-02 23:23:02 -08:00
Chris Lu
d4566d4aaa
more solid weed mount ( #4089 )
...
* compare chunks by timestamp
* fix slab clearing error
* fix test compilation
* move oldest chunk to sealed, instead of by fullness
* lock on fh.entryViewCache
* remove verbose logs
* revert slat clearing
* less logs
* less logs
* track write and read by timestamp
* remove useless logic
* add entry lock on file handle release
* use mem chunk only, swap file chunk has problems
* comment out code that maybe used later
* add debug mode to compare data read and write
* more efficient readResolvedChunks with linked list
* small optimization
* fix test compilation
* minor fix on writer
* add SeparateGarbageChunks
* group chunks into sections
* turn off debug mode
* fix tests
* fix tests
* tmp enable swap file chunk
* Revert "tmp enable swap file chunk"
This reverts commit 985137ec47
2023-01-02 23:20:45 -08:00
CommanderRoot
c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 ( #4094 )
2023-01-01 05:07:53 -08:00
CommanderRoot
c2280e94cf
filer.store.mysql: Replace deprecated upsert syntax ( #4096 )
2023-01-01 05:06:57 -08:00
CommanderRoot
265a56630b
filer.store.mysql: Escape table columns in SQL query ( #4095 )
2023-01-01 05:06:41 -08:00
Guo Lei
ed5f3f073b
add more help message, in case of misunderstanding ( #4092 )
2022-12-29 07:05:05 -08:00
Guo Lei
d8cfa1552b
support enable/disable vacuum ( #4087 )
...
* stop vacuum
* suspend/resume vacuum
* remove unused code
* rename
* rename param
2022-12-28 01:36:44 -08:00
Guo Lei
5f993a5f20
Fix ide warning ( #4082 )
...
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2022-12-26 00:03:47 -08:00
Konstantin Lebedev
569bc7b54d
fs.verify needles status of file ( #4060 )
...
* fs.verify needles status of file
* fix parse args
* refactor logging
* set modifyTimeAgoAtSec
2022-12-21 15:54:17 -08:00
chrislu
77b7c9f7cc
lock and unlock chunks for read
2022-12-21 14:15:35 -08:00
chrislu
d1797deccb
remove duplicated logic of removing from writableChunks
2022-12-21 14:15:11 -08:00
chrislu
6ede19e825
add a simple file replication progress bar
2022-12-20 19:47:21 -08:00
chrislu
f7beba8515
unlock before submitting the uploading jobs
2022-12-20 12:52:30 -08:00
chrislu
28d479e5c0
mount: adjust locking for upload pipeline
2022-12-19 15:07:22 -08:00
chrislu
f9fb3cd1fc
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-19 11:32:00 -08:00
chrislu
6c7fe40305
filer sink retries reading file chunks, skipping missing chunks
...
if the file chunk is not available during replication time, the file is skipped
2022-12-19 11:31:58 -08:00
chrislu
3fc3e7083c
metadata skip reading un-available logs
...
this means the volumes for metadata histories have been lost.
2022-12-19 11:30:50 -08:00
lfhy
038bab3726
add -filer.path to webdav command ( #4061 )
2022-12-17 20:07:02 -08:00
chrislu
8a40fa8993
more detailed logs
2022-12-17 13:18:35 -08:00
LHHDZ
f84c13dabd
fix response not allowed ( #4057 )
2022-12-13 05:48:50 -08:00
LHHDZ
bb289d7d69
sort tags by key ( #4047 )
...
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-12-11 22:49:57 -08:00
chrislu
438146249f
3.37
2022-12-11 21:08:06 -08:00
chrislu
ac9dea0ad9
rotate log files
2022-12-11 21:05:58 -08:00
chrislu
fc6b9e6e0c
volume server logs add url info
...
debug https://github.com/seaweedfs/seaweedfs/issues/3964
2022-12-11 13:15:38 -08:00
chrislu
38c63c429b
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-10 21:49:09 -08:00
chrislu
e1ca6308cb
add chunk etag when downloading from remote storage
...
fix https://github.com/seaweedfs/seaweedfs/issues/3987
2022-12-10 21:49:07 -08:00
wusong
549354e324
Fix hardlink counting ( #4042 )
...
Signed-off-by: wusong <wangwusong@virtaitech.com>
Signed-off-by: wusong <wangwusong@virtaitech.com>
Co-authored-by: wusong <wangwusong@virtaitech.com>
2022-12-08 10:50:57 -08:00
Jiffs Maverick
5f59d43c88
[filer] Fix uri escape during renaming ( #4039 )
2022-12-07 07:47:40 -08:00
chrislu
659f28b855
simple refactoring
2022-12-05 23:19:38 -08:00
chrislu
ca49ef8ed8
duplicated logic
2022-12-05 23:19:22 -08:00
Chris Lu
dac9c28d05
Revert "refactor: moved to locked entry" ( #4035 )
...
* Revert "refactor: moved to locked entry"
This reverts commit 94bc9afd9d
2022-12-05 12:32:27 -08:00
chrislu
fad7e1f7cb
3.36
2022-12-04 23:34:29 -08:00
chrislu
94bc9afd9d
refactor: moved to locked entry
2022-12-04 23:33:05 -08:00
aronneagu
2b783738d6
Update Makefile - use binary in current directory ( #4027 )
2022-12-01 14:09:14 -08:00
aronneagu
77699855a7
Return ETag from remote when file doesn't exist on Filer ( #4025 )
2022-11-30 07:43:30 -08:00
Konstantin Lebedev
f59c3acd5f
volume.list show volumes only from the specified dc/rack/dn ( #4024 )
2022-11-30 06:30:55 -08:00
Konstantin Lebedev
2b910d1cf8
avoid recursive deleting newly created empty directories ( #4016 )
2022-11-25 08:45:47 -08:00
Jiffs Maverick
4b0430e71d
[metrics] Add the ability to control bind ip ( #4012 )
2022-11-24 10:22:59 -08:00
famosss
15284e2bee
fix: Lost Content-Encoding metadata ( #4010 )
2022-11-23 22:52:34 -08:00
James Hartig
81624de27b
Include name/mime in ReadAllNeedles ( #4005 )
2022-11-23 15:59:38 -08:00
Eric Yang
26d9b662b1
ADHOC: fix fs.meta.save typo ( #3993 )
...
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-11-20 21:05:00 -08:00
James Hartig
4c85da7844
Include meta in ReadAllNeedles ( #3991 )
...
This is useful for doing backups on the data so we can accurately store the
last modified time, the compression state, and verify the crc.
Previously we were doing VolumeNeedleStatus and then an HTTP request which
needlessly read from the dat file twice.
2022-11-20 20:19:41 -08:00
chrislu
70a4c98b00
refactor filer_pb.Entry and filer.Entry to use GetChunks()
...
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
b1d7963e57
3.35
2022-11-14 00:21:00 -08:00
Guo Lei
5b905fb2b7
Lazy loading ( #3958 )
...
* types packages is imported more than onece
* lazy-loading
* fix bugs
* fix bugs
* fix unit tests
* fix test error
* rename function
* unload ldb after initial startup
* Don't load ldb when starting volume server if ldbtimeout is set.
* remove uncessary unloadldb
* Update weed/command/server.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* Update weed/command/volume.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: guol-fnst <goul-fnst@fujitsu.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-11-14 00:19:27 -08:00
chrislu
d90aa31d5f
filer store: skip disabled location specific filer store initialization
...
fix https://github.com/seaweedfs/seaweedfs/issues/3971
2022-11-13 23:30:23 -08:00
chrislu
7ac40ae3d4
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-11-13 23:01:58 -08:00
chrislu
aae82dca6f
file name escape uri
...
fix https://github.com/seaweedfs/seaweedfs/issues/3969
2022-11-13 23:01:56 -08:00
LHHDZ
39f0246deb
fix 'NoSuchUpload' when upload part ( #3968 )
2022-11-11 04:53:53 -08:00
famosss
df0662038c
feat:add a uuid after uploadid ( #3963 )
2022-11-10 07:17:29 -08:00
aronneagu
180853a2c9
Replace dashes with underscores in x-amz-meta headers ( #3965 )
2022-11-10 07:09:53 -08:00
chrislu
47db75a695
3.34
2022-11-06 23:06:22 -08:00
Konstantin Lebedev
5431c445cd
fix filer.remote.sync to azure with ContentType ( #3949 )
...
* fix filer.remote.sync to azure with ContentType
* fix pass X-Amz-Meta to X-Ms-Meta
2022-11-04 09:10:33 -07:00
zemul
cec55feeb0
[filer.backup] add retention_duration ( #3941 )
2022-11-04 00:34:40 -07:00
chrislu
4193dafce1
azure metadata: skip metadata prefixed with "X-"
...
fix https://github.com/seaweedfs/seaweedfs/issues/3875
2022-11-02 21:42:02 -07:00
pavelzhurov
abe4a61659
Bug fix: empty key in DeleteMultipleObjects request caused bucket delete ( #3939 )
2022-11-02 05:51:13 -07:00
Konstantin Lebedev
0999f9b7ff
[volume.fsck] collect ids without cut off time for finding missing data from volumes ( #3934 )
...
collect all file ids from the file without cut off time for finding missing data from volumes
2022-10-31 11:38:12 -07:00
Konstantin Lebedev
a322ba042e
[volume.fsck] param volumeId is comma separated the volume id ( #3933 )
...
volume.fsck param volumeId is comma separated the volume id
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-10-31 11:36:26 -07:00
Konstantin Lebedev
4a48332248
refactor error contains already deleted ( #3932 )
2022-10-31 11:34:45 -07:00
Konstantin Lebedev
c0deaa4948
[volume.fsck] check needles status from volume server ( #3926 )
...
check needles status from volume server
2022-10-31 11:33:04 -07:00
