Chris Lu
580940bf82
Merge accumulated changes related to message queue ( #5098 )
...
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
2023-12-11 12:05:54 -08:00
chrislu
deb86ff4a6
upgrading grpc
2023-10-12 21:38:34 -07:00
chrislu
de0b969b36
Revert "rename"
...
This reverts commit 35b5264ab7
2023-10-12 20:28:11 -07:00
Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file ( #4881 )
...
* fix: disallow file name too long when writing a file
* bool LongerName to MaxFilenameLength
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 14:29:55 -07:00
chrislu
35b5264ab7
rename
2023-10-11 21:44:56 -07:00
Konstantin Lebedev
2b3e39397e
fix: skipping checking active volumes with the same number of files at the moment ( #4893 )
...
* fix: skipping checking active volumes with the same number of files at the moment
https://github.com/seaweedfs/seaweedfs/issues/4140
* refactor with comments
https://github.com/seaweedfs/seaweedfs/issues/4140
* add TestShouldSkipVolume
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-09 09:57:26 -07:00
chrislu
b02fdeabff
RetryForever => RetryUntil
2023-10-01 11:33:56 -07:00
chrislu
ebd6f96d35
adjust mq.proto
2023-09-30 21:47:57 -07:00
chrislu
358cba43ef
update proto generated files
2023-09-30 13:19:25 -07:00
chrislu
d7c211fe8c
adjust mq proto
2023-09-30 13:18:49 -07:00
chrislu
9d589b48e6
rename function
2023-09-26 15:17:33 -07:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration ( #4859 )
...
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface
https://github.com/seaweedfs/seaweedfs/issues/4519
* fix: test bucket acl default and
adjust the variable names
* fix: s3 api config test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-25 08:34:12 -07:00
chrislu
3cf9b8d621
create topic and report topic
2023-09-24 22:00:43 -07:00
chrislu
b3f94feede
do create topic
2023-09-24 22:00:43 -07:00
chrislu
0361c321b4
add CreateTopic API
2023-09-24 22:00:43 -07:00
chrislu
6c6655de9d
wip
2023-09-19 14:03:42 -07:00
chrislu
4827425146
balancer works
2023-09-16 15:05:38 -07:00
chrislu
436d99443b
receive broker stats
2023-09-14 23:49:05 -07:00
chrislu
b771fefa37
Merge branch 'master' into sub
2023-09-13 00:03:47 -07:00
chrislu
2492a45499
ack interval
2023-09-06 22:39:46 -07:00
chrislu
ba67e6ca29
api for sub
2023-09-04 21:43:50 -07:00
chrislu
cb470d44df
can pub and sub
2023-09-01 00:36:51 -07:00
chrislu
504ae8383a
protoc version
2023-08-28 09:01:25 -07:00
chrislu
dbcba75271
rename to lookup
2023-08-27 18:59:04 -07:00
chrislu
4d6c18d86f
pub sub initial tests
2023-08-27 17:50:59 -07:00
chrislu
1990456670
sub
2023-08-27 13:13:14 -07:00
chrislu
905911853d
adjust proto
2023-08-26 13:39:21 -07:00
Nico D'Cotta
796b7508f3
Implement SRV lookups for filer ( #4767 )
2023-08-24 07:08:56 -07:00
chrislu
010c5e91e3
add stream assign proto
2023-08-22 09:53:54 -07:00
chrislu
01d70c21f3
Squashed commit of the following:
...
commit 32f4b1a13057d56b6de487cdb80ff7c205af01a6
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 22:52:19 2023 -0700
fix compilation
commit e77ad33b7ca0423138fbae26a4433b60923a9588
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 22:46:44 2023 -0700
pub
commit f431f30cc7ca277ca299e3cd118c05537fb9f5c3
Author: chrislu <chris.lu@gmail.com>
Date: Sun Aug 20 13:27:39 2023 -0700
fix generic type
commit 4e9dcb18293fd1e3e306e2dceb995dfd67a35e1d
Merge: 30f942580 16e3f2d52
2023-08-20 22:53:05 -07:00
chrislu
f8aa5ea844
adjust filer.proto
2023-06-25 17:38:19 -07:00
chrislu
5149b3d07b
filer can proxy to peer filer holding the lock
2023-06-25 15:28:01 -07:00
chrislu
464a71a373
add distributed lock manager
2023-06-25 14:14:40 -07:00
chrislu
3fd659df2a
add distributed lock manager
2023-06-25 00:58:21 -07:00
chrislu
8ec1bc2c99
remove unused cluster node leader
2023-06-19 18:19:13 -07:00
Konstantin Lebedev
25535e9c36
Delete volume is empty ( #4561 )
...
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559
* fix IsEmpty
* fix test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 10:42:44 -07:00
wusong
26f15d0079
Fix no more writable volumes by delay judgment ( #4548 )
...
* fix nomore writables volumes while disk free space is sufficient by time delay
* reset
---------
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-05 10:17:21 -07:00
mervynzhang
1ebb549f77
support swift ( #4480 )
2023-05-19 06:39:25 -07:00
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion ( #4430 )
2023-04-25 08:31:14 -07:00
chrislu
5db9fcccd4
refactoring
2023-03-21 23:01:49 -07:00
Muhammad Hallaj bin Subery
9bd422d2c9
adding support for B2 region ( #4177 )
...
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
chrislu
81fdf3651b
grpc connection to filer add sw-client-id header
2023-01-20 01:48:12 -08:00
Guo Lei
d8cfa1552b
support enable/disable vacuum ( #4087 )
...
* stop vacuum
* suspend/resume vacuum
* remove unused code
* rename
* rename param
2022-12-28 01:36:44 -08:00
chrislu
e1ca6308cb
add chunk etag when downloading from remote storage
...
fix https://github.com/seaweedfs/seaweedfs/issues/3987
2022-12-10 21:49:07 -08:00
James Hartig
81624de27b
Include name/mime in ReadAllNeedles ( #4005 )
2022-11-23 15:59:38 -08:00
James Hartig
4c85da7844
Include meta in ReadAllNeedles ( #3991 )
...
This is useful for doing backups on the data so we can accurately store the
last modified time, the compression state, and verify the crc.
Previously we were doing VolumeNeedleStatus and then an HTTP request which
needlessly read from the dat file twice.
2022-11-20 20:19:41 -08:00
chrislu
70a4c98b00
refactor filer_pb.Entry and filer.Entry to use GetChunks()
...
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
ea2637734a
refactor filer proto chunk variable from mtime to modified_ts_ns
2022-10-28 12:53:19 -07:00
Eric Yang
51d462f204
ADHOC: volume fsck using append at ns ( #3906 )
...
* ADHOC: volume fsck using append at ns
* nit
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-10-24 22:09:38 -07:00
chrislu
de286fe662
shell: volume.move handles volume moved to cloud tier
...
fix https://github.com/seaweedfs/seaweedfs/issues/3803
2022-10-16 17:52:22 -07:00
