Commit Graph

24 Commits

Author SHA1 Message Date
chrislu
49869eec83 edge cases for broker assignment 2024-03-07 10:45:38 -08:00
chrislu
fe03b1b522 handle single broker case 2024-03-01 05:59:40 -08:00
chrislu
50c5dd7313 test low active brokers 2024-02-29 20:50:57 -08:00
chrislu
7598922b41 assign followers 2024-02-29 14:51:06 -08:00
chrislu
1b4484bf0a go fmt 2024-02-29 09:38:52 -08:00
Chris Lu
0d74ac9224 fix divide by zero 2024-02-04 13:38:00 -08:00
chrislu
428fea45f3 updated and added assignments 2024-01-22 11:04:24 -08:00
chrislu
e8b05ecc91 add/remove assigned partitions 2024-01-22 10:47:39 -08:00
chrislu
b0a2e9aea3 fix assignments if brokers changed 2024-01-21 01:27:22 -08:00
chrislu
9ce7c482b3 refactor 2024-01-21 01:24:12 -08:00
chrislu
34839237ab refactor 2024-01-20 12:16:40 -08:00
chrislu
2828ccbb30 refactor 2024-01-20 11:41:11 -08:00
chrislu
be0c426dc7 simplify to LookupTopicPartitions(topic) 2024-01-16 09:30:46 -08:00
chrislu
db3670a3a5 simplify api 2024-01-16 08:55:47 -08:00
chrislu
34a78ffad0 remove isForPublish from LookupTopicBrokers
also adds a return parameter: whether the topic exists or not
2024-01-16 08:52:42 -08:00
chrislu
f782165638 rename 2024-01-16 08:40:45 -08:00
chrislu
dd1ec70e62 rename 2024-01-16 08:40:23 -08:00
chrislu
6a7a679137 passing partition generation timestamp 2024-01-10 22:36:17 -08:00
chrislu
35869b5c80 subscriber can be notified of the assignment change when topic is just configured
Next: Subscriber needs to read by the timestamp offset.
2024-01-03 13:30:30 -08:00
chrislu
efb695fd93 lookup existing topic partitions 2024-01-02 17:29:35 -08:00
chrislu
093fdc1621 subscriber can get assignments 2023-12-28 20:35:15 -08:00
chrislu
e0727071c8 go fmt 2023-12-22 11:33:50 -08:00
chrislu
61b05fad95 fix test 2023-12-11 20:57:29 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue (#5098)
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
2023-12-11 12:05:54 -08:00