Commit Graph

33 Commits

Author SHA1 Message Date
chrislu
7a9b115cc2 rename 2024-05-02 11:14:58 -07:00
chrislu
4a3e8869bb subscriber can unmarshal the record 2024-05-02 09:56:11 -07:00
chrislu
1c5043baf7 rename 2024-05-02 09:47:29 -07:00
chrislu
f2f4630029 rename field types 2024-05-02 08:59:22 -07:00
chrislu
b8af997e41 consistent with protobuf data types 2024-05-02 08:52:10 -07:00
chrislu
0f35b3a4ea add example to publish a record 2024-05-02 08:48:51 -07:00
chrislu
928a4e8dff rename 2024-05-02 08:35:06 -07:00
chrislu
ce2b2fa9b2 Merge branch 'master' into mq-subscribe 2024-04-15 20:09:14 -07:00
yudrywet
8833745fbc
chore: fix some typos in comments (#5497) 2024-04-14 07:42:40 -07:00
chrislu
5324e4288a rename parameter 2024-04-12 22:33:00 -07:00
chrislu
546ae87c39 send is_close message to broker 2024-03-31 01:28:40 -07:00
chrislu
c9df613b6b add publisher name for debugging 2024-03-31 00:19:16 -07:00
chrislu
1b4484bf0a go fmt 2024-02-29 09:38:52 -08:00
Chris Lu
16b4f9d9ab subscriber find broker leader first 2024-02-05 23:14:25 -08:00
chrislu
dedfd31dfb refactor 2024-01-28 12:30:08 -08:00
chrislu
cbf750a31f refactor 2024-01-28 12:23:20 -08:00
chrislu
b51dfe2bff wait for publishing clients 2024-01-28 12:06:58 -08:00
chrislu
841fafd0a8 publish to input buffer
currently, the input buffer may not exist when start to publish
2024-01-27 23:43:22 -08:00
chrislu
7e6497cc1c adjust publisher subscriber 2024-01-15 20:42:46 -08:00
chrislu
026c54a9bb fix publisher 2024-01-15 00:20:21 -08:00
chrislu
49428a303b add batch index for each memory buffer 2024-01-08 00:03:08 -08:00
chrislu
ddd0fde094 rename functions 2024-01-05 15:16:01 -08:00
chrislu
47a4963d7a subscription start from specified timestamp 2024-01-03 15:57:36 -08:00
chrislu
35869b5c80 subscriber can be notified of the assignment change when topic is just configured
Next: Subscriber needs to read by the timestamp offset.
2024-01-03 13:30:30 -08:00
chrislu
458ddbf919 clean up dead code 2023-12-31 13:25:33 -08:00
chrislu
c950a40aad subscriber keep connected to the balancer 2023-12-28 11:56:37 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue (#5098)
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
2023-12-11 12:05:54 -08:00
chrislu
2a578b9033 refactor 2023-10-01 11:59:19 -07:00
chrislu
39941edc0b add publisher shutdown 2023-09-07 23:55:19 -07:00
chrislu
ba67e6ca29 api for sub 2023-09-04 21:43:50 -07:00
chrislu
9e4f985698 publish, benchmark 2023-09-04 21:43:30 -07:00
chrislu
cb470d44df can pub and sub 2023-09-01 00:36:51 -07:00
chrislu
1eb2da46d5 connect and publish 2023-08-28 09:02:12 -07:00