97 Commits

Author	SHA1	Message	Date
chrislu	9a5cc065b0	fix	2024-05-30 08:29:06 -07:00
chrislu	23a72db1df	stop partitionOffsetChan if closed	2024-05-30 00:27:44 -07:00
chrislu	af19256dbf	skip control messages	2024-05-29 23:34:39 -07:00
chrislu	f8a22038c5	passing along publisher name	2024-05-29 23:31:48 -07:00
chrislu	edc87c16f9	balance subscribers ... need to ensure the offsets are persisted	2024-05-27 17:30:16 -07:00
chrislu	cdeaaf95b4	go fmt	2024-05-23 08:25:53 -07:00
chrislu	d40b350f54	coordinator receives unassignment ack	2024-05-23 08:23:35 -07:00
chrislu	fa98ecf71e	client side stop partition subscribing if unassigned	2024-05-21 08:42:04 -07:00
chrislu	6634b42981	fix compilation	2024-05-21 08:26:55 -07:00
chrislu	2190d59bc3	refactor ... ToDo 1. handle unassignment on client side 2. get notified of completed unassignment 3. in consumer_group.go, balance and send unassignment	2024-05-21 08:05:56 -07:00
chrislu	a7d30d0705	refactor	2024-05-20 12:32:12 -07:00
chrislu	33aecb7bb4	remove ProcessorConfiguration	2024-05-20 12:28:01 -07:00
chrislu	f13475ac2e	sending keyed offset	2024-05-20 11:09:48 -07:00
chrislu	d45b1d058d	minor	2024-05-20 11:07:54 -07:00
chrislu	3622196881	go fmt	2024-05-20 11:05:18 -07:00
chrislu	5038577f7e	Merge branch 'master' into mq	2024-05-20 11:04:53 -07:00
chrislu	d218fe54fa	go fmt	2024-05-20 11:03:56 -07:00
chrislu	d8ab22012a	track offset	2024-05-20 09:33:37 -07:00
chrislu	a2885512e1	fix	2024-05-19 15:06:33 -07:00
chrislu	8d02712c63	ConcurrentPartitionLimit	2024-05-19 14:55:03 -07:00
chrislu	c6db3f31a1	start consuming ASAP	2024-05-19 14:52:38 -07:00
chrislu	b1871427c3	persist consumer group offset ... 1. use one follower 2. read write consumer group offset	2024-05-19 00:46:12 -07:00
chrislu	279cb2b85e	consumer acks received messages	2024-05-16 20:28:19 -07:00
chrislu	1f20178ded	subscriber receives partitions and dispatch to processors	2024-05-14 23:22:43 -07:00
chrislu	372bd8d71d	consumer instance passing MaxPartitionCount to coordinator	2024-05-13 18:30:10 -07:00
chrislu	8cc5298a08	clean up unused variables	2024-05-10 13:10:45 -07:00
chrislu	664b704e30	add delay	2024-05-05 14:17:41 -07:00
chrislu	4e3701df77	reset last error	2024-05-05 14:16:54 -07:00
chrislu	7a9b115cc2	rename	2024-05-02 11:14:58 -07:00
chrislu	4a3e8869bb	subscriber can unmarshal the record	2024-05-02 09:56:11 -07:00
chrislu	1c5043baf7	rename	2024-05-02 09:47:29 -07:00
chrislu	f2f4630029	rename field types	2024-05-02 08:59:22 -07:00
chrislu	b8af997e41	consistent with protobuf data types	2024-05-02 08:52:10 -07:00
chrislu	0f35b3a4ea	add example to publish a record	2024-05-02 08:48:51 -07:00
chrislu	928a4e8dff	rename	2024-05-02 08:35:06 -07:00
chrislu	ce2b2fa9b2	Merge branch 'master' into mq-subscribe	2024-04-15 20:09:14 -07:00
yudrywet	8833745fbc	chore: fix some typos in comments (#5497)	2024-04-14 07:42:40 -07:00
chrislu	68ade92d4f	publisher send schema	2024-04-12 23:36:15 -07:00
chrislu	5324e4288a	rename parameter	2024-04-12 22:33:00 -07:00
chrislu	66a878af39	add control message to data message	2024-03-31 16:35:46 -07:00
chrislu	546ae87c39	send is_close message to broker	2024-03-31 01:28:40 -07:00
chrislu	c9df613b6b	add publisher name for debugging	2024-03-31 00:19:16 -07:00
chrislu	205d2285f3	setup follower by publisher ... * the subscriber would getOrGen a local partition and wait * the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.	2024-03-24 12:57:09 -07:00
chrislu	da31e9b939	correctly wait for the publisher to finish	2024-03-21 09:08:53 -07:00
chrislu	7003ce7425	publisher is able to fully send data and receive acks ... still needs to close the pipes cleanly	2024-03-20 12:25:40 -07:00
chrislu	33ab6cfcf7	adjust error	2024-03-07 10:44:09 -08:00
chrislu	ead27a4f59	publish init add followers	2024-02-29 09:39:12 -08:00
chrislu	1b4484bf0a	go fmt	2024-02-29 09:38:52 -08:00
Chris Lu	16b4f9d9ab	subscriber find broker leader first	2024-02-05 23:14:25 -08:00
Chris Lu	90fcde0e26	report EOF error	2024-02-05 17:47:34 -08:00