76 Commits

Author	SHA1	Message	Date
chrislu	b1871427c3	persist consumer group offset ... 1. use one follower 2. read write consumer group offset	2024-05-19 00:46:12 -07:00
chrislu	279cb2b85e	consumer acks received messages	2024-05-16 20:28:19 -07:00
chrislu	1f20178ded	subscriber receives partitions and dispatch to processors	2024-05-14 23:22:43 -07:00
chrislu	372bd8d71d	consumer instance passing MaxPartitionCount to coordinator	2024-05-13 18:30:10 -07:00
chrislu	8cc5298a08	clean up unused variables	2024-05-10 13:10:45 -07:00
chrislu	664b704e30	add delay	2024-05-05 14:17:41 -07:00
chrislu	4e3701df77	reset last error	2024-05-05 14:16:54 -07:00
chrislu	7a9b115cc2	rename	2024-05-02 11:14:58 -07:00
chrislu	4a3e8869bb	subscriber can unmarshal the record	2024-05-02 09:56:11 -07:00
chrislu	1c5043baf7	rename	2024-05-02 09:47:29 -07:00
chrislu	f2f4630029	rename field types	2024-05-02 08:59:22 -07:00
chrislu	b8af997e41	consistent with protobuf data types	2024-05-02 08:52:10 -07:00
chrislu	0f35b3a4ea	add example to publish a record	2024-05-02 08:48:51 -07:00
chrislu	928a4e8dff	rename	2024-05-02 08:35:06 -07:00
chrislu	ce2b2fa9b2	Merge branch 'master' into mq-subscribe	2024-04-15 20:09:14 -07:00
yudrywet	8833745fbc	chore: fix some typos in comments (#5497)	2024-04-14 07:42:40 -07:00
chrislu	68ade92d4f	publisher send schema	2024-04-12 23:36:15 -07:00
chrislu	5324e4288a	rename parameter	2024-04-12 22:33:00 -07:00
chrislu	66a878af39	add control message to data message	2024-03-31 16:35:46 -07:00
chrislu	546ae87c39	send is_close message to broker	2024-03-31 01:28:40 -07:00
chrislu	c9df613b6b	add publisher name for debugging	2024-03-31 00:19:16 -07:00
chrislu	205d2285f3	setup follower by publisher ... * the subscriber would getOrGen a local partition and wait * the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.	2024-03-24 12:57:09 -07:00
chrislu	da31e9b939	correctly wait for the publisher to finish	2024-03-21 09:08:53 -07:00
chrislu	7003ce7425	publisher is able to fully send data and receive acks ... still needs to close the pipes cleanly	2024-03-20 12:25:40 -07:00
chrislu	33ab6cfcf7	adjust error	2024-03-07 10:44:09 -08:00
chrislu	ead27a4f59	publish init add followers	2024-02-29 09:39:12 -08:00
chrislu	1b4484bf0a	go fmt	2024-02-29 09:38:52 -08:00
Chris Lu	16b4f9d9ab	subscriber find broker leader first	2024-02-05 23:14:25 -08:00
Chris Lu	90fcde0e26	report EOF error	2024-02-05 17:47:34 -08:00
Chris Lu	07028a4864	minor	2024-02-05 17:30:20 -08:00
Chris Lu	4dc6681833	proxy requests to lock owner	2024-02-05 16:46:40 -08:00
chrislu	0b2e5ddc7c	wait 3 seconds before shutting down publish client, to wait for all messages to be received	2024-01-28 22:04:42 -08:00
chrislu	545d5fbdf6	unused code	2024-01-28 22:03:13 -08:00
chrislu	a297247137	wait for each publish clients	2024-01-28 14:27:32 -08:00
chrislu	b6c5e57c30	read will block if no items	2024-01-28 13:10:34 -08:00
chrislu	0bf5424a2e	adjust visibility	2024-01-28 13:09:30 -08:00
chrislu	dedfd31dfb	refactor	2024-01-28 12:30:08 -08:00
chrislu	cbf750a31f	refactor	2024-01-28 12:23:20 -08:00
chrislu	b51dfe2bff	wait for publishing clients	2024-01-28 12:06:58 -08:00
chrislu	f8c55f101e	clean up	2024-01-28 12:06:30 -08:00
chrislu	841fafd0a8	publish to input buffer ... currently, the input buffer may not exist when start to publish	2024-01-27 23:43:22 -08:00
chrislu	91af1f3069	schedule jobs	2024-01-26 14:09:57 -08:00
chrislu	34a78ffad0	remove isForPublish from LookupTopicBrokers ... also adds a return parameter: whether the topic exists or not	2024-01-16 08:52:42 -08:00
chrislu	7e6497cc1c	adjust publisher subscriber	2024-01-15 20:42:46 -08:00
chrislu	026c54a9bb	fix publisher	2024-01-15 00:20:21 -08:00
chrislu	3aa3991f0f	adjust client side logs	2024-01-11 23:08:53 -08:00
chrislu	6a7a679137	passing partition generation timestamp	2024-01-10 22:36:17 -08:00
chrislu	49428a303b	add batch index for each memory buffer	2024-01-08 00:03:08 -08:00
chrislu	d0d24f1e40	rename functions	2024-01-05 17:10:43 -08:00
chrislu	aa5b6a8187	adjust logs	2024-01-05 15:35:32 -08:00