184 Commits

Author	SHA1	Message	Date
chrislu	25b2850e7d	refactor out FilerClientAccessor	2024-06-06 19:44:19 -07:00
chrislu	d1f0c40402	remove per-message debug logs	2024-05-30 09:49:08 -07:00
chrislu	ff14dfa0ee	debug	2024-05-30 09:15:50 -07:00
chrislu	b485b9695c	fix passing sub follower offset	2024-05-30 09:15:23 -07:00
chrislu	af19256dbf	skip control messages	2024-05-29 23:34:39 -07:00
chrislu	ad2b878130	fix inflight message tracker	2024-05-29 23:33:37 -07:00
chrislu	edc87c16f9	balance subscribers ... need to ensure the offsets are persisted	2024-05-27 17:30:16 -07:00
chrislu	cdeaaf95b4	go fmt	2024-05-23 08:25:53 -07:00
chrislu	d40b350f54	coordinator receives unassignment ack	2024-05-23 08:23:35 -07:00
chrislu	554ae09f82	rename	2024-05-21 10:02:07 -07:00
chrislu	67e6051585	rename Coordinator to SubCoordinator	2024-05-21 09:57:45 -07:00
chrislu	d5abffa42c	rename Balancer to PubBalancer	2024-05-21 09:56:30 -07:00
chrislu	3148dec1f6	avoid concurrent processing for the same key	2024-05-20 13:29:37 -07:00
chrislu	65dd5ac6fb	fix	2024-05-20 12:27:45 -07:00
chrislu	3622196881	go fmt	2024-05-20 11:05:18 -07:00
chrislu	5038577f7e	Merge branch 'master' into mq	2024-05-20 11:04:53 -07:00
chrislu	d218fe54fa	go fmt	2024-05-20 11:03:56 -07:00
chrislu	7a4ace5dfb	adjust logs	2024-05-20 09:34:07 -07:00
chrislu	d8ab22012a	track offset	2024-05-20 09:33:37 -07:00
chrislu	8ce2cced47	less logs	2024-05-19 14:54:46 -07:00
chrislu	c21b32dd20	logs	2024-05-19 11:02:17 -07:00
chrislu	b1871427c3	persist consumer group offset ... 1. use one follower 2. read write consumer group offset	2024-05-19 00:46:12 -07:00
chrislu	94742b8ace	add subscribe follower	2024-05-17 18:17:23 -07:00
chrislu	279cb2b85e	consumer acks received messages	2024-05-16 20:28:19 -07:00
chrislu	faffb2973c	adjust errors	2024-05-16 11:02:48 -07:00
chrislu	972e9faaa2	move initial assignment to rebalance logic	2024-05-14 09:09:36 -07:00
chrislu	6e5075e14e	move read write topic config into filer client accessor	2024-05-14 08:50:17 -07:00
chrislu	2142842f82	refactor	2024-05-13 18:25:38 -07:00
chrislu	53d1d2b78a	save schema when configuring topic	2024-04-12 22:29:53 -07:00
chrislu	ccdd9cd8de	refactor	2024-04-02 16:25:43 -07:00
chrislu	f37c0d0d7a	comment out println	2024-04-02 15:31:31 -07:00
chrislu	d935f70e3c	shutdown follower when leader stops	2024-04-01 22:32:30 -07:00
chrislu	f07875e8e1	send flush message to follower before shutting down logBuffer	2024-04-01 16:01:26 -07:00
chrislu	e568e742c9	renaming	2024-04-01 15:02:52 -07:00
chrislu	66a878af39	add control message to data message	2024-03-31 16:35:46 -07:00
chrislu	c9df613b6b	add publisher name for debugging	2024-03-31 00:19:16 -07:00
chrislu	0d55bddc75	logs	2024-03-30 20:37:46 -07:00
chrislu	ab9fc0ad70	simplify	2024-03-27 23:42:19 -07:00
chrislu	dab5454332	publish and send to follower	2024-03-27 23:10:24 -07:00
chrislu	059a120708	refactor	2024-03-27 22:58:20 -07:00
chrislu	7d87c1d2bb	refactor	2024-03-27 22:48:37 -07:00
chrislu	5cc94a05b9	separate goroutine to send ack to publisher	2024-03-27 10:27:08 -07:00
chrislu	17806cde2a	separate context ... If using the same context, the downstream grpc would be cancelled when the parent grpc is cancelled.	2024-03-27 10:24:57 -07:00
chrislu	f67c8309d8	rename	2024-03-26 21:52:12 -07:00
chrislu	d6a066495b	adjust logs	2024-03-25 08:15:47 -07:00
chrislu	1b1b96ea1e	close local partition if no more publishers and subscribers	2024-03-24 21:10:02 -07:00
chrislu	4f5c4c3388	refactor	2024-03-24 13:07:51 -07:00
chrislu	d558a58946	refactor	2024-03-24 13:04:59 -07:00
chrislu	205d2285f3	setup follower by publisher ... * the subscriber would getOrGen a local partition and wait * the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.	2024-03-24 12:57:09 -07:00
chrislu	7003ce7425	publisher is able to fully send data and receive acks ... still needs to close the pipes cleanly	2024-03-20 12:25:40 -07:00