89 Commits

Author	SHA1	Message	Date
Chris Lu	f88a8bda7b	ec deletion works	2019-06-21 01:14:10 -07:00
Chris Lu	4cea8aefd0	add grpc VolumeEcBlobDelete	2019-06-20 00:17:11 -07:00
Chris Lu	856da7aae2	ec volume support deletes	2019-06-19 22:57:14 -07:00
Chris Lu	115558e5f5	adjust counters	2019-06-17 21:02:50 -07:00
Chris Lu	289fd7eb39	count number of volumes and ec shards	2019-06-16 02:24:15 -07:00
Chris Lu	d344e0a035	fix ec related bugs	2019-06-05 23:20:26 -07:00
Chris Lu	ca8a2bb534	go fmt	2019-06-04 22:04:10 -07:00
Chris Lu	2215e81be7	ui add ec shard statuses	2019-06-04 21:52:37 -07:00
Chris Lu	0e52862586	avoid the test that depends on ordering in a different folder	2019-06-04 02:22:32 -07:00
Chris Lu	11cffb3168	fix ec.rebuild bugs	2019-06-03 11:50:54 -07:00
Chris Lu	7e80b2b882	fix multiple bugs	2019-06-03 02:26:31 -07:00
Chris Lu	ba18314aab	ec shard delete also check ec volumes, in addition to volumes	2019-06-01 01:41:22 -07:00
Chris Lu	47f1901843	ask for the ec volume version	2019-05-31 00:58:51 -07:00
Chris Lu	40ca2f2903	add collection.delete	2019-05-30 09:47:54 -07:00
Chris Lu	5c6c757619	clean up test files	2019-05-29 00:34:30 -07:00
Chris Lu	5dd67f9acf	reading by recover from other shards	2019-05-28 23:48:39 -07:00
Chris Lu	3f9ecee40f	working with reading remote intervals	2019-05-28 21:29:07 -07:00
Chris Lu	4f76342cbc	WIP ... no errors, but not returning file content * the interval needs to use actual file zie * need to read the actual version instead of the current version	2019-05-28 00:51:01 -07:00
Chris Lu	713596e781	caching ec shard locations	2019-05-27 22:54:58 -07:00
Chris Lu	3a8c1055a2	refactoring ecx to ecVolume	2019-05-27 22:00:36 -07:00
Chris Lu	217cde0a3b	refactoring	2019-05-27 21:40:51 -07:00
Chris Lu	b4b407e403	add grpc ec shard read	2019-05-27 11:59:03 -07:00
Chris Lu	a4f3d82c57	convert needle id to ec intervals to read from	2019-05-27 01:29:46 -07:00
Chris Lu	af67d99ca4	incrementally update master ec shards state	2019-05-26 00:21:17 -07:00
Chris Lu	db94a41f9e	mount/unmount ec shards	2019-05-25 23:23:19 -07:00
Chris Lu	f0e6574d5e	allocate ec shards to volume servers	2019-05-25 02:02:44 -07:00
Chris Lu	6f4b09b6a4	pb shard info uses ShardBits instead one message for one shard	2019-05-24 13:28:44 -07:00
Chris Lu	228850d588	shard id starts from zero	2019-05-24 11:52:23 -07:00
Chris Lu	8a96445f40	register ec shards to each data node	2019-05-23 22:51:18 -07:00
Chris Lu	4659d80035	prepare to register ec shard info in master	2019-05-23 00:42:28 -07:00
Chris Lu	7180520889	ec shard info can be queried via VolumeList()	2019-05-22 22:44:28 -07:00
Chris Lu	17ac1290c0	volume: load ec shards during heartbeats to master	2019-05-21 22:41:20 -07:00
Chris Lu	fbbc74abb4	adds VolumeEcGenerateSlices, VolumeEcCopy	2019-05-20 00:53:17 -07:00
Chris Lu	693d6c0f1b	clean up tests	2019-05-19 21:16:01 -07:00
Chris Lu	6386a3174b	able to validate by randomly selected ec files	2019-05-19 14:24:33 -07:00
Chris Lu	7c2c60c376	add locating data inside the ec files	2019-05-19 03:01:58 -07:00
Chris Lu	87f63b9c08	generate ec01~ec14, generate ecx file with sorted needle values	2019-05-18 22:46:24 -07:00
Chris Lu	8156958ee9	move function to make travis happy	2019-05-15 10:02:44 -07:00
Chris Lu	0a36f628c6	testing RS coding	2019-05-15 01:02:00 -07:00