438 Commits

Author	SHA1	Message	Date
Chris Lu	e63317fb08	ec deletion code complete, not tested yet	2019-06-20 00:55:30 -07:00
Chris Lu	4cea8aefd0	add grpc VolumeEcBlobDelete	2019-06-20 00:17:11 -07:00
Chris Lu	856da7aae2	ec volume support deletes	2019-06-19 22:57:14 -07:00
Chris Lu	d8ed73926d	volume servers get metrics address and interval from the master	2019-06-17 14:51:47 -07:00
Chris Lu	ff6a6dd11e	refactoring	2019-06-15 21:46:55 -07:00
Chris Lu	8b43679ae3	refactoring	2019-06-15 12:21:44 -07:00
Chris Lu	5336008dcd	refactoring	2019-06-14 13:06:01 -07:00
Chris Lu	d67189921f	go fmt	2019-06-14 00:55:12 -07:00
Chris Lu	5f6c9825f8	volume server: adds basic metrics	2019-06-14 00:54:56 -07:00
Chris Lu	a11525fe4e	filer: adds basic metrics pushing to Prometheus gateway	2019-06-13 02:01:54 -07:00
Chris Lu	57092d69ac	go fmt	2019-06-10 21:33:32 -07:00
Chris Lu	e66bddd84c	customizable sleep intervals	2019-06-06 00:39:08 -07:00
Chris Lu	50aa769554	jwt for read access control	2019-06-06 00:29:02 -07:00
Chris Lu	d344e0a035	fix ec related bugs	2019-06-05 23:20:26 -07:00
Chris Lu	450f4733ce	report empty volume or ec shards	2019-06-05 13:32:33 -07:00
Chris Lu	7f6fb9ffd0	every 17 minutes instead of seconds ... TODO: make this configurable	2019-06-05 13:32:08 -07:00
Chris Lu	ede876cfdb	periodic scripts exeuction from leader master	2019-06-05 01:30:24 -07:00
Chris Lu	ca8a2bb534	go fmt	2019-06-04 22:04:10 -07:00
Chris Lu	2215e81be7	ui add ec shard statuses	2019-06-04 21:52:37 -07:00
Chris Lu	b05456fe07	able to purge extra ec shard copies	2019-06-03 20:25:02 -07:00
Chris Lu	11cffb3168	fix ec.rebuild bugs	2019-06-03 11:50:54 -07:00
Chris Lu	d85b41b904	fix ec.encode not finding the local ec shards	2019-06-03 10:38:21 -07:00
Chris Lu	7e80b2b882	fix multiple bugs	2019-06-03 02:26:31 -07:00
Chris Lu	55be09996d	fix volume balance bug	2019-06-03 00:13:31 -07:00
Chris Lu	133b772fb5	destroy ec volume if it is empty	2019-06-01 01:51:28 -07:00
Chris Lu	ba18314aab	ec shard delete also check ec volumes, in addition to volumes	2019-06-01 01:41:22 -07:00
Chris Lu	f53024d79d	delete ec collection	2019-05-30 10:40:22 -07:00
Chris Lu	1d111d6ce8	collection list normal and ec volumes	2019-05-30 09:27:23 -07:00
Chris Lu	e941d0c2f4	list ec collections also	2019-05-30 09:17:58 -07:00
Chris Lu	3f9ecee40f	working with reading remote intervals	2019-05-28 21:29:07 -07:00
Chris Lu	2858a954b3	read ec volume shards locations from master	2019-05-28 00:13:13 -07:00
Chris Lu	713596e781	caching ec shard locations	2019-05-27 22:54:58 -07:00
Chris Lu	217cde0a3b	refactoring	2019-05-27 21:40:51 -07:00
Chris Lu	03b9291e5d	volum server passes grpc option and master location to store	2019-05-27 21:22:23 -07:00
Chris Lu	ac1ba3b667	remove comments	2019-05-27 21:21:52 -07:00
Chris Lu	b4b407e403	add grpc ec shard read	2019-05-27 11:59:03 -07:00
Chris Lu	a4f3d82c57	convert needle id to ec intervals to read from	2019-05-27 01:29:46 -07:00
Chris Lu	92f9c7b1aa	adjust ec shard status on disconnect	2019-05-26 01:05:08 -07:00
Chris Lu	b58e25e588	broadcast vid->location map to master clients for ec chards	2019-05-26 00:49:15 -07:00
Chris Lu	af67d99ca4	incrementally update master ec shards state	2019-05-26 00:21:17 -07:00
Chris Lu	db94a41f9e	mount/unmount ec shards	2019-05-25 23:23:19 -07:00
Chris Lu	41e8ae61f8	generate, balance, delete copied shards, delete old volume	2019-05-25 14:02:06 -07:00
Chris Lu	6f4b09b6a4	pb shard info uses ShardBits instead one message for one shard	2019-05-24 13:28:44 -07:00
Chris Lu	24b8ad9a14	push the ec shard info to master	2019-05-23 23:47:49 -07:00
Chris Lu	4659d80035	prepare to register ec shard info in master	2019-05-23 00:42:28 -07:00
Chris Lu	17ac1290c0	volume: load ec shards during heartbeats to master	2019-05-21 22:41:20 -07:00
Chris Lu	fbbc74abb4	adds VolumeEcGenerateSlices, VolumeEcCopy	2019-05-20 00:53:17 -07:00
Chris Lu	698dea779c	refactoring	2019-05-19 21:37:49 -07:00
Chris Lu	017d0957c3	avoid switching master node if same ip and host name ... fix https://github.com/chrislusf/seaweedfs/issues/955	2019-05-11 18:42:46 -07:00
Chris Lu	8ffdd5dcdb	writel throttling moving volumes	2019-05-06 13:58:42 -07:00
Chris Lu	3ff92037cc	shell: add volume.balance command	2019-05-05 21:17:23 -07:00
Chris Lu	25941e0500	master: add jwt expires_after_seconds	2019-05-04 08:42:25 -07:00
Chris Lu	bd8af92b54	remove unused code	2019-05-04 08:20:22 -07:00
Chris Lu	5681208f2d	refactor: remove logger	2019-05-03 22:21:06 -07:00
Chris Lu	b335f81a4f	volume: add option to limit compaction speed	2019-05-03 17:22:39 -07:00
Chris Lu	f0f981e7c8	webdav: fix directory renaming	2019-05-03 14:12:51 -07:00
Chris Lu	bfbecd7253	webdav: fix directory creation	2019-05-03 13:13:11 -07:00
Chris Lu	9d3c945b00	Merge pull request #950 from chrislusf/webdav ... add WebDav	2019-05-03 01:25:10 -07:00
Chris Lu	7064b2ea48	webdav: can write now	2019-05-03 00:55:52 -07:00
Chris Lu	b30c14b631	webdav: can read now	2019-05-03 00:24:35 -07:00
Chris Lu	47f14775d7	scaffolding, but seems hard to satisfy mac, windows, and different os versions ... https://www.reddit.com/r/golang/comments/3wrxo3/webdav_server_that_provide_filesystem_using/	2019-05-02 14:22:10 -07:00
Wine93	32f93fb09e	style: go fmt for all	2019-04-30 03:22:19 +00:00
Chris Lu	108d0fb08d	adjust log level	2019-04-26 09:32:37 -07:00
Chris Lu	00b6f653fa	shell: add fs.cat	2019-04-21 15:43:43 -07:00
Chris Lu	440111a349	volume: support http status 304 for the same file id	2019-04-21 13:33:23 -07:00
Chris Lu	0302b9496c	volume: also check cookie for possible duplicates	2019-04-21 13:31:45 -07:00
Chris Lu	132921ad41	adjust log or error	2019-04-21 10:14:17 -07:00
Chris Lu	2ffe98443b	refactoring	2019-04-20 12:05:28 -07:00
Chris Lu	6fc1f53018	shell: add command volume.move	2019-04-20 11:35:23 -07:00
Chris Lu	3b3651dea3	volume: atomic copying file, adds version and stopOffset	2019-04-19 12:29:49 -07:00
Chris Lu	ac2727853f	fix needle map entry size	2019-04-19 00:39:34 -07:00
Chris Lu	e5506152c0	refactoring	2019-04-18 21:43:36 -07:00
Chris Lu	fa176fe80f	volume tailing chunks large files	2019-04-18 19:22:13 -07:00
Chris Lu	3dce1016cb	add volume tailer	2019-04-18 11:05:02 -07:00
Chris Lu	b142f9f1d5	go fmt	2019-04-18 00:19:18 -07:00
Chris Lu	b09e8dbb37	add VolumeStreamFollow, but not used yet	2019-04-18 00:18:29 -07:00
Chris Lu	13ad5c1966	refactoring	2019-04-17 22:04:49 -07:00
Chris Lu	3b9446606a	adjust log level	2019-04-16 23:31:44 -07:00
Chris Lu	338e6d60a5	refactor: prepare for snapshotting	2019-04-16 09:55:37 -07:00
Chris Lu	79c2cca9c1	better error message	2019-04-16 01:15:30 -07:00
Chris Lu	6e116b3d67	volume: reset leader if error happens	2019-04-15 09:09:46 -07:00
Chris Lu	e85048bcdc	http exhaust and close response body	2019-04-14 23:28:24 -07:00
Chris Lu	3e8a3a8fec	fix race detector found problems	2019-04-14 23:00:37 -07:00
Chris Lu	0e33272f56	small refactoring	2019-04-10 23:39:53 -07:00
Chris Lu	cb2061f86a	Merge pull request #925 from stlpmo-jn/checkUnhealthReplication ... the implement of repair the error replications of the volume	2019-04-10 23:37:12 -07:00
Chris Lu	2f76681d62	refactor: adjust grpc API name	2019-04-10 21:41:17 -07:00
stlpmo-jn	f2031884f0	fix bug : CI build failed	2019-04-11 09:53:31 +08:00
stlpmo-jn	c1a0403da2	repair the error replications of the volume	2019-04-10 19:41:55 +08:00
Chris Lu	000ee725fc	refactor Offset into a struct of bytes	2019-04-08 19:40:56 -07:00
Chris Lu	766396d249	weed master: atomic volume counting ... possible fix for https://github.com/chrislusf/seaweedfs/issues/913	2019-04-04 19:27:00 -07:00
Chris Lu	189c890715	weed replicate: replicate atomic rename to other systems	2019-03-31 08:10:47 -07:00
Chris Lu	78ac2bef3c	go fmt	2019-03-30 23:09:16 -07:00
Chris Lu	97406333a5	support atomic renaming for mysql/postgres filer store	2019-03-30 23:08:29 -07:00
Chris Lu	c7e7b6229f	detect mime type	2019-03-27 14:25:18 -07:00
Chris Lu	65757ae2fd	weed filer: set content-type consistent with filer store	2019-03-27 11:41:11 -07:00
Chris Lu	7a14cdc90c	refactoring, go fmt	2019-03-25 23:18:40 -07:00
Chris Lu	19728fe3f6	remove deprecated code	2019-03-25 23:12:14 -07:00
Chris Lu	df95ce0b6c	weed backup: efficient delta backup ... fix https://github.com/chrislusf/seaweedfs/issues/399	2019-03-25 23:01:53 -07:00
Chris Lu	c913583e2d	Merge branch 'master' into add_volume_follow	2019-03-25 21:19:34 -07:00
Chris Lu	a32797518b	weed master: redirect to leader for default admin UI ... fix https://github.com/chrislusf/seaweedfs/issues/898	2019-03-25 09:20:05 -07:00