449 Commits

Author	SHA1	Message	Date
Chris Lu	b8f32bcab9	filer: compress stored metadata	2020-09-03 11:00:20 -07:00
Chris Lu	dd52037c27	1.92	2020-08-30 22:56:35 -07:00
Chris Lu	a41588279a	change log level 5 to 4	2020-08-30 20:12:04 -07:00
Chris Lu	b69cb74c03	read meta logs by timestamp ... pass in event ts when moving logs meta aggregator reads in memory logs only	2020-08-29 17:37:19 -07:00
Chris Lu	ca658a97c5	add signatures to messages to avoid double processing	2020-08-28 23:48:48 -07:00
Chris Lu	0be6863c87	rename	2020-08-26 22:40:15 -07:00
Chris Lu	5f55a87101	close http response	2020-08-24 00:32:44 -07:00
Chris Lu	81a8aa6581	1.91	2020-08-23 17:28:40 -07:00
Chris Lu	aee27ccbe1	multiple fixes ... * adjust isOpen count * move ContinuousDirtyPages lock to filehandle * fix problem with MergeIntoVisibles, avoid reusing slices * let filer delete the garbage	2020-08-23 15:48:02 -07:00
Chris Lu	99d05f758c	adjust logs	2020-08-18 23:39:18 -07:00
Chris Lu	6a92f0bc7a	refactoring to typed Size ... Go is amazing with refactoring!	2020-08-18 17:04:28 -07:00
Chris Lu	09e126bae5	refactoring: use interface	2020-08-17 20:20:08 -07:00
Chris Lu	be4d42b8e2	rename	2020-08-17 20:15:53 -07:00
Chris Lu	97e54a80d4	rename variables	2020-08-17 16:05:13 -07:00
Chris Lu	ff200398bb	1.90	2020-08-16 21:18:50 -07:00
Chris Lu	b71df82292	1.89	2020-08-16 00:57:28 -07:00
Chris Lu	003d48da21	adjust logs	2020-08-15 19:55:28 -07:00
Chris Lu	e74dc4e4bc	add back fs node cache for renaming	2020-08-09 21:56:09 -07:00
Chris Lu	b056707770	1.88	2020-08-08 22:55:12 -07:00
Chris Lu	20e2ac1add	filer: store md5 metadata for files uploaded by filer ... fix https://github.com/chrislusf/seaweedfs/issues/1412	2020-08-06 10:04:17 -07:00
Chris Lu	1b3a80dd3d	non-fatal error	2020-08-01 13:46:52 -07:00
Chris Lu	bbbea8159c	http request use gzip if possible	2020-08-01 13:20:52 -07:00
Chris Lu	2f03481cb2	in case when content is not compressed	2020-08-01 13:08:10 -07:00
Chris Lu	a6b59d50f7	1.87	2020-07-30 13:52:04 -07:00
Chris Lu	7a7e70b6cc	1.86	2020-07-20 23:39:58 -07:00
Chris Lu	befb396892	mount: resolve home directory ... fix https://github.com/chrislusf/seaweedfs/issues/1391	2020-07-16 22:38:55 -07:00
Chris Lu	ddec7b2bb9	go fmt	2020-07-14 07:34:16 -07:00
Chris Lu	5a4f258a8c	1.85	2020-07-12 17:34:09 -07:00
Chris Lu	224103e13b	aggregated logs has empty flushFn	2020-07-11 09:12:03 -07:00
Chris Lu	b813fac4a3	1.84	2020-06-28 07:28:21 -07:00
Chris Lu	f0f884edd5	1.83	2020-06-27 15:46:59 -07:00
Chris Lu	bef356ce4c	since we already know the chunk size, no need to iterate	2020-06-27 12:51:04 -07:00
Chris Lu	a808b3b5df	incase the memory data is too small	2020-06-27 11:59:15 -07:00
Chris Lu	3dbd51c3c2	a little bit more efficient	2020-06-26 10:02:37 -07:00
Chris Lu	212b6e7d42	error logging	2020-06-26 10:01:55 -07:00
Chris Lu	48b23f2fdd	FUSE mount: prevent concurrent modification	2020-06-26 10:00:48 -07:00
Chris Lu	3cec4b3c49	FUSE mount: fix file id written twice? ... fix https://github.com/chrislusf/seaweedfs/issues/1373	2020-06-25 20:11:49 -07:00
Chris Lu	5be12eea37	zstd fix	2020-06-24 11:39:12 -07:00
Chris Lu	fe60db404a	prepare for zstd	2020-06-23 09:12:02 -07:00
Chris Lu	3080c197e3	rename UnCompressData to DecompressData	2020-06-20 08:16:16 -07:00
Chris Lu	a735fd767c	adjust uncompress data	2020-06-20 08:01:00 -07:00
Chris Lu	e912fd15e3	renaming	2020-06-19 22:45:27 -07:00
Chris Lu	c48b407053	1.82	2020-06-19 13:47:27 -07:00
Chris Lu	c0283eee1a	more test cases	2020-06-19 09:45:42 -07:00
Chris Lu	55b6efb755	fix checking visited nodes	2020-06-19 09:45:42 -07:00
Chris Lu	546709c1de	check has visited by the boundary	2020-06-18 22:28:17 -07:00
Chris Lu	0e3b970c0c	added bounded tree to track exploration boundary	2020-06-18 21:09:58 -07:00
Chris Lu	95a9686eaf	1.81	2020-06-10 13:00:42 -07:00
Chris Lu	bb3e86fbd5	auto create non-existing cache directory ... fix https://github.com/chrislusf/seaweedfs/issues/1350	2020-06-08 08:49:57 -07:00
Chris Lu	b8e346cc86	1.80	2020-06-05 15:29:25 -07:00
Chris Lu	45a9d852d3	go fmt	2020-06-05 15:27:10 -07:00
Chris Lu	bc2ec6774d	inject git version into build	2020-06-02 00:10:38 -07:00
bingoohuang	1f8782a1ed	try showing the first 100 volume ids and an extra ...	2020-05-29 16:15:33 +08:00
bingoohuang	1a642b9876	add Volume Ids column only for max 100 volumes for convenience in the master ui.	2020-05-29 15:37:58 +08:00
bingoohuang	accb4964b7	util.BytesToHumanReadable	2020-05-29 10:00:07 +08:00
Chris Lu	ed9e4dc749	1.79 ... fix compilation problem	2020-05-19 08:21:51 -07:00
Chris Lu	85aeead3d2	1.78	2020-05-17 20:28:04 -07:00
Chris Lu	759cda0fe2	log	2020-05-16 18:47:41 -07:00
Chris Lu	4b7fa31468	ensure montonically increasing tsNs	2020-05-11 01:53:54 -07:00
Chris Lu	39e72fb23c	go fmt	2020-05-10 03:50:30 -07:00
Chris Lu	6bf3eb69cb	async chan write read, no write for closed chan	2020-05-10 03:48:35 -07:00
Chris Lu	dfccc3c263	able to read chan and write chan	2020-05-08 02:47:22 -07:00
Chris Lu	f9b6178b8f	log messages	2020-04-30 03:05:34 -07:00
Chris Lu	4e16a90454	for debugging	2020-04-30 02:19:32 -07:00
Chris Lu	5a0986dca0	reset on getting the buffer	2020-04-29 02:41:01 -07:00
wuyuxiang	6850d28d6b	refacotr: move signal handling and pprof to grace package	2020-04-28 14:10:23 +08:00
Chris Lu	f4fcde05b6	1.77	2020-04-26 05:54:47 -07:00
Chris Lu	369aa8a10a	1.76	2020-04-22 23:31:59 -07:00
Chris Lu	72c5dba4d4	1.75	2020-04-21 22:03:07 -07:00
Chris Lu	cb3985be70	go fmt	2020-04-20 17:48:06 -07:00
Chris Lu	5eb83dfbd3	add memory buffer size limit	2020-04-20 17:43:50 -07:00
Chris Lu	258fba8a0f	continue for reading from sealed memory buffer	2020-04-20 17:28:18 -07:00
Chris Lu	4bf959edf0	message broker: read also from sealed memory buffer	2020-04-20 17:26:38 -07:00
Chris Lu	bd43c62fbd	lower the test criteria	2020-04-20 03:08:10 -07:00
Chris Lu	c3f9fdf48f	adjust test	2020-04-20 02:54:21 -07:00
Chris Lu	5e8bb86995	close the connection with better accouting	2020-04-20 02:53:50 -07:00
Chris Lu	2955b96ef1	refactor	2020-04-19 23:54:32 -07:00
Chris Lu	ce3cb25cfb	working for in memory single log buffer	2020-04-19 23:37:50 -07:00
Chris Lu	e0f5996560	fix "call of Unmarshal passes non-pointer as second argument"	2020-04-17 10:04:41 -07:00
Chris Lu	f5a748d33c	refactoring	2020-04-16 02:55:09 -07:00
Chris Lu	23715af402	Merge pull request #1273 from wyxloading/config_race ... Fix data race about config	2020-04-13 22:24:51 -07:00
Chris Lu	2b5c4fbbf3	tiered caching ... 1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files	2020-04-13 22:19:27 -07:00
Chris Lu	f282ed444b	refactoring	2020-04-13 21:58:10 -07:00
wuyuxiang	b556d3d035	Fix data race about config	2020-04-14 10:22:40 +08:00
Chris Lu	5c9e91393f	1.74	2020-04-13 10:25:48 -07:00
Chris Lu	7764e0465c	refactoring	2020-04-12 21:00:55 -07:00
Chris Lu	94e35cdb35	mount: fix ... fix bug found by git bisect, but I do not understand why it can cause error!	2020-04-12 03:34:36 -07:00
Chris Lu	2a1f396df5	avoid duplicated setting chunks into cache	2020-04-12 01:06:50 -07:00
Chris Lu	b9b7da905e	handle nil chunk cache	2020-04-12 01:00:12 -07:00
Chris Lu	df97da25f9	mount: add on disk caching	2020-04-11 21:12:41 -07:00
Chris Lu	d7f3acb2c0	refactor	2020-04-11 12:45:24 -07:00
Chris Lu	ed54682598	refactoring	2020-04-11 12:43:17 -07:00
Chris Lu	e909b55633	refactoring	2020-04-11 12:39:46 -07:00
Chris Lu	43c62a8a95	1.73	2020-04-09 21:44:30 -07:00
Chris Lu	59f40e2027	volume: best effort to detect ip address ... fix https://github.com/chrislusf/seaweedfs/issues/1264	2020-04-09 00:26:24 -07:00
Chris Lu	f6a7e79dc3	weed shell: simplify CLI option for filer	2020-04-08 23:57:15 -07:00
Chris Lu	8764bdb9df	1.72	2020-04-08 22:35:14 -07:00
Chris Lu	00b993a234	add util for md5	2020-04-08 22:30:38 -07:00
Chris Lu	b51d8f03bb	1.71	2020-04-05 22:04:16 -07:00
Chris Lu	91da7057b1	refactoring	2020-04-05 13:11:43 -07:00
Chris Lu	7bc3c93512	add util.PathJoin	2020-04-05 12:40:46 -07:00
Chris Lu	50a5018b7f	writing meta logs is working	2020-03-30 01:19:33 -07:00
Chris Lu	be6c2f8511	1.70	2020-03-29 13:40:55 -07:00
Chris Lu	057722bbf4	return part of the chunk if chunkview is not the full chunk	2020-03-29 00:54:39 -07:00
Chris Lu	005b4ab3fe	1.69	2020-03-27 04:51:31 -07:00
Chris Lu	5463560912	small test	2020-03-26 23:50:07 -07:00
Chris Lu	0a05052b5c	1.68	2020-03-26 22:22:36 -07:00
Chris Lu	dd5b582d05	go fmt	2020-03-26 00:09:01 -07:00
Chris Lu	2e4fadd10a	fix fscache move	2020-03-25 22:19:19 -07:00
Chris Lu	c0f0fdb3ba	refactoring	2020-03-23 00:01:34 -07:00
Chris Lu	d6412f27a0	logging	2020-03-22 18:32:56 -07:00
Chris Lu	3b3e063f25	1.67	2020-03-20 23:42:49 -07:00
Chris Lu	d3e4a31058	filer: fix where deletion can miss under high concurrency	2020-03-20 23:39:32 -07:00
Chris Lu	4bc19f1e52	master: fix connections count reporting	2020-03-20 22:41:37 -07:00
Chris Lu	e1b5ac5c04	1.66	2020-03-20 20:39:48 -07:00
Chris Lu	165b0d22a4	1.65	2020-03-20 00:05:42 -07:00
Chris Lu	db9854fa0a	1.64	2020-03-17 10:58:35 -07:00
Chris Lu	cc52e8ca87	1.63	2020-03-14 14:25:07 -07:00
Chris Lu	91e0a987e4	1.62	2020-03-14 01:03:59 -07:00
Chris Lu	e2e691d9c2	clean up, add test	2020-03-13 23:53:15 -07:00
Chris Lu	60f5f05c78	1.61	2020-03-10 13:52:49 -07:00
Chris Lu	e6de42f888	1.60	2020-03-09 22:32:40 -07:00
Chris Lu	c32f95c380	1.59	2020-03-08 22:02:34 -07:00
Chris Lu	2e3f6ad3a9	filer: remember content is gzipped or not	2020-03-08 21:39:33 -07:00
Chris Lu	ea1169dc80	filer cipher: single chunk http POST and PUT and read	2020-03-07 06:08:08 -08:00
Chris Lu	13e215ee5c	filer: option to encrypt data on volume server	2020-03-06 00:49:47 -08:00
Chris Lu	f90c43635d	refactoring	2020-03-04 00:39:47 -08:00
Chris Lu	4532640ffd	1.58	2020-03-01 17:59:12 -08:00
Chris Lu	ed0acd1722	go fmt	2020-02-26 16:52:57 -08:00
Chris Lu	37a3628b2e	grpc add PermitWithoutStream to client and server options	2020-02-26 16:51:38 -08:00
Chris Lu	892e726eb9	avoid reusing context object ... fix https://github.com/chrislusf/seaweedfs/issues/1182	2020-02-25 21:50:12 -08:00
Chris Lu	6ea394b802	1.57	2020-02-22 21:34:49 -08:00
Chris Lu	621cdbdf58	filer: avoid possible timeouts for updates and deletions	2020-02-20 15:44:17 -08:00
Chris Lu	443a68484a	1.56	2020-02-17 12:54:45 -08:00
Chris Lu	9232d3ac68	ignore error when counting closed connections	2020-02-14 10:28:02 -08:00
Chris Lu	96c1ae8471	refactoring the close http response	2020-02-14 09:46:36 -08:00
Chris Lu	cf5064d702	properly close http response	2020-02-14 09:09:15 -08:00
Chris Lu	62c34454d8	1.55	2020-02-11 23:00:31 -08:00
Chris Lu	33b3bd467c	Revert "HEAD operation changes to fasthttp" ... This reverts commit 58f126fd27.	2020-02-10 13:43:53 -08:00
Chris Lu	58f126fd27	HEAD operation changes to fasthttp	2020-02-10 09:13:29 -08:00
Chris Lu	29945fad51	1.54	2020-02-09 18:07:17 -08:00
Chris Lu	beb0a1599e	1.53	2020-01-31 00:32:09 -08:00
Chris Lu	a80ecbfe84	s3: add s3 copy ... fix https://github.com/chrislusf/seaweedfs/issues/1190	2020-01-31 00:11:12 -08:00
Chris Lu	3b043ead49	Merge pull request #1189 from eryx67/compression-wav ... Support wav files compression	2020-01-29 22:37:35 -08:00
Chris Lu	abf90ad7b7	remove testing for viper	2020-01-29 21:38:53 -08:00
Chris Lu	20b1fb9ab7	fix test	2020-01-29 21:24:06 -08:00
eryx67	2a87087590	support wav files compression	2020-01-30 10:11:47 +05:00
Chris Lu	d335f04de6	support env variables to overwrite toml file	2020-01-29 09:09:55 -08:00
Chris Lu	27b94cb65b	fix wrong url ... fix https://github.com/chrislusf/seaweedfs/issues/1187	2020-01-28 00:49:47 -08:00
Chris Lu	234f69452b	1.52	2020-01-26 21:04:40 -08:00
Chris Lu	72a64a5cf8	use the same context object in order to retry	2020-01-26 14:42:11 -08:00
Chris Lu	107e8a56ea	retry context canceled request	2020-01-24 01:40:51 -08:00
Chris Lu	6a5c037099	fix http range requests	2020-01-22 22:59:12 -08:00
Chris Lu	bb1be61602	1.51	2020-01-20 20:30:03 -08:00
Chris Lu	39edcfde4c	filer: set inode ... avoid "getcwd: No such file or directory"	2020-01-19 12:06:19 -08:00
Chris Lu	30c7148020	1.50	2020-01-08 22:08:35 -08:00
Chris Lu	89e16bd2e8	skip error when draining reader ... fix https://github.com/chrislusf/seaweedfs/issues/1179	2020-01-08 18:07:07 -08:00
Chris Lu	66ab09aa4a	1.49	2020-01-08 09:51:35 -08:00
Chris Lu	aa1807e082	1.48	2020-01-01 09:27:02 -08:00
Chris Lu	c06f7eb48a	load volume info from .vif file, use superblock as a backup	2019-12-28 12:28:58 -08:00
Chris Lu	1fd8926ac7	ignore draining error	2019-12-25 21:39:33 -08:00
divinerapier	4cbb6fa199	feat: drains http body if buffer is too small ... Signed-off-by: divinerapier <poriter.coco@gmail.com>	2019-12-26 11:28:43 +08:00
divinerapier	84640d07b7	fix: handle errors for ReadUrl ... Signed-off-by: divinerapier <poriter.coco@gmail.com>	2019-12-26 10:07:07 +08:00
Chris Lu	573b74f24f	add SetDefault function for Configuration	2019-12-20 20:56:14 -08:00
Chris Lu	54369eccea	1.47	2019-12-17 15:11:01 -08:00
Chris Lu	6808bfec49	1.47	2019-12-17 15:08:45 -08:00
Chris Lu	392678f8f3	upload skipping mimetype if not needed	2019-12-15 20:57:08 -08:00
Chris Lu	a631b2a9e9	1.46	2019-12-14 11:28:48 -08:00
Chris Lu	10bd3c6b4b	refactoring	2019-12-08 19:44:16 -08:00
Chris Lu	f81d43442b	filer: speed up filer.meta.save by parallelizing	2019-12-06 22:20:59 -08:00
Chris Lu	0da7b894cc	pass backend config from master to volume servers	2019-11-29 01:05:09 -08:00
Chris Lu	a9dbd52e06	Merge pull request #1145 from stlpmo-jn/volume_graceful_stop ... let volume server graceful stop	2019-11-29 00:11:57 -08:00
Chris Lu	249b9d8d5d	1.45	2019-11-22 14:51:16 -08:00
stlpmo	753138a3f7	test passed	2019-11-17 11:40:36 +08:00
Chris Lu	de0c983808	refactoring scaffolding for master.toml	2019-11-14 22:20:38 -08:00
Lei Liu	46755ea1e1	fix master maintenance logic ... Signed-off-by: Lei Liu <lei01.liu@horizon.ai>	2019-11-12 14:46:10 +08:00
stlpmo	d07701fa75	delete unused function	2019-11-11 18:08:48 +08:00
stlpmo	62d393d6c9	ut pass	2019-11-11 10:52:21 +08:00
stlpmo	802a0eb3fe	move from cmd-line to scaffold	2019-11-11 09:15:17 +08:00
Chris Lu	50e885da45	1.44	2019-10-11 23:56:58 -07:00
Chris Lu	20d90dea5a	filer: avoid hard-coded upload timeout	2019-09-05 01:26:59 -07:00
Chris Lu	89a0cb2e11	1.43	2019-09-03 01:31:58 -07:00
Chris Lu	35d471a818	Revert "use std encoding lib"	2019-08-23 07:59:09 -07:00
divinerapier	e798e4e9c5	use std encoding lib ... Signed-off-by: divinerapier <poriter.coco@gmail.com>	2019-08-19 18:07:54 +08:00
Chris Lu	3f851feb59	1.42	2019-07-21 21:52:48 -07:00
Chris Lu	cf2804eebd	1.41	2019-07-07 20:40:20 -07:00
Chris Lu	23d36dfbaf	1.40 ... enable azure replication	2019-07-02 21:27:05 -07:00
Chris Lu	36022f9416	1.39	2019-06-30 00:50:17 -07:00
Chris Lu	fdff927b37	1.38	2019-06-24 00:36:46 -07:00
Chris Lu	95ad56d99d	1.37	2019-06-21 20:56:34 -07:00
Chris Lu	fe4eb6bffa	1.36	2019-06-21 13:11:40 -07:00
Chris Lu	be3f1f84a2	1.35	2019-06-10 21:34:06 -07:00
Chris Lu	ede876cfdb	periodic scripts exeuction from leader master	2019-06-05 01:30:24 -07:00
Chris Lu	7e80b2b882	fix multiple bugs	2019-06-03 02:26:31 -07:00
Chris Lu	f2c4c888f6	1.34	2019-05-17 22:53:30 -07:00
Chris Lu	cf58fc0e63	refactor: extract out the write throttler	2019-05-06 13:56:08 -07:00
Chris Lu	8c982db25d	1.33	2019-05-03 17:27:50 -07:00
Wine93	32f93fb09e	style: go fmt for all	2019-04-30 03:22:19 +00:00
Chris Lu	59d532a8c3	1.31	2019-04-21 14:09:50 -07:00
Chris Lu	e5506152c0	refactoring	2019-04-18 21:43:36 -07:00