387 Commits

Author	SHA1	Message	Date
Chris Lu	dc481c081c	remove unused function	2021-08-27 22:18:54 -07:00
Chris Lu	9242f3aaeb	fix build	2021-08-26 17:33:57 -07:00
Chris Lu	05a648bb96	refactor: separating out remote.proto	2021-08-26 15:18:34 -07:00
Chris Lu	9bcf94b2b1	ensure multi-threaded correctness	2021-08-25 17:28:50 -07:00
Chris Lu	40dc283b2d	fix locating data chunks	2021-08-15 23:07:58 -07:00
Chris Lu	72eb6d5b9d	ensure no writes to remote storage if content is not changed	2021-08-15 20:23:41 -07:00
Chris Lu	5d5a21ba2d	adjust log format	2021-08-15 19:46:45 -07:00
Chris Lu	49b5e47bd1	retry forever with filer.remote.sync, and some refactoring	2021-08-15 12:38:26 -07:00
Chris Lu	8f7d2d317f	readerAt need to use the right offset ... fix https://github.com/chrislusf/seaweedfs/issues/2259	2021-08-15 11:55:58 -07:00
Chris Lu	9462f5129a	shell: add "remote.meta.sync"	2021-08-15 01:53:46 -07:00
Chris Lu	0c66b173a4	fix	2021-08-13 11:31:43 -07:00
Chris Lu	e02a317d3d	adjust retry logic in case some data is partially written	2021-08-13 11:30:38 -07:00
Chris Lu	2d519c6cb6	adjust the retry logic	2021-08-13 11:13:30 -07:00
Chris Lu	b961fcd338	filer: stream read from volume server, reduce memory usage	2021-08-13 11:00:11 -07:00
Chris Lu	5a0f92423e	use grpc and jwt	2021-08-12 21:40:33 -07:00
Chris Lu	bfac55e6c0	avoid integer overflow ... fix https://github.com/chrislusf/seaweedfs/issues/2254	2021-08-11 22:22:49 -07:00
Chris Lu	9d85569c55	ensure using local quorum consistency	2021-08-10 05:10:57 -07:00
Chris Lu	713c035a6e	shell: remote.cache remote.uncache	2021-08-09 14:35:18 -07:00
Chris Lu	c5f38c365d	go fmt	2021-08-08 22:30:36 -07:00
Chris Lu	c0b12da4ef	shell: add filer.remote.unmount	2021-08-08 22:26:37 -07:00
Chris Lu	13e45e1605	filer.remote.sync can work now	2021-08-08 01:21:42 -07:00
Chris Lu	46b0cb8c86	Merge branch 'master' into add_remote_storage	2021-08-07 15:41:27 -07:00
Chris Lu	de730b079d	ChunkStreamReader implenents io.ReaderAt	2021-08-07 15:41:07 -07:00
Chris Lu	59732a0529	refactoring	2021-08-07 15:35:27 -07:00
Chris Lu	ecb234f75a	refactor	2021-08-07 14:46:23 -07:00
Chris Lu	679f800caa	Merge branch 'master' into add_remote_storage	2021-08-06 20:27:12 -07:00
byunghwa.yun	f3dc909b21	Change default permissions	2021-08-06 12:24:35 +09:00
Chris Lu	6b743dbbf9	refactor client subscribe metadata	2021-08-04 16:25:46 -07:00
Chris Lu	f6a9ad8001	fix tests	2021-08-04 00:31:06 -07:00
Chris Lu	81b255df8b	Merge branch 'master' into add_remote_storage	2021-08-01 15:44:21 -07:00
Chris Lu	89933c46d2	s3: skip hidden directories in /buckets folder	2021-08-01 12:28:08 -07:00
Chris Lu	767edd3c08	rename	2021-07-31 23:52:09 -07:00
Chris Lu	9df7d16791	read <- remote_storage	2021-07-31 22:39:38 -07:00
Chris Lu	899963ac20	remote storage location changed to struct	2021-07-29 02:08:55 -07:00
Chris Lu	c090d6bb25	add ReadRemote(), add read remote setup when filer starts	2021-07-28 22:43:12 -07:00
Chris Lu	1752eeb538	remote.mount saves the mapping	2021-07-27 03:26:35 -07:00
Chris Lu	4b94b03d90	directory to remote storage mapping	2021-07-27 01:16:28 -07:00
Chris Lu	99b599aa8a	remote.mount	2021-07-26 22:53:44 -07:00
byunghwa.yun	2b28a818f1	Fix mysql sql for batch delete	2021-07-25 11:06:14 +09:00
Chris Lu	bdb632fa62	Merge pull request #2211 from qieqieplus/filer-bloom-filter ... add bloom filter for filer leveldb/rocksdb	2021-07-22 23:35:54 -07:00
qieqieplus	2b46df13f8	add bloom filter for filer leveldb/rocksdb	2021-07-23 14:05:59 -06:00
Chris Lu	182288f860	filer: fix mysql, postgres batch delete error	2021-07-22 08:23:20 -07:00
Chris Lu	7359193e97	go fmt	2021-07-21 14:38:12 -07:00
Chris Lu	70effac0d3	configure and store remote configurations	2021-07-21 02:24:34 -07:00
Chris Lu	a125c8fbe2	Merge pull request #2204 from nivekuil/tokenpolicy ... cassandra: use LocalQuorum for all queries	2021-07-20 15:37:31 -07:00
nivekuil	b9a67d46c5	cassandra: use LocalQuorum for all queries ... This changes this filer store from eventual to strong consistency at the cost of read performance.	2021-07-20 14:47:39 -07:00
Chris Lu	54c8bc8673	Merge branch 'remote_overlay'	2021-07-19 23:18:35 -07:00
Chris Lu	7ab389e7ec	optimization: improve random range query for large files	2021-07-19 23:07:22 -07:00
Chris Lu	93c37cfded	Update filechunk_manifest.go	2021-07-19 18:41:41 -07:00
Chris Lu	450222dd64	add remote to filer.Entry and filer_pb entry, add RemoteConf	2021-07-19 02:47:27 -07:00