4017 Commits

Author	SHA1	Message	Date
Chris Lu	8cfd487608	2.62	2021-08-08 23:33:12 -07:00
Chris Lu	734c980040	volume: support concurrent download data size limit	2021-08-08 23:25:16 -07:00
Chris Lu	96ce85f5ae	rename	2021-08-08 22:33:31 -07:00
Chris Lu	c5f38c365d	go fmt	2021-08-08 22:30:36 -07:00
Chris Lu	df85f7a1eb	adjust help message	2021-08-08 22:30:12 -07:00
Chris Lu	c0b12da4ef	shell: add filer.remote.unmount	2021-08-08 22:26:37 -07:00
Chris Lu	882a93dacd	fix tests	2021-08-08 17:56:26 -07:00
Chris Lu	7412ccdf88	write back remote entry to local entry after uploading to remote	2021-08-08 17:55:03 -07:00
Chris Lu	dcf614a8c3	skip if the remote entry update is because of internal managerial operations	2021-08-08 15:58:10 -07:00
Chris Lu	eed26af266	Merge branch 'master' into add_remote_storage	2021-08-08 15:48:04 -07:00
Chris Lu	4370a4db63	use int64 for volume count in case of negative overflow	2021-08-08 15:19:39 -07:00
Chris Lu	26c222f596	shell: volume.tier.move avoid moving all volumes to one destination	2021-08-08 15:12:39 -07:00
Chris Lu	13e45e1605	filer.remote.sync can work now	2021-08-08 01:21:42 -07:00
Chris Lu	8f5170c138	remove imports	2021-08-07 16:20:17 -07:00
Chris Lu	46b0cb8c86	Merge branch 'master' into add_remote_storage	2021-08-07 15:41:27 -07:00
Chris Lu	de730b079d	ChunkStreamReader implenents io.ReaderAt	2021-08-07 15:41:07 -07:00
Chris Lu	59732a0529	refactoring	2021-08-07 15:35:27 -07:00
Chris Lu	ecb234f75a	refactor	2021-08-07 14:46:23 -07:00
Chris Lu	270770d7d7	refactor	2021-08-07 14:18:53 -07:00
Chris Lu	679f800caa	Merge branch 'master' into add_remote_storage	2021-08-06 20:27:12 -07:00
Chris Lu	0c0f77e2ae	skip not found error on deletion	2021-08-06 19:35:47 -07:00
Chris Lu	8828f485c0	print volume deletion error	2021-08-06 19:30:22 -07:00
Chris Lu	d2b23f3d38	Merge pull request #2238 from combineads/change_default_mode ... Change default permissions	2021-08-05 21:28:06 -07:00
Chris Lu	e23ca3845b	Merge pull request #2237 from combineads/add_mtime ... Add crtime and mtime	2021-08-05 21:27:13 -07:00
Chris Lu	286e5dd375	Merge branch 'master' into add_remote_storage	2021-08-05 21:07:04 -07:00
Chris Lu	1e22166939	adjust error message	2021-08-05 21:06:55 -07:00
byunghwa.yun	f3dc909b21	Change default permissions	2021-08-06 12:24:35 +09:00
byunghwa.yun	e84fad9acf	Add crtime and mtime	2021-08-06 12:13:37 +09:00
Chris Lu	1a5d29520c	add tests	2021-08-05 14:49:24 -07:00
Chris Lu	6b743dbbf9	refactor client subscribe metadata	2021-08-04 16:25:46 -07:00
Chris Lu	b9ecf1e3a8	refacotring	2021-08-04 14:56:13 -07:00
Chris Lu	d84c311699	refactoring	2021-08-04 12:30:18 -07:00
Chris Lu	f6a9ad8001	fix tests	2021-08-04 00:31:06 -07:00
Chris Lu	42969c9c62	Merge branch 'master' into add_remote_storage	2021-08-03 21:20:05 -07:00
Chris Lu	8cf0c515bf	shell: volume.fix.replication retries even when there is no error #2235 ... fix https://github.com/chrislusf/seaweedfs/issues/2235	2021-08-03 09:32:55 -07:00
Chris Lu	d2ddf1dbdb	typo	2021-08-03 09:32:31 -07:00
Chris Lu	3afbf04007	add TLS grpc support for filer meta clients	2021-08-03 01:30:35 -07:00
Chris Lu	fe60c6ef9a	minor changes	2021-08-02 11:49:40 -07:00
byunghwa.yun	bdc7730fdb	Add autocomplete	2021-08-03 02:56:52 +09:00
Chris Lu	2ae9705442	adjust text	2021-08-01 22:55:19 -07:00
Chris Lu	3739717092	Revert "adds a test" ... This reverts commit f690643b47.	2021-08-01 22:54:45 -07:00
Chris Lu	f690643b47	adds a test	2021-08-01 22:53:50 -07:00
Chris Lu	6de786185d	volume.balance: balance read only volumes first	2021-08-01 22:32:50 -07:00
Chris Lu	3bb640b786	add -force option following #2228	2021-08-01 20:03:05 -07:00
byunghwa.yun	9e839cb5cc	Add force option in volume.deleteEmpty command	2021-08-02 11:37:20 +09:00
Chris Lu	56ee1d5ef1	2.61	2021-08-01 15:50:19 -07:00
Chris Lu	81b255df8b	Merge branch 'master' into add_remote_storage	2021-08-01 15:44:21 -07:00
Chris Lu	58bc3ecf47	add default quietFor value	2021-08-01 15:36:06 -07:00
Chris Lu	2ca1839d77	shell: add volume.deleteEmpty command	2021-08-01 15:33:45 -07:00
Chris Lu	89933c46d2	s3: skip hidden directories in /buckets folder	2021-08-01 12:28:08 -07:00
Chris Lu	1c7e404abe	remove buckets folder option ... the related code still works for old deployments	2021-08-01 12:23:16 -07:00
Chris Lu	cb1dbd3135	refactor	2021-08-01 11:53:46 -07:00
Chris Lu	1161c70dab	Merge branch 'master' into add_remote_storage	2021-08-01 01:10:13 -07:00
Chris Lu	9cc84a910f	volume: deletion can skip volume size checking ... fix https://github.com/chrislusf/seaweedfs/issues/2225	2021-08-01 00:32:51 -07:00
Chris Lu	767edd3c08	rename	2021-07-31 23:52:09 -07:00
Chris Lu	9df7d16791	read <- remote_storage	2021-07-31 22:39:38 -07:00
Chris Lu	f5a69a0e44	Merge branch 'master' into add_remote_storage	2021-07-31 09:39:48 -07:00
Chris Lu	6ba65c3382	customizable debug port	2021-07-31 09:18:41 -07:00
byunghwa.yun	5d4438a72f	Fix typo	2021-07-31 22:23:52 +09:00
Chris Lu	1ff8285d82	debug from any server	2021-07-31 02:13:21 -07:00
Chris Lu	0d6f45cb46	add debug option to generate full goroutine stack dump	2021-07-31 02:00:01 -07:00
Chris Lu	a3290faf17	shell command to calculate size / number of volumes in a collection ... fix https://github.com/chrislusf/seaweedfs/issues/2224	2021-07-30 15:49:46 -07:00
Chris Lu	c6f992b2a3	remove dead code	2021-07-30 15:18:01 -07:00
Chris Lu	899963ac20	remote storage location changed to struct	2021-07-29 02:08:55 -07:00
Chris Lu	c090d6bb25	add ReadRemote(), add read remote setup when filer starts	2021-07-28 22:43:12 -07:00
Chris Lu	4deac06da0	log fix	2021-07-28 22:25:04 -07:00
divanikus	5c6270a93a	shell: ability to use wildcards for collections, all collections if ommited	2021-07-27 23:53:01 +03:00
Chris Lu	035b0bae29	refactor	2021-07-27 03:32:24 -07:00
Chris Lu	1752eeb538	remote.mount saves the mapping	2021-07-27 03:26:35 -07:00
Chris Lu	4b94b03d90	directory to remote storage mapping	2021-07-27 01:16:28 -07:00
Chris Lu	99b599aa8a	remote.mount	2021-07-26 22:53:44 -07:00
Chris Lu	5dede5d38d	2.60	2021-07-25 22:09:09 -07:00
Chris Lu	35f70c51b0	refactor	2021-07-25 03:37:37 -07:00
Chris Lu	72eec84167	shell: add fs.mkdir	2021-07-25 02:37:39 -07:00
byunghwa.yun	2b28a818f1	Fix mysql sql for batch delete	2021-07-25 11:06:14 +09:00
Chris Lu	10464f47d0	Merge pull request #2217 from combineads/add_webp_format	2021-07-24 09:42:41 -07:00
byunghwa.yun	7374b5b473	Add webp extension for resizing	2021-07-24 14:32:31 +09:00
byunghwa.yun	2595f269d1	Add the webp image type	2021-07-24 14:26:40 +09:00
Chris Lu	ac28611817	snowflake sequencer need an unique id ... fix https://github.com/chrislusf/seaweedfs/issues/2213	2021-07-23 20:54:03 -07:00
Chris Lu	5c14da0f1e	filer.copy: fail early if assign request failed ... fix https://github.com/chrislusf/seaweedfs/issues/2216	2021-07-23 20:01:43 -07:00
Chris Lu	4cc2165061	weed -h subcommand list is not sorted, makes discovery difficult ... fix https://github.com/chrislusf/seaweedfs/issues/2214	2021-07-23 18:44:53 -07:00
Chris Lu	10fc478557	scaffold -config= should output to stdout ... fix https://github.com/chrislusf/seaweedfs/issues/2212	2021-07-23 18:41:25 -07:00
Chris Lu	092932af8f	shell: faster bucket delete ... avoid waiting for large object metadata	2021-07-23 11:25:46 -07:00
Chris Lu	bdb632fa62	Merge pull request #2211 from qieqieplus/filer-bloom-filter ... add bloom filter for filer leveldb/rocksdb	2021-07-22 23:35:54 -07:00
qieqieplus	2b46df13f8	add bloom filter for filer leveldb/rocksdb	2021-07-23 14:05:59 -06:00
Chris Lu	60f5c0a2f5	fix security alert on github.com/dgrijalva/jwt-go ... resolve https://github.com/chrislusf/seaweedfs/security/dependabot/go.mod/github.com%2Fdgrijalva%2Fjwt-go/open	2021-07-22 10:22:09 -07:00
Chris Lu	182288f860	filer: fix mysql, postgres batch delete error	2021-07-22 08:23:20 -07:00
Chris Lu	84d91f143f	mount: hide /etc folder also from mount	2021-07-22 00:40:16 -07:00
Chris Lu	7359193e97	go fmt	2021-07-21 14:38:12 -07:00
Chris Lu	cc32436d63	Merge pull request #2209 from combineads/fuse_option ... Fix error handling	2021-07-21 12:37:20 -07:00
Chris Lu	67537ff05b	Merge pull request #2208 from combineads/add_rm_options	2021-07-21 11:08:43 -07:00
byunghwa.yun	e0c7708b03	Fix error handling	2021-07-21 22:48:04 +09:00
byunghwa.yun	41d559f54e	Add fs rm options	2021-07-21 21:10:36 +09:00
Chris Lu	70effac0d3	configure and store remote configurations	2021-07-21 02:24:34 -07:00
Chris Lu	3138805b33	Merge pull request #2200 from combineads/add_fs_rm ... weed shell: fs.rm remove file or directory	2021-07-21 00:35:54 -07:00
Chris Lu	a125c8fbe2	Merge pull request #2204 from nivekuil/tokenpolicy ... cassandra: use LocalQuorum for all queries	2021-07-20 15:37:31 -07:00
nivekuil	b9a67d46c5	cassandra: use LocalQuorum for all queries ... This changes this filer store from eventual to strong consistency at the cost of read performance.	2021-07-20 14:47:39 -07:00
Chris Lu	54c8bc8673	Merge branch 'remote_overlay'	2021-07-19 23:18:35 -07:00
Chris Lu	7ab389e7ec	optimization: improve random range query for large files	2021-07-19 23:07:22 -07:00
Chris Lu	93c37cfded	Update filechunk_manifest.go	2021-07-19 18:41:41 -07:00
byunghwa.yun	b23b307e08	Add fs rm	2021-07-20 08:49:28 +09:00
Chris Lu	b938df97a2	remove unused parameter	2021-07-19 02:59:12 -07:00
Chris Lu	450222dd64	add remote to filer.Entry and filer_pb entry, add RemoteConf	2021-07-19 02:47:27 -07:00
Chris Lu	8dc5def435	proto: add remote	2021-07-19 01:12:31 -07:00
Chris Lu	99155e4880	add -retry option for volume.fix.replication ... fix https://github.com/chrislusf/seaweedfs/issues/2191	2021-07-16 12:13:46 -07:00
Chris Lu	2f209675ab	Added -retry option for volumeServer.evacuate ... related to https://github.com/chrislusf/seaweedfs/issues/2191	2021-07-16 12:08:21 -07:00
Chris Lu	fb7a1be1c4	refactor	2021-07-16 11:47:37 -07:00
nivekuil	2faf96f002	cassandra: Use LocalOne instead of One consistency	2021-07-16 04:29:46 -07:00
Chris Lu	4be5b4ff5c	[volume.check.disk] skip errors and check next volumes ... fix https://github.com/chrislusf/seaweedfs/issues/2129	2021-07-16 02:03:32 -07:00
Chris Lu	e95166d739	S	2021-07-15 17:43:38 -07:00
Chris Lu	a45bbc0b75	2.59	2021-07-15 15:52:22 -07:00
Chris Lu	f0042f62dd	readable logs	2021-07-15 12:17:48 -07:00
Chris Lu	18c40686d9	s3: multipart upload miss data if file is chunked in 4MB ... fix https://github.com/chrislusf/seaweedfs/issues/2195	2021-07-15 11:56:28 -07:00
Chris Lu	5a838dbe53	fix compilation	2021-07-13 11:40:21 -07:00
Chris Lu	6103649ffb	shell: volume.check.disk adds retries in case the volumes are just moved ... related to https://github.com/chrislusf/seaweedfs/issues/2194	2021-07-13 11:19:56 -07:00
Chris Lu	49c66e88a0	volume: change all writes to fsync during graceful stopping ... fix https://github.com/chrislusf/seaweedfs/issues/2193	2021-07-13 01:29:57 -07:00
Chris Lu	01adc567aa	shell: volume.fsck deletes a volume is the volume has only orphaned data. ... fix https://github.com/chrislusf/seaweedfs/issues/2190	2021-07-12 11:22:00 -07:00
Chris Lu	297b41266b	2.58	2021-07-12 01:33:47 -07:00
Chris Lu	b194f91f47	add version to filer configuration response	2021-07-12 01:23:20 -07:00
bingoohuang	9e48bff8ee	fix typo in fs.configure help message text	2021-07-12 13:51:21 +08:00
Chris Lu	d013d6d968	shell: volume.fsck "reallyDeleteFromVolume" should send padded file ids to delete ... fix https://github.com/chrislusf/seaweedfs/issues/2188	2021-07-10 23:16:06 -07:00
Chris Lu	3d624d1e16	rename	2021-07-09 03:19:21 -07:00
Chris Lu	ecce300964	s3 config read via grpc	2021-07-09 02:48:03 -07:00
Chris Lu	da7bd62822	/etc files are stored inside metadata store	2021-07-09 02:33:14 -07:00
Chris Lu	a6d73e0a66	Merge pull request #2185 from bingoohuang/master ... show RemoteVolumes/EcVolumes only if it is not empty	2021-07-06 00:28:07 -07:00
bingoohuang	ed57a55eae	show RemoteVolumes/EcVolumes only if it is not empty	2021-07-06 15:20:18 +08:00
Chris Lu	44b50b2fdf	another fix related to 64 bit alignment ... fix https://github.com/chrislusf/seaweedfs/issues/2177	2021-07-05 16:30:43 -07:00
Chris Lu	141388367e	Merge pull request #2183 from bingoohuang/master ... extract embed html of master/volume/filer ui to separate files	2021-07-05 03:22:26 -07:00
bingoohuang	44a2538f67	extract embed html of master/volume/filer ui to separate files	2021-07-05 18:09:44 +08:00
Chris Lu	e2da647fa3	Merge pull request #2182 from qieqieplus/fix-event-notification ... Subscribe meta data: sync empty notification with timestamp	2021-07-05 02:00:53 -07:00
qieqieplus	233103f6b2	sync empty notification with timestamp	2021-07-05 16:01:16 +08:00
bingoohuang	5dbbe19c8b	extract embed toml example to separate files	2021-07-05 11:16:49 +08:00
Chris Lu	5bcc77b46c	volume: default readMode to proxy	2021-07-03 15:55:56 -07:00
Chris Lu	f5fa0b08fd	2.57	2021-07-03 15:10:57 -07:00
Chris Lu	a024254ad7	logging	2021-07-03 14:51:01 -07:00
Chris Lu	d39b2689a5	S3 authorization: StreamingSigned enforces access control ... fix https://github.com/chrislusf/seaweedfs/issues/2180	2021-07-03 14:50:53 -07:00
Chris Lu	fa0dab6029	mount: rename also recursively move file handles ... related to https://github.com/chrislusf/seaweedfs/issues/2169	2021-07-03 02:59:35 -07:00
Chris Lu	8fe75692ee	volume: address "unaligned 64-bit atomic operation" ... fix https://github.com/chrislusf/seaweedfs/issues/2177	2021-07-02 13:57:43 -07:00
Chris Lu	2cb8b31ea7	Merge pull request #2179 from nivekuil/tokenpolicy ... Cassandra: Use TokenAwareHostPolicy with fallback to localDC by default	2021-07-02 13:51:53 -07:00
nivekuil	8425705643	Cassandra: Use TokenAwareHostPolicy by default with fallback ... See https://pkg.go.dev/github.com/gocql/gocql#hdr-Data_center_awareness_and_query_routing	2021-07-02 13:50:01 -07:00
Chris Lu	d8bda0b229	locate the weed binary	2021-07-02 13:33:17 -07:00
danielflira	586e066897	fix weed fuse parameters parsing	2021-07-02 16:52:52 -03:00
lyg	1a4db87e19	fix s3 metadata error with multipart upload	2021-07-02 11:00:42 +08:00
Chris Lu	c6d4c16079	S3: add metadata with multipart upload ... fix https://github.com/chrislusf/seaweedfs/issues/2173	2021-07-01 19:12:11 -07:00
Chris Lu	2420c60fc4	log reading adds delay between retries	2021-07-01 14:01:25 -07:00
Chris Lu	067eb15e70	remove debug messages	2021-07-01 01:24:07 -07:00
Chris Lu	b624090398	go fmt	2021-07-01 01:21:14 -07:00
Chris Lu	215b169562	mount: recursively rename locally	2021-07-01 01:19:31 -07:00
Chris Lu	c795183887	return node itself as directory handler	2021-07-01 01:19:31 -07:00
Chris Lu	30dbe98dff	Update Makefile	2021-07-01 01:19:31 -07:00