849 Commits

Author	SHA1	Message	Date
石昌林	3dd60529c5	some code optimizations	2022-06-17 19:07:39 +08:00
石昌林	37df209195	add some unit tests and some code optimizes	2022-06-17 17:11:18 +08:00
石昌林	78b3728169	add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations ... configure s3 circuit breaker by 'command_s3_circuitbreaker.go': usage eg: # Configure the number of simultaneous global (current s3api node) requests s3.circuit.breaker -global -type count -actions Write -values 1000 -apply # Configure the number of simultaneous requests for bucket x read and write s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply # Configure the total bytes of simultaneous requests for bucket write s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply # Disable circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -enable false -apply # Delete circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -delete -apply	2022-06-15 21:07:55 +08:00
chrislu	d12f431d98	collect volume server status	2022-06-12 11:56:23 -07:00
shichanglin5	a63559ee6c	fix: fix command prompt information	2022-06-10 12:46:10 +08:00
chrislu	4fd5f96598	filer: remove replication, collection, disk_type info from entry metadata ... these metadata can change and are not used	2022-06-06 00:39:35 -07:00
chrislu	f97acdd489	volume.fix.replication fix retry logic ... fix https://github.com/chrislusf/seaweedfs/issues/3136	2022-06-03 08:45:29 -07:00
chrislu	6793bc853c	help message when in simulation mode	2022-05-31 14:48:46 -07:00
chrislu	89948a373b	fix error reporting for "Need to a hdd disk type" ... related to https://github.com/chrislusf/seaweedfs/issues/3128	2022-05-31 12:43:55 -07:00
chrislu	5b9347c938	typo	2022-05-30 12:37:41 -07:00
chrislu	ca16fbf0ef	shell: cluster.ps display filer group	2022-05-29 16:37:14 -07:00
Konstantin Lebedev	44f53ceda6	fix collectionIsMismatch charset	2022-05-16 13:23:23 +05:00
Konstantin Lebedev	10d435f2c2	fix skip loop	2022-05-16 13:16:27 +05:00
Konstantin Lebedev	279053572c	avoid delete volume replica if collection mismatch	2022-05-16 13:07:05 +05:00
chrislu	271b5aed96	shell: volume.fsck add a note for -reallyDeleteFromVolume option	2022-05-15 11:07:04 -07:00
chrislu	94635e9b5c	filer: add filer group	2022-05-01 21:59:16 -07:00
guosj	1a5de36257	fix return value	2022-04-29 11:19:36 +08:00
Chris Lu	b557faf175	Merge pull request #2967 from kmlebedev/vl_chk_disk_vlid ... volume.check.disk add param volumeId	2022-04-26 22:09:06 -07:00
Konstantin Lebedev	d4343ab7da	forcePurging desc	2022-04-25 23:11:56 +05:00
Konstantin Lebedev	ae56b2c00f	change forcePurging to a pointer	2022-04-25 23:10:01 +05:00
Konstantin Lebedev	6d2fda27d2	delete missing data from volumes in one replica	2022-04-25 22:59:46 +05:00
Konstantin Lebedev	67476d830a	volume.check.disk add param volumeId	2022-04-25 18:27:14 +05:00
Chris Lu	88a669dd19	Merge pull request #2858 from kmlebedev/fsck_replicas ... fsck replicas	2022-04-19 00:17:43 -07:00
Konstantin Lebedev	2364fab927	volume.list show only readonly, collectionPattern and volumeId	2022-04-19 11:44:41 +05:00
Konstantin Lebedev	36c5a59ed8	add help	2022-04-18 19:36:14 +05:00
Konstantin Lebedev	1e35b4929f	shell vacuum volume by collection and volume id	2022-04-18 18:40:58 +05:00
leyou240	89eb87c1d1	Merge branch 'master' into slices.SortFunc	2022-04-18 10:39:29 +08:00
justin	3551ca2fcf	enhancement: replace sort.Slice with slices.SortFunc to reduce reflection	2022-04-18 10:35:43 +08:00
chrislu	460d56d283	shell: cluster.check prints out clock delta and network latency	2022-04-16 13:24:17 -07:00
chrislu	1f03fcccb1	fix nil in cluster_check shell command ... fix https://github.com/chrislusf/seaweedfs/issues/2905	2022-04-12 08:47:27 -07:00
Konstantin Lebedev	76b1c5ce67	cleanupUploads use jwt token	2022-04-12 13:46:11 +05:00
Konstantin Lebedev	2dfbe210e0	avoid breaking loop in cleanupUploads if error is empty	2022-04-11 11:16:20 +05:00
Konstantin Lebedev	931cb9e581	use "cluster.raft.{ps,add,remove}"	2022-04-11 10:50:01 +05:00
Konstantin Lebedev	f5246b748d	Merge branch 'new_master' into hashicorp_raft ... # Conflicts: # weed/pb/master_pb/master.pb.go	2022-04-07 18:50:27 +05:00
Konstantin Lebedev	85d80fd36d	fix removing old raft server	2022-04-07 15:31:37 +05:00
Konstantin Lebedev	357aa818fe	add raft shell cmds	2022-04-06 15:23:53 +05:00
chrislu	bc888226fc	erasure coding: tracking encoded/decoded volumes ... If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.	2022-04-05 19:03:02 -07:00
chrislu	4aae87f405	check missing hdd disk type	2022-04-04 14:48:00 -07:00
chrislu	6a2bcd03aa	configure mount quota	2022-04-02 21:34:26 -07:00
chrislu	105578a2f2	skip pinging self for master and volume server	2022-04-01 20:25:35 -07:00
chrislu	4ecba915f3	add check between peers	2022-04-01 17:40:25 -07:00
chrislu	4b5c0e3fa9	check cluster connectivities	2022-04-01 17:27:49 -07:00
Konstantin Lebedev	7f1383a41e	findExtraChunksInVolumeServers in consideration of replication	2022-04-01 14:45:41 +05:00
Konstantin Lebedev	3817e05dd0	fix collect filer files	2022-04-01 10:17:09 +05:00
Konstantin Lebedev	3cedb21bb7	skip new entities	2022-03-31 21:36:10 +05:00
Konstantin Lebedev	1d9e30d8c0	fsck replicas	2022-03-31 19:10:06 +05:00
Konstantin Lebedev	4f5b018a02	collecting only bucket volumes	2022-03-31 13:35:58 +05:00
chrislu	21e0898631	refactor: change masters from a slice to a map	2022-03-26 13:33:17 -07:00
banjiaojuhao	f7f2a597dd	minor	2022-03-08 16:22:55 +08:00
chrislu	ede6ce44c6	fix test	2022-03-06 17:09:55 -08:00
chrislu	6e49e75a5b	use logical number of files and sizes for statistics and quota	2022-03-04 18:47:44 -08:00
chrislu	95717d1006	simpler output	2022-02-24 13:50:08 -08:00
Chris Lu	6834df77a0	Merge pull request #2701 from guo-sj/fix_bugs_in_return_value	2022-02-23 19:23:37 -08:00
guosj	26f3ab8d4b	fix a return bug in func (c *commandVacuum) Do	2022-02-24 09:04:38 +08:00
Chris Lu	38dcaaa76e	Merge pull request #2683 from guo-sj/fix_fsconfigure_bug ... fix minor bug in commandFsConfigure.Do	2022-02-16 18:16:47 -08:00
guosj	ca121ecd66	fix minor bug in commandFsConfigure.Do	2022-02-17 09:57:52 +08:00
Chris Lu	64afbc5235	Merge pull request #2640 from Radtoo/fsck_with_delete ... Added basic deletion capabilities to fsck.	2022-02-14 23:00:01 -08:00
Radtoo	724ce04b1c	We picked the second flag's name.	2022-02-13 00:53:35 +01:00
Radtoo	fbe7ed7927	Revert the the flag to "reallyDeleteFromVolume".	2022-02-10 20:01:48 +01:00
chrislu	21aaa4c1f1	ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount) ... fix https://github.com/chrislusf/seaweedfs/issues/2642	2022-02-08 01:51:13 -08:00
chrislu	13f6ec1c4e	test checking ec distribution	2022-02-08 01:50:05 -08:00
chrislu	f34c2ff7c5	use embed txt	2022-02-08 01:14:27 -08:00
chrislu	c8c7c10c3f	volume.tier.move: avoid double counting ... related to https://github.com/chrislusf/seaweedfs/issues/2637	2022-02-08 00:57:35 -08:00
chrislu	f18803424a	volume.balance: add delay during tight loop ... fix https://github.com/chrislusf/seaweedfs/issues/2637	2022-02-08 00:53:55 -08:00
chrislu	85c1615b43	filer read empty file may cause OOM in some cases ... fix https://github.com/chrislusf/seaweedfs/issues/2641	2022-02-07 23:08:54 -08:00
Radtoo	fbb14e0ea8	Adding separate toggle to purge absent vols ... While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected. Also renamed forcePurge to just purge.	2022-02-06 23:46:52 +01:00
Radtoo	6b17f45da2	Also delete paths for which a volume is entirely absent.	2022-02-06 23:22:04 +01:00
Radtoo	fa0cfdfb7f	Added basic deletion capabilities to fsck.	2022-02-06 17:05:58 +01:00
chrislu	6bee1e9714	[volume.check.disk] fix wrong logic to compare ... fix https://github.com/chrislusf/seaweedfs/issues/2626 minuend - subtrahend	2022-02-01 07:48:28 -08:00
chrislu	b8490fe427	adjust volume count even when not applying the changes	2022-01-28 19:11:46 -08:00
divanikus	67e3fe996a	async volumeTierMove	2022-01-26 18:22:31 +03:00
chrislu	9d0f58c329	skip printing fs configuration	2022-01-21 13:29:47 -08:00
chrislu	ce2049cdb6	refactoring, move genFn before saveFn	2022-01-21 12:08:58 -08:00
chrislu	e47f63d159	enforce bucket quota	2022-01-21 02:34:42 -08:00
chrislu	6e57d8d0de	s3: check bucket usage and adjust read only according to quota	2022-01-21 02:15:27 -08:00
chrislu	606667f205	able to configure the quota for a bucket	2022-01-21 01:42:20 -08:00
chrislu	b1063162b6	display bucket quota	2022-01-21 00:55:04 -08:00
chrislu	f103491912	s3: list bucket size from weed shell	2022-01-21 00:26:49 -08:00
chrislu	ea57654e34	refactoring	2022-01-21 00:16:50 -08:00
chrislu	15c01d8b7f	add some notes	2022-01-12 15:04:48 -08:00
chrislu	107a4884a8	shell: tighter memory allocation	2022-01-12 14:59:29 -08:00
chrislu	adfd54e7c4	fix compilation	2022-01-12 01:24:24 -08:00
chrislu	826a7b307e	master: remove hard coded filer settings in master.toml ... fix https://github.com/chrislusf/seaweedfs/issues/2529	2022-01-12 01:11:25 -08:00
chrislu	60dc450091	skip fixing read only volumes ... fix https://github.com/chrislusf/seaweedfs/issues/2562	2022-01-06 09:52:28 -08:00
chrislu	9f9ef1340c	use streaming mode for long poll grpc calls ... streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed.	2021-12-26 00:15:03 -08:00
banjiaojuhao	083bf3a137	filer server: add "datacenter, rack and datanode" for path specific configuration	2021-12-23 23:25:05 +08:00
chrislu	94e5c0a454	skip purging from ec volumes	2021-12-12 22:55:27 -08:00
chrislu	a2d3f89c7b	add lock messages	2021-12-10 13:24:38 -08:00
chrislu	991a3dca0d	rename file	2021-12-10 13:11:00 -08:00
chrislu	7f0a97c7b6	shell: volume.list supports different verbosity level	2021-12-05 21:54:40 -08:00
chrislu	e6c026db65	volume.fix.replication: fix misplaced volumes ... fix https://github.com/chrislusf/seaweedfs/issues/2416	2021-12-05 16:56:25 -08:00
chrislu	53e2dee177	[volume.check.disk] was using the wrong source and target locations ... fix https://github.com/chrislusf/seaweedfs/issues/2268	2021-12-05 14:32:04 -08:00
Chris Lu	f3a334965d	Update command_remote_uncache.go ... minor	2021-11-28 23:10:32 -08:00
Chris Lu	9ccfc1cfee	Update command_volume_fsck.go ... minor	2021-11-28 22:39:24 -08:00
Chris Lu	71a94267c9	Update command_volume_fsck.go ... remove unused code	2021-11-28 22:37:11 -08:00
user	9668b15f38	Filtering by volume id is prioritized.	2021-11-11 18:18:56 +09:00
user	563a74a9eb	Volume filter by collection pattern added.	2021-11-11 18:01:47 +09:00
user	dbb8003ce3	Volume filter function added.	2021-11-11 17:45:17 +09:00
user	c387fe957b	Unused parameter removed.	2021-11-11 16:08:41 +09:00
Chris Lu	4729a57cc0	use constants	2021-11-08 17:47:56 -08:00
Chris Lu	a8b0f8864d	fix help message	2021-11-07 12:38:35 -08:00
Chris Lu	5ea86ef1da	Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()" ... This reverts commit af71ae11aa.	2021-11-05 17:52:15 -07:00
Chris Lu	314c32514b	add command to list filers	2021-11-05 17:39:41 -07:00
Chris Lu	00ae965d8d	randomize a bit for ec shards distribution	2021-11-04 09:23:40 -07:00
Chris Lu	af71ae11aa	master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()	2021-11-03 01:09:48 -07:00
Chris Lu	ab97b17e62	better printout	2021-11-02 23:45:47 -07:00
Chris Lu	5160eb08f7	shell: optionally read filer address from master	2021-11-02 23:38:45 -07:00
Chris Lu	18bfbf62fc	add one unit test	2021-11-02 23:36:43 -07:00
Chris Lu	794375ca0a	adjust help message since both fullPercent and quietFor are needed.	2021-11-01 17:22:47 -07:00
Chris Lu	24858507cc	rename API to avoid confusion	2021-10-30 19:27:25 -07:00
Chris Lu	d774fa6c9a	rename variable	2021-10-25 14:39:20 -07:00
Chris Lu	2539ba0b62	fix compilation	2021-10-25 14:38:48 -07:00
Chris Lu	5f2d7c1589	erasure coding: skip erasure coding if less than recommended 4 nodes	2021-10-25 14:38:11 -07:00
Chris Lu	5435027ff0	volume copy: stream out copying progress and avoid grpc request timeout ... fix https://github.com/chrislusf/seaweedfs/issues/2386	2021-10-24 02:52:56 -07:00
Eng Zer Jun	a23bcbb7ec	refactor: move from io/ioutil to io and os package ... The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2021-10-14 12:27:58 +08:00
Konstantin Lebedev	be4b3ed509	AclHandlers	2021-10-11 15:03:56 +05:00
Chris Lu	e862b2529a	refactor	2021-10-01 12:10:11 -07:00
Konstantin Lebedev	5e64b22b45	check that the topology has been updated	2021-10-01 18:51:22 +05:00
Konstantin Lebedev	2cecde89c3	rename opt volumesPerStep	2021-10-01 00:17:54 +05:00
Konstantin Lebedev	fc51ffce2b	https://github.com/chrislusf/seaweedfs/issues/1846	2021-09-30 20:24:24 +05:00
Chris Lu	ad5099e570	refactor	2021-09-19 12:02:23 -07:00
Chris Lu	2789d10342	go fmt	2021-09-14 10:37:06 -07:00
Chris Lu	119d5908dd	shell: do not need to lock to see volume -h	2021-09-13 22:13:34 -07:00
Chris Lu	6cd1ce8b74	erasure coding: add cleanup step if anything goes wrong	2021-09-13 01:55:49 -07:00
Chris Lu	e5fc35ed0c	change server address from string to a type	2021-09-12 22:47:52 -07:00
Chris Lu	0b4269b6a8	remember commands even if failed	2021-09-08 15:55:19 -07:00
Chris Lu	0128239c0f	handle ipv6 addresses	2021-09-07 16:43:54 -07:00
Chris Lu	8e4b43a017	minor	2021-09-06 15:13:38 -07:00
Chris Lu	57a95887d2	remote.cache remote.uncache supports all mounted directories	2021-09-05 14:47:06 -07:00
Chris Lu	c735608685	obfuscate secret keys on display	2021-09-05 14:23:49 -07:00
Chris Lu	32e94de86a	default auto trimming suffix to true	2021-09-05 13:27:35 -07:00
Chris Lu	60573fd3e2	option to map remote bucket to trimmed bucket name	2021-09-05 11:55:52 -07:00
Chris Lu	d57d4c5f8f	shell: add remote.mount.buckets	2021-09-04 21:37:25 -07:00
Chris Lu	d983aa4c7d	correct filtering	2021-09-04 13:58:14 -07:00
Chris Lu	49952ba905	fix cache/uncache filters	2021-09-04 13:45:39 -07:00
Chris Lu	df29281536	rename	2021-09-04 05:11:06 -07:00
Chris Lu	c3db389e42	refactor	2021-09-04 05:09:38 -07:00
Chris Lu	8707ef00a6	refactor	2021-09-04 05:02:20 -07:00
Chris Lu	63c84584bb	refactor	2021-09-04 04:53:36 -07:00
Chris Lu	da49d25950	auto list of storage types	2021-09-04 00:18:21 -07:00
Chris Lu	38f73c8324	add gcs project id	2021-09-03 23:38:33 -07:00
Chris Lu	7ce97b59d8	go fmt	2021-09-01 02:45:42 -07:00
Chris Lu	3bd48c4f29	filer.remote.sync: exit when directory is unmounted ... this will not propagate the deletions back to the cloud	2021-09-01 01:29:22 -07:00
Chris Lu	cca588429b	shell: fs.meta.save skip saving system logs	2021-08-31 23:05:56 -07:00
Chris Lu	0ee208c1a4	adjust example	2021-08-31 02:04:32 -07:00
Chris Lu	97b7948cd9	adjust error message	2021-08-31 01:07:34 -07:00
Chris Lu	c9205458a1	cloud drive: create mount directory if not exists	2021-08-31 01:07:19 -07:00
Chris Lu	1dfcdc53b7	cloud drive: add storj, filebase	2021-08-30 17:43:14 -07:00
Chris Lu	701cc133f3	cloud drive: s3 add option for v4 signature	2021-08-30 17:28:33 -07:00
Chris Lu	f117f7bf54	adjust comment	2021-08-30 15:56:14 -07:00
Chris Lu	44d0ac95f6	add cluster id if already set	2021-08-29 22:19:46 -07:00
Chris Lu	18bf556d93	fix comments	2021-08-29 22:19:25 -07:00
Chris Lu	21f37a0a75	remove unused code	2021-08-29 22:19:11 -07:00
Chris Lu	001a472057	cloud mount: remote storage support hdfs	2021-08-29 18:41:29 -07:00
Chris Lu	975fbc9ee3	add wasabi example	2021-08-28 22:49:38 -07:00
Chris Lu	65a29ad42a	add warning on unmount a folder	2021-08-28 02:25:48 -07:00
Chris Lu	a22f37b01c	remove tencent region	2021-08-28 02:23:03 -07:00
Chris Lu	0d8936f6cd	add aliyun example	2021-08-28 01:09:48 -07:00
Chris Lu	49a8dfb976	adjust default concurrent level	2021-08-26 17:05:56 -07:00
Chris Lu	6a0bb7106b	cloud drive: parallelize remote storage downloading	2021-08-26 16:16:26 -07:00
Chris Lu	05a648bb96	refactor: separating out remote.proto	2021-08-26 15:18:34 -07:00
Chris Lu	c08ac536ed	cloud drive: add support for Wasabi ... * disable md5, sha256 checking to avoid reading one chunk twice * single threaded upload to avoid chunk swapping (to be enhanced later)	2021-08-25 17:34:29 -07:00
Chris Lu	a7a914f120	cloud drive: add support for Baidu BOS	2021-08-24 23:46:33 -07:00
Chris Lu	2d805beed4	add help mesages	2021-08-24 23:28:35 -07:00
Chris Lu	c6c97bd83d	add default env variables	2021-08-24 23:25:36 -07:00
Chris Lu	a19c728034	cloud drive: add support for Tencent COS	2021-08-24 23:19:45 -07:00
Chris Lu	19a81d25af	cloud drive: add support for Aliyun OSS	2021-08-24 23:14:24 -07:00
Chris Lu	47d775cf68	cloud drive: add support for BackBlaze	2021-08-24 22:30:06 -07:00
Chris Lu	6bab20d862	cat entry content	2021-08-24 02:38:32 -07:00
Chris Lu	e9ebe24f2e	cloud drive: add support for Azure	2021-08-24 01:18:30 -07:00
Chris Lu	00c4e06caa	cloud drive: s3 configurable force path style	2021-08-23 03:30:41 -07:00
Chris Lu	2836a58d87	cloud drive: S3 supports storage class	2021-08-23 02:18:59 -07:00
Chris Lu	ca35a77e85	adjust help message	2021-08-23 00:37:55 -07:00
Chris Lu	258063de26	cloud drive: add google cloud storage	2021-08-23 00:29:27 -07:00
Chris Lu	2158d4fe4d	adjust help message	2021-08-21 02:17:10 -07:00
Chris Lu	ccb3df41f8	elide secret key	2021-08-17 11:27:08 -07:00
Chris Lu	49b5e47bd1	retry forever with filer.remote.sync, and some refactoring	2021-08-15 12:38:26 -07:00
Chris Lu	a539d64896	refactor	2021-08-15 12:09:54 -07:00
Chris Lu	ec989b0377	remove println	2021-08-15 02:10:27 -07:00
Chris Lu	9462f5129a	shell: add "remote.meta.sync"	2021-08-15 01:53:46 -07:00
Chris Lu	c34747c79d	rename, fix wrong logic.	2021-08-14 21:46:34 -07:00
Chris Lu	cb53802752	adjust help message	2021-08-14 15:55:53 -07:00
Chris Lu	889b143fa7	adjust modification detection logic	2021-08-14 15:44:47 -07:00
Chris Lu	708debca14	remote.cache and uncache: more flexible options to select files to cache or uncache	2021-08-14 15:11:55 -07:00
Chris Lu	0db2517994	go fmt	2021-08-14 02:55:44 -07:00
Chris Lu	0f7d4556d8	shell: volume.tier.move makes up changes if volume move failed	2021-08-13 03:09:28 -07:00
Chris Lu	333cdce485	add verbose message	2021-08-13 02:57:14 -07:00
Chris Lu	057ef429ac	format	2021-08-10 12:33:29 -07:00
Chris Lu	b63b042afc	dedup keeps the largest replica	2021-08-10 12:30:41 -07:00
Chris Lu	0526db12e2	do not treat read only volumes differently	2021-08-10 11:37:12 -07:00
Chris Lu	e50a5b8e28	minor: print disk type	2021-08-10 11:10:09 -07:00
Chris Lu	db6275a0c8	print out balance ratio	2021-08-10 10:43:42 -07:00
Chris Lu	85832d02c0	wait for goroutines	2021-08-10 04:13:12 -07:00
Chris Lu	8ff6c9a0c6	output format	2021-08-10 03:25:18 -07:00
Chris Lu	48f448ee09	parallelize tier move	2021-08-10 03:08:29 -07:00
Chris Lu	69a6da7969	avoid fail on tail error	2021-08-10 02:50:28 -07:00
Chris Lu	18228f3044	fix help message	2021-08-10 02:48:41 -07:00
Chris Lu	02f728cb54	fix bug with remote.uncache	2021-08-09 15:24:21 -07:00
Chris Lu	402315f117	go fmt	2021-08-09 14:37:34 -07:00
Chris Lu	713c035a6e	shell: remote.cache remote.uncache	2021-08-09 14:35:18 -07:00
Chris Lu	c5f38c365d	go fmt	2021-08-08 22:30:36 -07:00
Chris Lu	df85f7a1eb	adjust help message	2021-08-08 22:30:12 -07:00
Chris Lu	c0b12da4ef	shell: add filer.remote.unmount	2021-08-08 22:26:37 -07:00
Chris Lu	882a93dacd	fix tests	2021-08-08 17:56:26 -07:00
Chris Lu	eed26af266	Merge branch 'master' into add_remote_storage	2021-08-08 15:48:04 -07:00
Chris Lu	26c222f596	shell: volume.tier.move avoid moving all volumes to one destination	2021-08-08 15:12:39 -07:00
Chris Lu	13e45e1605	filer.remote.sync can work now	2021-08-08 01:21:42 -07:00
Chris Lu	679f800caa	Merge branch 'master' into add_remote_storage	2021-08-06 20:27:12 -07:00
Chris Lu	0c0f77e2ae	skip not found error on deletion	2021-08-06 19:35:47 -07:00
Chris Lu	8828f485c0	print volume deletion error	2021-08-06 19:30:22 -07:00
Chris Lu	e23ca3845b	Merge pull request #2237 from combineads/add_mtime ... Add crtime and mtime	2021-08-05 21:27:13 -07:00
Chris Lu	286e5dd375	Merge branch 'master' into add_remote_storage	2021-08-05 21:07:04 -07:00
Chris Lu	1e22166939	adjust error message	2021-08-05 21:06:55 -07:00
byunghwa.yun	e84fad9acf	Add crtime and mtime	2021-08-06 12:13:37 +09:00
Chris Lu	1a5d29520c	add tests	2021-08-05 14:49:24 -07:00
Chris Lu	6b743dbbf9	refactor client subscribe metadata	2021-08-04 16:25:46 -07:00
Chris Lu	b9ecf1e3a8	refacotring	2021-08-04 14:56:13 -07:00
Chris Lu	d84c311699	refactoring	2021-08-04 12:30:18 -07:00
Chris Lu	f6a9ad8001	fix tests	2021-08-04 00:31:06 -07:00
Chris Lu	42969c9c62	Merge branch 'master' into add_remote_storage	2021-08-03 21:20:05 -07:00
Chris Lu	8cf0c515bf	shell: volume.fix.replication retries even when there is no error #2235 ... fix https://github.com/chrislusf/seaweedfs/issues/2235	2021-08-03 09:32:55 -07:00
Chris Lu	2ae9705442	adjust text	2021-08-01 22:55:19 -07:00
Chris Lu	3739717092	Revert "adds a test" ... This reverts commit f690643b47.	2021-08-01 22:54:45 -07:00
Chris Lu	f690643b47	adds a test	2021-08-01 22:53:50 -07:00
Chris Lu	6de786185d	volume.balance: balance read only volumes first	2021-08-01 22:32:50 -07:00
byunghwa.yun	9e839cb5cc	Add force option in volume.deleteEmpty command	2021-08-02 11:37:20 +09:00
Chris Lu	81b255df8b	Merge branch 'master' into add_remote_storage	2021-08-01 15:44:21 -07:00
Chris Lu	2ca1839d77	shell: add volume.deleteEmpty command	2021-08-01 15:33:45 -07:00
Chris Lu	9df7d16791	read <- remote_storage	2021-07-31 22:39:38 -07:00
Chris Lu	f5a69a0e44	Merge branch 'master' into add_remote_storage	2021-07-31 09:39:48 -07:00
byunghwa.yun	5d4438a72f	Fix typo	2021-07-31 22:23:52 +09:00
Chris Lu	a3290faf17	shell command to calculate size / number of volumes in a collection ... fix https://github.com/chrislusf/seaweedfs/issues/2224	2021-07-30 15:49:46 -07:00
Chris Lu	c6f992b2a3	remove dead code	2021-07-30 15:18:01 -07:00
Chris Lu	899963ac20	remote storage location changed to struct	2021-07-29 02:08:55 -07:00
Chris Lu	4deac06da0	log fix	2021-07-28 22:25:04 -07:00
divanikus	5c6270a93a	shell: ability to use wildcards for collections, all collections if ommited	2021-07-27 23:53:01 +03:00
Chris Lu	1752eeb538	remote.mount saves the mapping	2021-07-27 03:26:35 -07:00
Chris Lu	4b94b03d90	directory to remote storage mapping	2021-07-27 01:16:28 -07:00
Chris Lu	99b599aa8a	remote.mount	2021-07-26 22:53:44 -07:00
Chris Lu	35f70c51b0	refactor	2021-07-25 03:37:37 -07:00
Chris Lu	72eec84167	shell: add fs.mkdir	2021-07-25 02:37:39 -07:00
Chris Lu	092932af8f	shell: faster bucket delete ... avoid waiting for large object metadata	2021-07-23 11:25:46 -07:00
Chris Lu	7359193e97	go fmt	2021-07-21 14:38:12 -07:00
Chris Lu	67537ff05b	Merge pull request #2208 from combineads/add_rm_options	2021-07-21 11:08:43 -07:00
byunghwa.yun	41d559f54e	Add fs rm options	2021-07-21 21:10:36 +09:00
Chris Lu	70effac0d3	configure and store remote configurations	2021-07-21 02:24:34 -07:00
Chris Lu	3138805b33	Merge pull request #2200 from combineads/add_fs_rm ... weed shell: fs.rm remove file or directory	2021-07-21 00:35:54 -07:00
Chris Lu	7ab389e7ec	optimization: improve random range query for large files	2021-07-19 23:07:22 -07:00
byunghwa.yun	b23b307e08	Add fs rm	2021-07-20 08:49:28 +09:00
Chris Lu	99155e4880	add -retry option for volume.fix.replication ... fix https://github.com/chrislusf/seaweedfs/issues/2191	2021-07-16 12:13:46 -07:00
Chris Lu	2f209675ab	Added -retry option for volumeServer.evacuate ... related to https://github.com/chrislusf/seaweedfs/issues/2191	2021-07-16 12:08:21 -07:00
Chris Lu	fb7a1be1c4	refactor	2021-07-16 11:47:37 -07:00
Chris Lu	4be5b4ff5c	[volume.check.disk] skip errors and check next volumes ... fix https://github.com/chrislusf/seaweedfs/issues/2129	2021-07-16 02:03:32 -07:00
Chris Lu	5a838dbe53	fix compilation	2021-07-13 11:40:21 -07:00
Chris Lu	6103649ffb	shell: volume.check.disk adds retries in case the volumes are just moved ... related to https://github.com/chrislusf/seaweedfs/issues/2194	2021-07-13 11:19:56 -07:00
Chris Lu	01adc567aa	shell: volume.fsck deletes a volume is the volume has only orphaned data. ... fix https://github.com/chrislusf/seaweedfs/issues/2190	2021-07-12 11:22:00 -07:00
bingoohuang	9e48bff8ee	fix typo in fs.configure help message text	2021-07-12 13:51:21 +08:00
Chris Lu	d013d6d968	shell: volume.fsck "reallyDeleteFromVolume" should send padded file ids to delete ... fix https://github.com/chrislusf/seaweedfs/issues/2188	2021-07-10 23:16:06 -07:00
Chris Lu	3d624d1e16	rename	2021-07-09 03:19:21 -07:00
Chris Lu	da7bd62822	/etc files are stored inside metadata store	2021-07-09 02:33:14 -07:00
Chris Lu	08377fecb8	fixes	2021-06-27 23:32:57 -07:00
Chris Lu	1cac2f2278	shell: volume.fsck checks missing chunks in filer ... fix https://github.com/chrislusf/seaweedfs/issues/2154	2021-06-24 23:56:24 -07:00
Chris Lu	9dd09bbb33	refactor	2021-06-24 17:22:53 -07:00
Daniel Nagy	7ce3bee0c0	Check for history file errors as well	2021-06-08 11:14:49 +02:00
Chris Lu	ab606dec2a	filer: add path-specific option to enforce readonly	2021-06-04 01:03:41 -07:00
Chris Lu	615cb24ba6	shell: use tab print style for completion	2021-05-31 03:29:29 -07:00
Chris Lu	b70aeb7585	bucket list only directories	2021-05-23 10:36:22 -07:00
Chris Lu	30c67e3652	minor	2021-05-19 23:59:39 -07:00
Chris Lu	76c48ffe27	optional parallel copy ec shards ... fix https://github.com/chrislusf/seaweedfs/issues/2048	2021-05-06 01:53:35 -07:00
Chris Lu	ac71117ee6	revert PR #1903 avoid http error: superfluous response.WriteHeader	2021-05-05 15:11:39 -07:00
Chris Lu	f0ad172e80	shell: show which server holds the lock ... fix https://github.com/chrislusf/seaweedfs/issues/1983	2021-04-22 23:56:35 -07:00
Chris Lu	46ef1811a1	correct help message	2021-04-22 22:26:38 -07:00
Chris Lu	b971317a16	avoid possible corrupted file names	2021-04-15 11:41:34 -07:00
Chris Lu	609e228578	avoid forward slash in file names	2021-04-15 10:53:04 -07:00
Chris Lu	ba92f2e714	add node.selectedVolumes ... fix https://github.com/chrislusf/seaweedfs/issues/1990	2021-04-15 03:19:28 -07:00
Chris Lu	fc0cbf565f	add option to obfuscate the file names	2021-04-14 15:37:24 -07:00
Chris Lu	e75633c64f	volume.check.disk: break loop for read only volumes ... fix https://github.com/chrislusf/seaweedfs/issues/2002	2021-04-14 12:40:13 -07:00
Chris Lu	9d50867d08	volume.tier.move: avoid data loss when destination volume server already has the volume ... fix https://github.com/chrislusf/seaweedfs/issues/2001	2021-04-14 10:26:26 -07:00
Chris Lu	90677e1097	ensure to call line.Close() ... fix https://github.com/chrislusf/seaweedfs/issues/1995 similar to https://github.com/peterh/liner/issues/104	2021-04-14 01:33:21 -07:00
Konstantin Lebedev	020a5d40c3	avoid counter is null	2021-03-24 22:51:22 +05:00
Konstantin Lebedev	df6cf0a2fa	nonRepairThreshold	2021-03-24 22:07:13 +05:00
Konstantin Lebedev	60972f1c97	rename option repairThreshold	2021-03-24 13:24:49 +05:00
Konstantin Lebedev	40a9e88c07	rename var	2021-03-23 21:42:35 +05:00
Konstantin Lebedev	d21a5bf5d0	add missingKeysThreshold for fsck	2021-03-23 15:04:07 +05:00
Chris Lu	8eb14967ef	shell: volume.check.disk adds optional slow mode ... fix https://github.com/chrislusf/seaweedfs/issues/1930	2021-03-22 21:01:43 -07:00
Chris Lu	f315eb2bb8	fix build error	2021-03-22 00:12:53 -07:00
Chris Lu	b465095db1	shell: add volume.check.disk to fix inconsistency for replicated volumes ... fix https://github.com/chrislusf/seaweedfs/issues/1923	2021-03-22 00:03:16 -07:00
Chris Lu	27e24162f2	minor	2021-03-20 06:37:03 -07:00
Chris Lu	41cc6e661b	fs.mv ensures there are 2 arguments	2021-03-20 06:34:13 -07:00
Chris Lu	b2c1c209a5	refactor	2021-03-20 06:34:13 -07:00
Chris Lu	e9b1853d63	go fmt	2021-03-19 01:31:56 -07:00
Konstantin Lebedev	06da02739d	CheckAllChunkViews() for HEAD requests only	2021-03-16 14:15:17 +05:00
Chris Lu	a4cfffc264	shell: fix moving volume, volume server evacuate ... fix https://github.com/chrislusf/seaweedfs/issues/1534	2021-03-14 21:29:55 -07:00
Chris Lu	ca100568f7	fix parsing	2021-03-14 20:50:14 -07:00
Chris Lu	352ba23f83	revert previous change ... revert 29e62aba00	2021-03-14 20:49:56 -07:00
Chris Lu	29e62aba00	possible fix for volume balance ... address https://github.com/chrislusf/seaweedfs/issues/1534	2021-03-14 20:23:19 -07:00
Chris Lu	35d939a1d2	go fmt	2021-03-14 13:21:02 -07:00
Chris Lu	cca66c7fbe	print out the url	2021-03-11 18:39:45 -08:00
Chris Lu	7f887c78d2	fix	2021-03-11 18:18:16 -08:00
Chris Lu	fcc52d1e4e	shell: add s3.clean.uploads to clean up stale multipart uploads ... fix https://github.com/chrislusf/seaweedfs/issues/1855	2021-03-11 18:05:24 -08:00