Commit Graph

601 Commits

Author SHA1 Message Date
chrislu
1d0c53ea56 remote storage: stop supporting hdfs as a remote storage 2022-06-20 14:15:59 -07:00
石昌林
9e036df356 remove go.uber.org/atomic 2022-06-20 12:35:29 +08:00
石昌林
3dd60529c5 some code optimizations 2022-06-17 19:07:39 +08:00
石昌林
37df209195 add some unit tests and some code optimizes 2022-06-17 17:11:18 +08:00
石昌林
78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
2022-06-15 21:07:55 +08:00
chrislu
d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
shichanglin5
a63559ee6c fix: fix command prompt information 2022-06-10 12:46:10 +08:00
chrislu
4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
chrislu
f97acdd489 volume.fix.replication fix retry logic
fix https://github.com/chrislusf/seaweedfs/issues/3136
2022-06-03 08:45:29 -07:00
chrislu
6793bc853c help message when in simulation mode 2022-05-31 14:48:46 -07:00
chrislu
89948a373b fix error reporting for "Need to a hdd disk type"
related to https://github.com/chrislusf/seaweedfs/issues/3128
2022-05-31 12:43:55 -07:00
chrislu
5b9347c938 typo 2022-05-30 12:37:41 -07:00
chrislu
ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
Konstantin Lebedev
44f53ceda6 fix collectionIsMismatch charset 2022-05-16 13:23:23 +05:00
Konstantin Lebedev
10d435f2c2 fix skip loop 2022-05-16 13:16:27 +05:00
Konstantin Lebedev
279053572c avoid delete volume replica if collection mismatch 2022-05-16 13:07:05 +05:00
chrislu
271b5aed96 shell: volume.fsck add a note for -reallyDeleteFromVolume option 2022-05-15 11:07:04 -07:00
chrislu
94635e9b5c filer: add filer group 2022-05-01 21:59:16 -07:00
guosj
1a5de36257 fix return value 2022-04-29 11:19:36 +08:00
Chris Lu
b557faf175
Merge pull request #2967 from kmlebedev/vl_chk_disk_vlid
volume.check.disk add param volumeId
2022-04-26 22:09:06 -07:00
Konstantin Lebedev
d4343ab7da forcePurging desc 2022-04-25 23:11:56 +05:00
Konstantin Lebedev
ae56b2c00f change forcePurging to a pointer 2022-04-25 23:10:01 +05:00
Konstantin Lebedev
6d2fda27d2 delete missing data from volumes in one replica 2022-04-25 22:59:46 +05:00
Konstantin Lebedev
67476d830a volume.check.disk add param volumeId 2022-04-25 18:27:14 +05:00
Chris Lu
88a669dd19
Merge pull request #2858 from kmlebedev/fsck_replicas
fsck replicas
2022-04-19 00:17:43 -07:00
Konstantin Lebedev
2364fab927 volume.list show only readonly, collectionPattern and volumeId 2022-04-19 11:44:41 +05:00
Konstantin Lebedev
36c5a59ed8 add help 2022-04-18 19:36:14 +05:00
Konstantin Lebedev
1e35b4929f shell vacuum volume by collection and volume id 2022-04-18 18:40:58 +05:00
leyou240
89eb87c1d1
Merge branch 'master' into slices.SortFunc 2022-04-18 10:39:29 +08:00
justin
3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
chrislu
460d56d283 shell: cluster.check prints out clock delta and network latency 2022-04-16 13:24:17 -07:00
chrislu
1f03fcccb1 fix nil in cluster_check shell command
fix https://github.com/chrislusf/seaweedfs/issues/2905
2022-04-12 08:47:27 -07:00
Konstantin Lebedev
76b1c5ce67 cleanupUploads use jwt token 2022-04-12 13:46:11 +05:00
Konstantin Lebedev
2dfbe210e0 avoid breaking loop in cleanupUploads if error is empty 2022-04-11 11:16:20 +05:00
Konstantin Lebedev
931cb9e581 use "cluster.raft.{ps,add,remove}" 2022-04-11 10:50:01 +05:00
Konstantin Lebedev
f5246b748d Merge branch 'new_master' into hashicorp_raft
# Conflicts:
#	weed/pb/master_pb/master.pb.go
2022-04-07 18:50:27 +05:00
Konstantin Lebedev
85d80fd36d fix removing old raft server 2022-04-07 15:31:37 +05:00
Konstantin Lebedev
357aa818fe add raft shell cmds 2022-04-06 15:23:53 +05:00
chrislu
bc888226fc erasure coding: tracking encoded/decoded volumes
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
2022-04-05 19:03:02 -07:00
chrislu
4aae87f405 check missing hdd disk type 2022-04-04 14:48:00 -07:00
chrislu
6a2bcd03aa configure mount quota 2022-04-02 21:34:26 -07:00
chrislu
105578a2f2 skip pinging self for master and volume server 2022-04-01 20:25:35 -07:00
chrislu
4ecba915f3 add check between peers 2022-04-01 17:40:25 -07:00
chrislu
4b5c0e3fa9 check cluster connectivities 2022-04-01 17:27:49 -07:00
Konstantin Lebedev
7f1383a41e findExtraChunksInVolumeServers in consideration of replication 2022-04-01 14:45:41 +05:00
Konstantin Lebedev
3817e05dd0 fix collect filer files 2022-04-01 10:17:09 +05:00
Konstantin Lebedev
3cedb21bb7 skip new entities 2022-03-31 21:36:10 +05:00
Konstantin Lebedev
1d9e30d8c0 fsck replicas 2022-03-31 19:10:06 +05:00
Konstantin Lebedev
4f5b018a02 collecting only bucket volumes 2022-03-31 13:35:58 +05:00
chrislu
21e0898631 refactor: change masters from a slice to a map 2022-03-26 13:33:17 -07:00