chrislu
67814a5c79
refactor and fix strings.Split
2022-08-07 01:34:32 -07:00
Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center ( #3405 )
...
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404
* GetDataCenter
* prefer same data center for ReplicationSource
* GetDataCenterId
* remove glog
2022-08-04 17:35:00 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
chrislu
a3fae7cd16
add an empty placeholder for mq.topic.list
2022-07-28 23:24:38 -07:00
chrislu
9f479aab98
allocate brokers to serve segments
2022-07-28 23:24:38 -07:00
chrislu
f25e273e32
display data center and rack in cluster.ps
2022-07-28 23:22:52 -07:00
chrislu
68065128b8
add dc and rack
2022-07-28 23:22:51 -07:00
chrislu
87ccef72db
cluster.ps can list brokers
2022-07-28 23:22:07 -07:00
chrislu
7a6c559ab4
fix Change replication via volume.configure.replication by collection
...
fix https://github.com/chrislusf/seaweedfs/issues/3346
2022-07-21 22:01:05 -07:00
Chris Lu
4820b594a7
Merge pull request #3298 from kmlebedev/evacuateTargetServer
...
volume server evacuate to target server
2022-07-19 13:56:49 -07:00
Konstantin Lebedev
d3f7c09c03
remove ticker
...
update the topology before each file
2022-07-20 00:54:23 +05:00
Konstantin Lebedev
d422e7769c
ticker.Stop
2022-07-18 16:38:19 +05:00
Konstantin Lebedev
73a0dea16b
sync update topologyInfo
2022-07-18 16:27:02 +05:00
Konstantin Lebedev
2b4112e462
update otherNodes
2022-07-18 11:32:28 +05:00
Konstantin Lebedev
3c2774ec3d
fix update topologyInfo
2022-07-18 01:46:31 +05:00
Konstantin Lebedev
4d5144e50d
clouse background update
2022-07-15 13:51:08 +05:00
guosj
ab1b9697e6
supplement check duplicate accesskey
2022-07-13 17:28:20 +08:00
石昌林
392da4e038
Make the prompt information clearer
2022-07-13 11:39:45 +08:00
石昌林
ad5e8f68ec
Check whether there is a duplicate accessKey when modifying iam
2022-07-13 11:36:11 +08:00
Konstantin Lebedev
6622240df7
fix TestVolumeServerEvacuate
2022-07-12 14:56:34 +05:00
Konstantin Lebedev
8372721a62
update topologyInfo
2022-07-12 13:47:21 +05:00
Konstantin Lebedev
ee95d23a22
help rack
2022-07-12 11:56:58 +05:00
Konstantin Lebedev
087fa1347f
volume server evacuate from rack
2022-07-12 11:33:08 +05:00
Konstantin Lebedev
4236c36599
volume server evacuate to target server
2022-07-11 16:58:15 +05:00
chrislu
57975b93ff
adjust used size reporting
2022-07-04 13:47:42 -07:00
chrislu
a79f5bd0d5
adjust used size reporting
2022-07-04 13:46:32 -07:00
guol-fnst
5bc895673e
add getting bucket quota
2022-07-04 15:36:44 +08:00
guol-fnst
dd4664e610
fix usage
2022-07-04 11:31:39 +08:00
chrislu
141f662734
edge case checking when volume server does not have capacity to balance
...
fix https://github.com/chrislusf/seaweedfs/issues/3257
2022-07-01 12:16:18 -07:00
chrislu
bee3a7c798
fix tests
2022-07-01 12:09:42 -07:00
Konstantin Lebedev
5ed8165161
fix logic
...
add option targetServer
https://github.com/chrislusf/seaweedfs/issues/3255
2022-06-30 17:58:24 +05:00
Konstantin Lebedev
bcbdc4cb37
use const multipart uploads folder
...
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
chrislu
1d0c53ea56
remote storage: stop supporting hdfs as a remote storage
2022-06-20 14:15:59 -07:00
石昌林
9e036df356
remove go.uber.org/atomic
2022-06-20 12:35:29 +08:00
石昌林
3dd60529c5
some code optimizations
2022-06-17 19:07:39 +08:00
石昌林
37df209195
add some unit tests and some code optimizes
2022-06-17 17:11:18 +08:00
石昌林
78b3728169
add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
...
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply
# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply
# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply
# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply
# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
2022-06-15 21:07:55 +08:00
chrislu
d12f431d98
collect volume server status
2022-06-12 11:56:23 -07:00
shichanglin5
a63559ee6c
fix: fix command prompt information
2022-06-10 12:46:10 +08:00
chrislu
4fd5f96598
filer: remove replication, collection, disk_type info from entry metadata
...
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
chrislu
f97acdd489
volume.fix.replication fix retry logic
...
fix https://github.com/chrislusf/seaweedfs/issues/3136
2022-06-03 08:45:29 -07:00
chrislu
6793bc853c
help message when in simulation mode
2022-05-31 14:48:46 -07:00
chrislu
89948a373b
fix error reporting for "Need to a hdd disk type"
...
related to https://github.com/chrislusf/seaweedfs/issues/3128
2022-05-31 12:43:55 -07:00
chrislu
5b9347c938
typo
2022-05-30 12:37:41 -07:00
chrislu
ca16fbf0ef
shell: cluster.ps display filer group
2022-05-29 16:37:14 -07:00
Konstantin Lebedev
44f53ceda6
fix collectionIsMismatch charset
2022-05-16 13:23:23 +05:00
Konstantin Lebedev
10d435f2c2
fix skip loop
2022-05-16 13:16:27 +05:00
Konstantin Lebedev
279053572c
avoid delete volume replica if collection mismatch
2022-05-16 13:07:05 +05:00
chrislu
271b5aed96
shell: volume.fsck add a note for -reallyDeleteFromVolume option
2022-05-15 11:07:04 -07:00
chrislu
94635e9b5c
filer: add filer group
2022-05-01 21:59:16 -07:00
guosj
1a5de36257
fix return value
2022-04-29 11:19:36 +08:00
Chris Lu
b557faf175
Merge pull request #2967 from kmlebedev/vl_chk_disk_vlid
...
volume.check.disk add param volumeId
2022-04-26 22:09:06 -07:00
Konstantin Lebedev
d4343ab7da
forcePurging desc
2022-04-25 23:11:56 +05:00
Konstantin Lebedev
ae56b2c00f
change forcePurging to a pointer
2022-04-25 23:10:01 +05:00
Konstantin Lebedev
6d2fda27d2
delete missing data from volumes in one replica
2022-04-25 22:59:46 +05:00
Konstantin Lebedev
67476d830a
volume.check.disk add param volumeId
2022-04-25 18:27:14 +05:00
Chris Lu
88a669dd19
Merge pull request #2858 from kmlebedev/fsck_replicas
...
fsck replicas
2022-04-19 00:17:43 -07:00
Konstantin Lebedev
2364fab927
volume.list show only readonly, collectionPattern and volumeId
2022-04-19 11:44:41 +05:00
Konstantin Lebedev
36c5a59ed8
add help
2022-04-18 19:36:14 +05:00
Konstantin Lebedev
1e35b4929f
shell vacuum volume by collection and volume id
2022-04-18 18:40:58 +05:00
leyou240
89eb87c1d1
Merge branch 'master' into slices.SortFunc
2022-04-18 10:39:29 +08:00
justin
3551ca2fcf
enhancement: replace sort.Slice with slices.SortFunc to reduce reflection
2022-04-18 10:35:43 +08:00
chrislu
460d56d283
shell: cluster.check prints out clock delta and network latency
2022-04-16 13:24:17 -07:00
chrislu
1f03fcccb1
fix nil in cluster_check shell command
...
fix https://github.com/chrislusf/seaweedfs/issues/2905
2022-04-12 08:47:27 -07:00
Konstantin Lebedev
76b1c5ce67
cleanupUploads use jwt token
2022-04-12 13:46:11 +05:00
Konstantin Lebedev
2dfbe210e0
avoid breaking loop in cleanupUploads if error is empty
2022-04-11 11:16:20 +05:00
Konstantin Lebedev
931cb9e581
use "cluster.raft.{ps,add,remove}"
2022-04-11 10:50:01 +05:00
Konstantin Lebedev
f5246b748d
Merge branch 'new_master' into hashicorp_raft
...
# Conflicts:
# weed/pb/master_pb/master.pb.go
2022-04-07 18:50:27 +05:00
Konstantin Lebedev
85d80fd36d
fix removing old raft server
2022-04-07 15:31:37 +05:00
Konstantin Lebedev
357aa818fe
add raft shell cmds
2022-04-06 15:23:53 +05:00
chrislu
bc888226fc
erasure coding: tracking encoded/decoded volumes
...
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
2022-04-05 19:03:02 -07:00
chrislu
4aae87f405
check missing hdd disk type
2022-04-04 14:48:00 -07:00
chrislu
6a2bcd03aa
configure mount quota
2022-04-02 21:34:26 -07:00
chrislu
105578a2f2
skip pinging self for master and volume server
2022-04-01 20:25:35 -07:00
chrislu
4ecba915f3
add check between peers
2022-04-01 17:40:25 -07:00
chrislu
4b5c0e3fa9
check cluster connectivities
2022-04-01 17:27:49 -07:00
Konstantin Lebedev
7f1383a41e
findExtraChunksInVolumeServers in consideration of replication
2022-04-01 14:45:41 +05:00
Konstantin Lebedev
3817e05dd0
fix collect filer files
2022-04-01 10:17:09 +05:00
Konstantin Lebedev
3cedb21bb7
skip new entities
2022-03-31 21:36:10 +05:00
Konstantin Lebedev
1d9e30d8c0
fsck replicas
2022-03-31 19:10:06 +05:00
Konstantin Lebedev
4f5b018a02
collecting only bucket volumes
2022-03-31 13:35:58 +05:00
chrislu
21e0898631
refactor: change masters from a slice to a map
2022-03-26 13:33:17 -07:00
banjiaojuhao
f7f2a597dd
minor
2022-03-08 16:22:55 +08:00
chrislu
ede6ce44c6
fix test
2022-03-06 17:09:55 -08:00
chrislu
6e49e75a5b
use logical number of files and sizes for statistics and quota
2022-03-04 18:47:44 -08:00
chrislu
95717d1006
simpler output
2022-02-24 13:50:08 -08:00
Chris Lu
6834df77a0
Merge pull request #2701 from guo-sj/fix_bugs_in_return_value
2022-02-23 19:23:37 -08:00
guosj
26f3ab8d4b
fix a return bug in func (c *commandVacuum) Do
2022-02-24 09:04:38 +08:00
Chris Lu
38dcaaa76e
Merge pull request #2683 from guo-sj/fix_fsconfigure_bug
...
fix minor bug in commandFsConfigure.Do
2022-02-16 18:16:47 -08:00
guosj
ca121ecd66
fix minor bug in commandFsConfigure.Do
2022-02-17 09:57:52 +08:00
Chris Lu
64afbc5235
Merge pull request #2640 from Radtoo/fsck_with_delete
...
Added basic deletion capabilities to fsck.
2022-02-14 23:00:01 -08:00
Radtoo
724ce04b1c
We picked the second flag's name.
2022-02-13 00:53:35 +01:00
Radtoo
fbe7ed7927
Revert the the flag to "reallyDeleteFromVolume".
2022-02-10 20:01:48 +01:00
chrislu
21aaa4c1f1
ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount)
...
fix https://github.com/chrislusf/seaweedfs/issues/2642
2022-02-08 01:51:13 -08:00
chrislu
13f6ec1c4e
test checking ec distribution
2022-02-08 01:50:05 -08:00
chrislu
f34c2ff7c5
use embed txt
2022-02-08 01:14:27 -08:00
chrislu
c8c7c10c3f
volume.tier.move: avoid double counting
...
related to https://github.com/chrislusf/seaweedfs/issues/2637
2022-02-08 00:57:35 -08:00
chrislu
f18803424a
volume.balance: add delay during tight loop
...
fix https://github.com/chrislusf/seaweedfs/issues/2637
2022-02-08 00:53:55 -08:00
chrislu
85c1615b43
filer read empty file may cause OOM in some cases
...
fix https://github.com/chrislusf/seaweedfs/issues/2641
2022-02-07 23:08:54 -08:00
Radtoo
fbb14e0ea8
Adding separate toggle to purge absent vols
...
While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected.
Also renamed forcePurge to just purge.
2022-02-06 23:46:52 +01:00
