Commit Graph

235 Commits

Author SHA1 Message Date
Chris Lu
a87f57e47c
Merge pull request #2868 from kmlebedev/hashicorp_raft
hashicorp raft
2022-04-10 23:00:05 -07:00
Konstantin Lebedev
f5246b748d Merge branch 'new_master' into hashicorp_raft
# Conflicts:
#	weed/pb/master_pb/master.pb.go
2022-04-07 18:50:27 +05:00
shibinbin
c20e1edd99 fix: master lose some volumes 2022-04-07 15:18:28 +08:00
chrislu
bc888226fc erasure coding: tracking encoded/decoded volumes
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
2022-04-05 19:03:02 -07:00
Konstantin Lebedev
14dd971890 hashicorp raft with state machine 2022-04-04 17:51:51 +05:00
Konstantin Lebedev
c514710b7b initial add hashicorp raft 2022-04-04 13:50:56 +05:00
chrislu
ae558fa073 log reasons volumes became unwritable 2022-03-21 00:41:44 -07:00
chrislu
57c6eddd22 avoid possible deadlock if volume layout is used in some logs 2022-03-21 00:04:01 -07:00
Konstantin Lebedev
9ea09cc41c healthz check to avoid drain pod with last replicas 2022-02-16 14:18:36 +05:00
Konstantin Lebedev
0ed76a0556 clearly 2022-02-14 14:10:06 +05:00
Konstantin Lebedev
36013f63ed https://github.com/chrislusf/seaweedfs/issues/2648 2022-02-14 13:59:12 +05:00
chrislu
433fde4b18 move error to a separate file
This file contains metric names for all errors
The naming convention is ErrorSomeThing = "error.some.thing"
2022-02-04 22:57:51 -08:00
Konstantin Lebedev
3f4e17aa24 error metrics for filer and store 2022-02-04 14:07:14 +05:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
b0665a15f4
Merge pull request #2527 from banjiaojuhao/master-assign-by-datanode 2021-12-21 08:56:51 -08:00
banjiaojuhao
dda6b90d25 assign fileId according to DataNode with empty DataCenter and Rack 2021-12-21 17:28:33 +08:00
chrislu
5eacff9d4f log message adds server name
address https://github.com/chrislusf/seaweedfs/issues/2514#issuecomment-995925733
2021-12-16 10:46:26 -08:00
Chris Lu
3be3c17f59 volume vacuum: avoid timeout with streaming progress report
fix https://github.com/chrislusf/seaweedfs/issues/2396
2021-10-24 01:55:34 -07:00
Chris Lu
e4830bd93d go fmt 2021-10-07 21:13:31 -07:00
Chris Lu
332d49432d reduce concurrent volume grow requests 2021-10-05 01:58:30 -07:00
Chris Lu
96119eab00 refactor 2021-10-05 00:40:04 -07:00
Chris Lu
8a66306064 calculate disk usage in case of race condition
related to https://github.com/chrislusf/seaweedfs/issues/2357
2021-10-04 23:32:07 -07:00
Chris Lu
a067deaabc avoid possible modified location list
fix issue 1 of https://github.com/chrislusf/seaweedfs/issues/2345
2021-09-28 16:54:18 -07:00
Chris Lu
2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
574485ec69 better IP v6 support 2021-09-07 19:29:42 -07:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
e93d4935e3 add other replica locations when assigning volumes 2021-09-05 23:32:25 -07:00
Chris Lu
7a13816e94 refactor 2021-09-05 23:17:15 -07:00
Chris Lu
65af3cf4df master: disconnect only the phantom volume server
fix https://github.com/chrislusf/seaweedfs/issues/2311
2021-09-05 15:20:03 -07:00
Chris Lu
78e8ddf910 Only when tailing volume, the zero-ed cookie should skip checking.
This only happens when checkCookie == false and fsync == false.
2021-08-13 02:09:35 -07:00
Chris Lu
d1d1fc772c move some volume lookup operations to grpc
jwt related lookup will come in next commit
2021-08-12 20:33:00 -07:00
Chris Lu
01336d71eb minor 2021-08-10 13:04:33 -07:00
Chris Lu
eed26af266 Merge branch 'master' into add_remote_storage 2021-08-08 15:48:04 -07:00
Chris Lu
4370a4db63 use int64 for volume count in case of negative overflow 2021-08-08 15:19:39 -07:00
Chris Lu
cb1dbd3135 refactor 2021-08-01 11:53:46 -07:00
Chris Lu
b624090398 go fmt 2021-07-01 01:21:14 -07:00
Chris Lu
d474ce6fe3 master: avoid repeated leader redirection
fix https://github.com/chrislusf/seaweedfs/issues/2146
2021-06-21 22:56:07 -07:00
Chris Lu
87a32bfef4 avoid possible nil when node is disconnected from its parent
fix https://github.com/chrislusf/seaweedfs/issues/2073
2021-05-19 10:02:01 -07:00
Chris Lu
d2d36a3f9d master: avoid creating too many volumes
fix https://github.com/chrislusf/seaweedfs/issues/2062
2021-05-11 10:05:31 -07:00
Chris Lu
9a6aa00e9d avoid nil locations
fix https://github.com/chrislusf/seaweedfs/issues/2059
2021-05-10 02:39:52 -07:00
qieqieplus
c4d32f6937 ahead of time volume assignment 2021-05-06 18:55:44 +08:00
Patrick Schmidt
7413d59750 Fix EC shard count logic
This fixes the calculation of the amount of EC shards a node holds.
Previously a global counter was increased, but also used inside the
loop to apply disk usage deltas. This led to wrong absolute numbers.
The fix is to apply only deltas of single EC shards per iteration.
2021-03-05 12:50:58 +01:00
Patrick Schmidt
5f7b024891 Show the real disk usage in stats calls
Currently the file size of only one volume location is taken into
account in the stats. This commit multiplies the disk usages by the
amount of nodes holding a replica of the volume.
This will yield the expected amount of disk usage and matches the
total size calculations from before.
2021-02-26 13:58:40 +01:00
Chris Lu
2270737344 volume: avoid fixed vacuum timeout for large volumes
1GB for 3 minutes, about 5.7MB/s
2021-02-22 12:52:37 -08:00
Chris Lu
565f7a6e72 Update data_node.go 2021-02-19 14:22:36 -08:00
Chris Lu
a37473ae60 add back volume ids
address https://github.com/chrislusf/seaweedfs/issues/1792#issuecomment-782339576
2021-02-19 14:22:12 -08:00
Chris Lu
c576ad04ac fix volume server display for volumes 2021-02-19 01:38:56 -08:00
Chris Lu
73958e357d add descriptive error if no free volumes 2021-02-18 19:10:20 -08:00
Chris Lu
3575d41009 go fmt 2021-02-17 20:57:08 -08:00