mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-28 05:15:00 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,625 Commits 32 Branches 296 Tags 148 MiB
8db58450a0
Commit Graph

193 Commits

Author SHA1 Message Date
Patrick Schmidt
7413d59750 Fix EC shard count logic 
This fixes the calculation of the amount of EC shards a node holds.
Previously a global counter was increased, but also used inside the
loop to apply disk usage deltas. This led to wrong absolute numbers.
The fix is to apply only deltas of single EC shards per iteration.
 2021-03-05 12:50:58 +01:00
Patrick Schmidt
5f7b024891 Show the real disk usage in stats calls 
Currently the file size of only one volume location is taken into
account in the stats. This commit multiplies the disk usages by the
amount of nodes holding a replica of the volume.
This will yield the expected amount of disk usage and matches the
total size calculations from before.
 2021-02-26 13:58:40 +01:00
Chris Lu
2270737344 volume: avoid fixed vacuum timeout for large volumes 
1GB for 3 minutes, about 5.7MB/s
 2021-02-22 12:52:37 -08:00
Chris Lu
565f7a6e72 Update data_node.go 2021-02-19 14:22:36 -08:00
Chris Lu
a37473ae60 add back volume ids 
address https://github.com/chrislusf/seaweedfs/issues/1792#issuecomment-782339576
 2021-02-19 14:22:12 -08:00
Chris Lu
c576ad04ac fix volume server display for volumes 2021-02-19 01:38:56 -08:00
Chris Lu
73958e357d add descriptive error if no free volumes 2021-02-18 19:10:20 -08:00
Chris Lu
3575d41009 go fmt 2021-02-17 20:57:08 -08:00
Chris Lu
6daa932f5c refactoring to get master function, instead of passing master values directly 
this will enable retrying later
 2021-02-17 20:55:55 -08:00
Chris Lu
68775d29e3 fix tests 2021-02-16 10:51:03 -08:00
Chris Lu
b314d78e97 fix print 2021-02-16 10:48:28 -08:00
Chris Lu
53ca7e66ef avoid dead lock 2021-02-16 10:48:16 -08:00
Chris Lu
3097b9a9b7 fix existence checking 2021-02-16 05:59:43 -08:00
Chris Lu
cb9cc29518 volume.list display; fix updating maxVolumeCount for disk 2021-02-16 03:55:24 -08:00
Chris Lu
3fe628f04e use hdd instead of empty string 2021-02-16 03:03:00 -08:00
Chris Lu
f8446b42ab this can compile now!!! 2021-02-16 02:47:02 -08:00
Chris Lu
4bd8a692d8 disk type can be generic tags 2021-02-13 13:50:14 -08:00
Chris Lu
821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu
1102ae32c4 fix concurrent map reads 2021-01-31 18:26:26 -08:00
Chris Lu
9c9ba3c209 nil related 
related to https://github.com/chrislusf/seaweedfs/issues/1676
 2021-01-03 12:25:58 -08:00
Chris Lu
d9e8479c06 adjust UI max count 2020-12-17 13:47:51 -08:00
Chris Lu
3cdf5945a2 adjust UI 2020-12-17 13:37:00 -08:00
Chris Lu
f696a2b2a7 assign volumes based on disk type 2020-12-17 13:25:05 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu
a9db24cd05 master allocate volumes if ssd type runs out 2020-12-13 19:44:57 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
715b199eeb fix tests 2020-12-13 04:14:50 -08:00
Chris Lu
d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
003b6245e7 fix nil 2020-12-02 00:09:19 -08:00
Chris Lu
965413c21b shell: add volume.vacuum command 2020-11-28 23:18:02 -08:00
Chris Lu
c7ebadc25d avoid possible concurrent access inside ensureCorrectWritables() 2020-11-22 17:15:59 -08:00
Chris Lu
8cb8cd4cc5 add locks 2020-11-17 16:59:48 -08:00
Chris Lu
c6bd244ebd add TODO 2020-11-11 12:51:27 -08:00
Chris Lu
e0002f8dd7 check existing volumes for writable status 2020-10-24 01:34:31 -07:00
Chris Lu
720b1d9b88 adding locking to avoid nil VolumeLocationList 
fix panic: runtime error: invalid memory address or nil pointer dereference
Oct 22 00:53:44 bedb-master1 weed[8055]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x17658da]
Oct 22 00:53:44 bedb-master1 weed[8055]: goroutine 310 [running]:
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLocationList).Length(...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_location_list.go:35
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLayout).enoughCopies(...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_layout.go:376
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*VolumeLayout).ensureCorrectWritables(0xc000111d50, 0xc000b55438)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/volume_layout.go:202 +0x5a
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/topology.(*Topology).SyncDataNodeRegistration(0xc00042ac60, 0xc001454d30, 0x1, 0x1, 0xc0005fc000, 0xc00135de40, 0x4, 0xc00135de50, 0x10, 0x10d, ...)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/topology/topology.go:224 +0x616
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).SendHeartbeat(0xc000162700, 0x23b97c0, 0xc000ae2c90, 0x0, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/server/master_grpc_server.go:106 +0x325
Oct 22 00:53:44 bedb-master1 weed[8055]: github.com/chrislusf/seaweedfs/weed/pb/master_pb._Seaweed_SendHeartbeat_Handler(0x1f8e7c0, 0xc000162700, 0x23b0a60, 0xc00024b440, 0x3172c38, 0xc000ab7100)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/seaweedfs/weed/pb/master_pb/master.pb.go:4250 +0xad
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).processStreamingRPC(0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100, 0xc0001fea80, 0x311fec0, 0x0, 0x0, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1329 +0xcd8
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).handleStream(0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100, 0x0)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1409 +0xc5c
Oct 22 00:53:44 bedb-master1 weed[8055]: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0001ce8b0, 0xc0001f31e0, 0x23bb800, 0xc000ac5500, 0xc000ab7100)
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:746 +0xa5
Oct 22 00:53:44 bedb-master1 weed[8055]: created by google.golang.org/grpc.(*Server).serveStreams.func1
Oct 22 00:53:44 bedb-master1 weed[8055]: #011/root/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:744 +0xa5
Oct 22 00:53:44 bedb-master1 systemd[1]: weedmaster.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 22 00:53:44 bedb-master1 systemd[1]: weedmaster.service: Failed with result 'exit-code'.
 2020-10-21 23:15:48 -07:00
Chris Lu
410b818aa7 master: avoid timer leakage 2020-10-19 14:24:57 -07:00
Chris Lu
c7d7b1a0f6
Merge pull request #1485 from LIBA-S/fix_oversized 
Correct the oversized state of volume after compaction
 2020-09-23 19:24:30 -07:00
LIBA-S
eecd6b5d35 Fix a race condition when handle VolumeLocationList 2020-09-23 20:56:51 +08:00
LIBA-S
0157798ebf Correct the oversized state of volume after compaction 2020-09-23 20:27:42 +08:00
Chris Lu
289e62a305 master: better locking of in memory volume data 
related to https://github.com/chrislusf/seaweedfs/issues/1436#issuecomment-695880135
 2020-09-20 23:07:55 -07:00
Chris Lu
6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu
152a6cbc2b minor adjustments 2020-08-10 20:42:27 -07:00
cheng.li01
25fbff5d52 fix bug: two same volumeId in different collections 
1, there will be two leader when master server startup in a few seconds
2, raft server will get a leader even there is only one master, so there is no need to do hard code to set the server to be leader
 2020-08-10 16:37:47 +08:00
cheng.li01
dad1161c70 fix dn.volumes Iterate when write issue 2020-07-08 19:57:19 +08:00
Chris Lu
e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
Evgenii Kozlov
0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
bingoohuang
1f8782a1ed try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
bingoohuang
1a642b9876 add Volume Ids column only for max 100 volumes for convenience in the master ui. 2020-05-29 15:37:58 +08:00