mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-01-06 02:07:53 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,473 Commits 32 Branches 296 Tags 155 MiB
03bb82043c
Commit Graph

653 Commits

Author SHA1 Message Date
Chris Lu
6e43e8ce17 add logs for remote file loading 2021-03-01 00:48:59 -08:00
Chris Lu
015d16f43f add vif file versions in case loading superblock fails 2021-03-01 00:48:30 -08:00
Chris Lu
d680676d45 skip already loaded backends 2021-03-01 00:47:03 -08:00
Chris Lu
540441fd38 go fmt 2021-02-28 20:34:14 -08:00
Chris Lu
2cca07b44c cloud tier to non-AWS s3 gateways 2021-02-28 18:59:09 -08:00
Chris Lu
30b30b8fe0 volume.tier.move: passing non-empty disk type 2021-02-22 01:59:03 -08:00
Chris Lu
03c643aa6b fix test 
fileSize need to be divided by 8
 2021-02-20 21:19:21 -08:00
Chris Lu
7635f6b9fa disk file avoid file.Stat() 2021-02-20 20:06:06 -08:00
Chris Lu
a2383b3b12 Revert "Revert "avoid file.Stat()"" 
This reverts commit 7ef4c24f18.
 2021-02-20 12:42:09 -08:00
Chris Lu
7ef4c24f18 Revert "avoid file.Stat()" 
This reverts commit 98c93ca465.
 2021-02-20 12:39:33 -08:00
Chris Lu
1ad3200094 skip seek() when index file writes 2021-02-20 12:39:25 -08:00
Chris Lu
98c93ca465 avoid file.Stat() 
avoid one Syscall, but did not help on performance though
 2021-02-20 09:44:17 -08:00
Chris Lu
a1210d1e8d Revert "volume: avoid file.stat(), file.seek() if possible during writes" 
This reverts commit c78409a598.
 2021-02-19 03:56:27 -08:00
Chris Lu
c78409a598 volume: avoid file.stat(), file.seek() if possible during writes 2021-02-18 23:22:54 -08:00
Chris Lu
73958e357d add descriptive error if no free volumes 2021-02-18 19:10:20 -08:00
bingoohuang
352ac2f271 Merge remote-tracking branch 'origin/master' 2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34 use backticks instead of double quotes to avoid escaped additionally in regex 2021-02-18 14:05:28 +08:00
Chris Lu
3575d41009 go fmt 2021-02-17 20:57:08 -08:00
Chris Lu
7403cd43c5 only use "" for hdd to avoid two values for the same thing 2021-02-16 10:50:42 -08:00
Chris Lu
ebf320ec22 avoid divided by zero 2021-02-16 05:59:24 -08:00
Chris Lu
3fe628f04e use hdd instead of empty string 2021-02-16 03:03:00 -08:00
Chris Lu
f8446b42ab this can compile now!!! 2021-02-16 02:47:02 -08:00
Chris Lu
7ce647f27e support customizable disk type 2021-02-13 15:42:42 -08:00
Chris Lu
4bd8a692d8 disk type can be generic tags 2021-02-13 13:50:14 -08:00
Chris Lu
885ca34748 volume: fail fast if idx files are missing 
fix https://github.com/chrislusf/seaweedfs/issues/1796
 2021-02-11 00:44:40 -08:00
Chris Lu
821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
bingoohuang
9e06ef66b9 fix renaming for 5bytes 2021-02-09 09:44:28 +08:00
Chris Lu
a833021132 fix refactoring left over 2021-02-07 23:03:03 -08:00
bingoohuang
7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
bingoohuang
94ea3bd3a5 renaming NeedleMapType to NeedleMapKind 2021-02-07 09:00:03 +08:00
Chris Lu
cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu
039ad101b1 manifest file also need to detect compression 
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753585046
 2021-01-03 01:42:21 -08:00
Chris Lu
4f31c1bb94 go fmt 2020-12-22 02:34:08 -08:00
Chris Lu
9abf016af1 Merge branch 'master' into support_ssd_volume 2020-12-17 22:30:19 -08:00
Chris Lu
738c4fd203 fix logging when error 
address https://github.com/chrislusf/seaweedfs/pull/1685#issuecomment-747845601
 2020-12-17 19:37:46 -08:00
Chris Lu
daa8157fc2 Merge branch 'master' into support_ssd_volume 2020-12-17 13:05:20 -08:00
Chris Lu
e2076201d7 volume: avoid reprocessing the same volume 
fix https://github.com/chrislusf/seaweedfs/issues/1682
 2020-12-17 13:03:39 -08:00
Chris Lu
986cbdf7d9 Revert "Merge pull request #1683 from qieqieplus/master" 
This reverts commit 8cb67952db, reversing
changes made to 200e56215a.
 2020-12-17 12:46:42 -08:00
Chris Lu
f56e6d231e Revert "Merge pull request #1683 from qieqieplus/master" 
This reverts commit 8cb67952db, reversing
changes made to 200e56215a.
 2020-12-17 12:46:20 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
1d88865869 passing disk type along 2020-12-16 09:10:14 -08:00
Chris Lu
23014b6810 Merge branch 'master' into support_ssd_volume 2020-12-16 08:26:51 -08:00
chenqieqie
45f902a9b7 smaller critical section 2020-12-16 20:23:51 +08:00
chenqieqie
4e58a4f24e fix race condition when loading volumes concurrently 2020-12-16 18:49:10 +08:00
Chris Lu
2e8dba571b adjust volume server UI 2020-12-14 00:51:57 -08:00
Chris Lu
933f2f4cfd find location with matching disk type 2020-12-13 23:39:00 -08:00
Chris Lu
94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu
f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
ae655033ac adjust logging 2020-12-11 16:57:53 -08:00
Chris Lu
d986c7196d use append time instead of filer's own modification time 
fix https://github.com/chrislusf/seaweedfs/issues/1669
 2020-12-11 16:55:18 -08:00
Chris Lu
d171d9f988 volume: ensure the volume dat and idx files are always closed 
fix https://github.com/chrislusf/seaweedfs/issues/1646
 2020-12-04 21:50:26 -08:00
Chris Lu
cc839f935d better unit detection 2020-12-03 19:23:59 -08:00
Chris Lu
b219ccfe68 Update volume_ttl_test.go 2020-12-03 12:55:44 -08:00
Chris Lu
a83b8e7b01 ttl fix: 3y was converted into 96m 2020-12-03 10:53:29 -08:00
Chris Lu
f3bb645018 file open error 2020-12-01 23:37:49 -08:00
Chris Lu
dc0bc48257 return file open error 2020-12-01 23:36:49 -08:00
Chris Lu
96c48bc8a8 fix test 2020-11-28 03:03:43 -08:00
Chris Lu
2c913dde04 volume: detect and drop volumes with disk IO error 
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
 2020-11-28 00:09:29 -08:00
Chris Lu
9ac4935f22 read from volume index file directly instead of open a separate file 
fix https://github.com/chrislusf/seaweedfs/issues/1640

read from volume index file directly instead of open a separate file,
to ensure reading latest index entries.
 2020-11-27 16:18:48 -08:00
Chris Lu
6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu
3c229eb677 refactor 2020-11-26 17:21:55 -08:00
Chris Lu
c0fb4a3766 add comments 2020-11-26 15:22:42 -08:00
Chris Lu
6783ae5f8f rename 2020-11-26 15:19:43 -08:00
Chris Lu
97f706f35b just refactoring 2020-11-26 15:08:05 -08:00
Chris Lu
b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu
0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc remove zstd 
fix https://github.com/chrislusf/seaweedfs/issues/1629
 2020-11-21 13:06:45 -08:00
Chris Lu
1dd3a6ac36 avoid verbose logs 2020-11-09 15:56:11 -08:00
Chris Lu
0dafcf1f5a volume: detect max volume count changes based on disk usage 
fix https://github.com/chrislusf/seaweedfs/issues/1594
 2020-11-03 14:43:17 -08:00
Chris Lu
de86945aeb go fmt 2020-10-31 16:45:38 -07:00
Chris Lu
9708df47d1 ensure offset in the index file is positive 2020-10-31 15:50:01 -07:00
Konstantin Lebedev
46303c36bf When the volume server is stopped, the master server immediately sees the deletion of volumes 2020-10-31 02:48:25 +05:00
Chris Lu
6560ac6466 volume loading: trim out unreachable idx file content 
fix https://github.com/chrislusf/seaweedfs/issues/1583
 2020-10-29 22:25:23 -07:00
Konstantin Lebedev
1ad1b8c4f6 collection Volume ReadOnly Count with detailed status 2020-10-28 17:38:26 +05:00
Chris Lu
744c5594da volume: automatically trim out unreachable entries 2020-10-28 01:14:39 -07:00
Chris Lu
3e925faddd trim out name extension 2020-10-28 01:14:05 -07:00
shibinbin
a67eb1afa1 fix: restart volumeserver sometime occurred out of memory error 2020-10-28 12:03:06 +08:00
Chris Lu
53c3aad875 volume: add a note file to avoid incomplete volume files 
fix https://github.com/chrislusf/seaweedfs/issues/1567
 2020-10-27 15:56:49 -07:00
Chris Lu
06c15ab35c volume: add special handling for .dat larger than 32GB 2020-10-27 13:11:56 -07:00
Konstantin Lebedev
2fb1fce8a0 avoid old values when missing read only flags 2020-10-25 16:21:40 +05:00
Chris Lu
9104cfa744 reduce locks 2020-10-24 19:40:35 -07:00
Chris Lu
e9d40b80b2 less verbose logs 2020-10-24 09:42:54 -07:00
Chris Lu
575d7952a1 add available resource stats 
fix https://github.com/chrislusf/seaweedfs/issues/1555
 2020-10-22 09:13:47 -07:00
Chris Lu
4ff2ceee33 UI fix on rendering EC volumes 
addressing UI problem with https://github.com/chrislusf/seaweedfs/issues/1551
 2020-10-21 22:05:58 -07:00
Konstantin Lebedev
dc2e13092d add number of read only volumes metric 2020-10-15 15:48:40 +05:00
Chris Lu
bd8b9b0c2e some changes to prepare fix after failed reads 2020-10-08 18:33:06 -07:00
Chris Lu
d522df0ecd adjust log level 2020-10-04 01:35:32 -07:00
Chris Lu
a1c01d716b volume: avoid deadlock when deleting volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1501
 2020-10-01 07:10:03 -07:00
Chris Lu
103fafe00b weed export: print out [start,stop) content range 2020-09-14 22:57:23 -07:00
Chris Lu
f2723c1bc8 do not idx file format 
revert c9ab8d05fa
 2020-09-12 12:42:36 -07:00
Chris Lu
446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu
ea26a98753 volume: validate volume correctness if last entry is a deletion 2020-09-12 04:07:04 -07:00
Chris Lu
9d4bdfcfdf fix volume integrity checking 2020-09-11 11:34:10 -07:00
Chris Lu
401ccf1509 fix test 2020-09-10 15:13:23 -07:00
Chris Lu
c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
James Hartig
8e54e34576 volume: Don't unmount before deleting volume in copy 
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
 2020-09-01 22:00:07 -04:00
Chris Lu
38e06d783d volume: check disk space before compaction 
fix https://github.com/chrislusf/seaweedfs/issues/1440
 2020-08-31 18:10:53 -07:00