mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-29 05:49:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,529 Commits 32 Branches 296 Tags 148 MiB
97c963bac9
Commit Graph

4288 Commits

Author SHA1 Message Date
Chris Lu
8668d49c9d test with real redis 
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkNameList-12    	    1789	    760599 ns/op
BenchmarkRedis-12       	   17539	     64122 ns/op
PASS
 2021-10-06 01:25:37 -07:00
Chris Lu
6b31f3c97a add benchmark test 2021-10-06 00:37:57 -07:00
Chris Lu
893f0587b1 redis3 adds distributed locking 2021-10-06 00:03:54 -07:00
Chris Lu
f0d1e7bd05 skip ec volumes when loading normal volumes 2021-10-05 02:31:44 -07:00
Chris Lu
332d49432d reduce concurrent volume grow requests 2021-10-05 01:58:30 -07:00
Chris Lu
96119eab00 refactor 2021-10-05 00:40:04 -07:00
Chris Lu
8a66306064 calculate disk usage in case of race condition 
related to https://github.com/chrislusf/seaweedfs/issues/2357
 2021-10-04 23:32:07 -07:00
Chris Lu
4ed2994555 use tsMemory to determine whether read from disk or memory 
remove lastFlushTime
 2021-10-04 16:02:56 -07:00
Chris Lu
6a030547a2 server: remove peer check if not starting master 
more fix https://github.com/chrislusf/seaweedfs/issues/2352
 2021-10-04 03:27:10 -07:00
Chris Lu
62c2732fd1 add back logic to check master peers 
fix https://github.com/chrislusf/seaweedfs/issues/2352
 2021-10-04 02:51:26 -07:00
Chris Lu
d4bb16e20e
Merge pull request #2354 from chrislusf/bptree 
Add Redis3
 2021-10-04 02:43:15 -07:00
Chris Lu
947add39e6 clean up *SkipListElementReference loaded from Redis 2021-10-04 02:31:38 -07:00
Chris Lu
513fed323a SkipListElementReference can be an empty object 2021-10-04 02:30:44 -07:00
Chris Lu
280ab7f95c add test 2021-10-04 02:30:24 -07:00
Chris Lu
04662126bb add redis3 2021-10-04 01:04:27 -07:00
Chris Lu
2b9aab3442 use 1000 per batch 2021-10-04 01:03:40 -07:00
Chris Lu
366f522a2d add redis3 2021-10-04 01:01:31 -07:00
Chris Lu
ba7fbac07f rename 2021-10-03 19:23:34 -07:00
Chris Lu
e6196cdc50 add name list 2021-10-03 17:54:25 -07:00
Chris Lu
a481c4a45e return previous element if visited 2021-10-03 13:50:52 -07:00
Chris Lu
22d8684e88 refactor out listStore 2021-10-03 02:19:21 -07:00
Chris Lu
d343b0db57 update value 2021-10-03 01:15:14 -07:00
Chris Lu
4f50f8c2ca insert key and value 2021-10-03 01:07:35 -07:00
Chris Lu
69b84bb771 TestFindGreaterOrEqual 2021-10-02 14:15:49 -07:00
Chris Lu
57e2fd3f9b remove bptree 2021-10-02 14:03:54 -07:00
Chris Lu
4c1741fdbb working skiplist 2021-10-02 14:02:56 -07:00
Chris Lu
b297849147 typo 2021-10-01 23:24:54 -07:00
Chris Lu
af207bbaf0 retry both assign volume and uploading data 
fix https://github.com/chrislusf/seaweedfs/issues/2351
 2021-10-01 23:23:39 -07:00
Chris Lu
b6694279d7 Merge branch 'master' into bptree 2021-10-01 16:55:44 -07:00
Chris Lu
1e3fdf366f go fmt 2021-10-01 12:10:24 -07:00
Chris Lu
e862b2529a refactor 2021-10-01 12:10:11 -07:00
Chris Lu
88e006bc6d
Merge pull request #2349 from kmlebedev/fix_repl_volumes_per_step 
Topology update for every Nth id volume
 2021-10-01 12:05:51 -07:00
Konstantin Lebedev
5e64b22b45 check that the topology has been updated 2021-10-01 18:51:22 +05:00
Chris Lu
f58ea6a2ee add source name to error message 2021-10-01 02:19:30 -07:00
chrislusf
3ffbaaa071 use github.com/linxGnu/grocksdb 2021-10-01 03:16:03 +00:00
Konstantin Lebedev
2cecde89c3 rename opt volumesPerStep 2021-10-01 00:17:54 +05:00
Konstantin Lebedev
fc51ffce2b https://github.com/chrislusf/seaweedfs/issues/1846 2021-09-30 20:24:24 +05:00
Chris Lu
a067deaabc avoid possible modified location list 
fix issue 1 of https://github.com/chrislusf/seaweedfs/issues/2345
 2021-09-28 16:54:18 -07:00
Chris Lu
040443e2d1 fix possible error case 2021-09-27 23:59:45 -07:00
Chris Lu
2e9372dcf7 volume stream read skips deleted content 2021-09-27 03:07:44 -07:00
Chris Lu
225b019fe0 stream read multiple volumes in a volume server 2021-09-27 02:51:31 -07:00
Chris Lu
1904448d4e adjust starting offset 2021-09-27 02:01:29 -07:00
Chris Lu
5956a8b05a adjust comment 2021-09-27 01:58:03 -07:00
Chris Lu
aa64f2ac4c send needle data instead of raw needle body 2021-09-27 01:53:41 -07:00
Chris Lu
c4d7ee6c5c volume server: read all files in a volume 2021-09-27 01:45:32 -07:00
Chris Lu
4a1d4d7462 s3: default to allow empty folder for better performance 2021-09-26 22:34:14 -07:00
Chris Lu
cee4d20bc1 2.70 2021-09-26 17:37:46 -07:00
Chris Lu
603ea2db73 avoid looping forever if there are no more metadata updates 2021-09-26 11:55:27 -07:00
Chris Lu
9887610b54 log tsNs should be processing time 2021-09-26 11:54:13 -07:00
Chris Lu
2baed2e1e9 avoid possible metadata subscription data loss 
Previous implementation append filer logs into one file. So one file is not always sorted, which can lead to miss reading some entries, especially when different filers have different write throughput.
 2021-09-25 01:18:44 -07:00
Chris Lu
a814f3f0a8 adjust metadata tail output 2021-09-25 01:04:51 -07:00
Chris Lu
52fe86df45 use default 10000 for grpc port 2021-09-20 14:05:59 -07:00
Chris Lu
b3d88180ca Merge branch 'master' into bptree 2021-09-19 23:56:59 -07:00
Chris Lu
fa7c65bd4b 2.69 2021-09-19 21:44:06 -07:00
Chris Lu
84fdda85e7 go fmt 2021-09-19 12:06:15 -07:00
Chris Lu
ad5099e570 refactor 2021-09-19 12:02:23 -07:00
Chris Lu
5abdc0be77 s3: avoid overwriting object with ACL/LegalHold/Retension/LockConfiguration requests 2021-09-19 03:24:47 -07:00
Chris Lu
ede7a65a50 update log 2021-09-19 01:33:50 -07:00
Chris Lu
c1255d30ad add cache error log 2021-09-19 01:33:26 -07:00
Chris Lu
59dd271734 more glog 2021-09-19 00:29:51 -07:00
Chris Lu
4cbba2b1c6 add more glog to s3 2021-09-19 00:28:22 -07:00
Chris Lu
71175461ef add glog for s3 handlers 2021-09-19 00:18:59 -07:00
Chris Lu
e066e2642c add NodeStore 2021-09-18 15:32:17 -07:00
Chris Lu
198fa58e3c Merge branch 'master' into bptree 2021-09-18 14:06:16 -07:00
Chris Lu
49d971e602 filer: redis store needs to clean up batch deleted sub folders 2021-09-18 14:05:16 -07:00
Chris Lu
8f2e4be074 wip 2021-09-18 14:04:30 -07:00
Chris Lu
b751debd31 split node based on the last inserted key 2021-09-18 01:29:47 -07:00
Chris Lu
2226c3c8b6 Merge branch 'master' into bptree 2021-09-17 10:35:21 -07:00
Chris Lu
94a01fcfcb filer.remote.gateway: add options to include or exclude new bucket names to mirror 2021-09-15 23:04:16 -07:00
Chris Lu
b5f4910412 adjust help messages 2021-09-15 22:53:10 -07:00
Chris Lu
f0907eb83c add command filre.remote.gateway 2021-09-15 22:48:04 -07:00
Chris Lu
63da4bbb54 separate filer.remote.gateway command to avoid confusion 2021-09-15 22:47:17 -07:00
Chris Lu
2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu
119d5908dd shell: do not need to lock to see volume -h 2021-09-13 22:13:34 -07:00
Chris Lu
7504be58f9 Avoid xattr printed out as HTTP headers 
fix https://github.com/chrislusf/seaweedfs/issues/2336
 2021-09-13 04:00:57 -07:00
Chris Lu
96514f0f00 fix tests 2021-09-13 02:19:48 -07:00
Chris Lu
20ac710ceb 2.68 2021-09-13 02:16:09 -07:00
Chris Lu
6cd1ce8b74 erasure coding: add cleanup step if anything goes wrong 2021-09-13 01:55:49 -07:00
Chris Lu
f74b29416a better etag matching 2021-09-13 00:31:46 -07:00
Chris Lu
e9760f261e minor 2021-09-13 00:31:06 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
2c9d4c8f43 custom grpc port: filer 2021-09-12 02:28:37 -07:00
Chris Lu
e690a2be16 custom grpc port: volume server 2021-09-12 02:25:15 -07:00
Chris Lu
232ad2fe65 custom grpc port: master follower 2021-09-12 02:25:00 -07:00
Chris Lu
bd1f800efe custom grpc port: master 2021-09-12 02:19:10 -07:00
Chris Lu
7591336a22 log format 2021-09-11 14:27:57 -07:00
Chris Lu
8c6ff55226 add volume not found error type, to reduce error log 2021-09-11 14:26:41 -07:00
Chris Lu
7e600bff5f adjust log level 2021-09-11 12:59:15 -07:00
Chris Lu
a243d7e047 EC: clean up if failed in the middle 2021-09-11 02:07:14 -07:00
Chris Lu
f7f01ac209 return if error 2021-09-11 02:06:43 -07:00
Chris Lu
59cb6146a3 refactor 2021-09-11 02:06:30 -07:00
Chris Lu
5496d68f6a increase counter only if not early terminated 2021-09-11 02:05:55 -07:00
Chris Lu
2b8ab1534a mysql table DDL: minor, not necessary, but a little more strict 
the directory is already hashed into dirhash. So this change is not strictly required.
 2021-09-08 23:55:18 -07:00
joshuafc
5654d0d60d CompactionTableSizeMultiplier of leveldb use default value. #2325 
To improve performance of leveldb find key in condition of large directory(millions of files) which use uuid as filename.
 2021-09-09 10:42:34 +08:00
Chris Lu
0b4269b6a8 remember commands even if failed 2021-09-08 15:55:19 -07:00
Chris Lu
0207f5fe9b replicated remote.cache 2021-09-08 15:54:55 -07:00
Chris Lu
00edcf71fc fix compilation 2021-09-08 07:27:03 -07:00
Chris Lu
574485ec69 better IP v6 support 2021-09-07 19:29:42 -07:00
Chris Lu
0128239c0f handle ipv6 addresses 2021-09-07 16:43:54 -07:00
zhoub
6a7ed1bd0e add bloom filter to leveldb_store to improve fuse performance. 2021-09-07 21:09:10 +08:00