Commit Graph

431 Commits

Author SHA1 Message Date
Konstantin Lebedev
9d6ffa0ea1 GetBucketLifecycleConfigurationHandler 2021-10-14 01:35:33 +05:00
Konstantin Lebedev
be4b3ed509 AclHandlers 2021-10-11 15:03:56 +05:00
Chris Lu
f4676824a7 fix test code compilation 2021-10-10 22:14:13 -07:00
Chris Lu
8205166009 disable testing with redis server
--- FAIL: TestNameList (0.00s)
panic: exec: "redis-server": executable file not found in $PATH [recovered]
	panic: exec: "redis-server": executable file not found in $PATH

goroutine 37 [running]:
testing.tRunner.func1.2({0xde2f80, 0xc0003da160})
	/opt/hostedtoolcache/go/1.17.1/x64/src/testing/testing.go:1209 +0x24e
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.17.1/x64/src/testing/testing.go:1212 +0x218
panic({0xde2f80, 0xc0003da160})
	/opt/hostedtoolcache/go/1.17.1/x64/src/runtime/panic.go:1038 +0x215
github.com/chrislusf/seaweedfs/weed/filer/redis3.TestNameList(0x407c59)
	/home/runner/work/seaweedfs/seaweedfs/weed/filer/redis3/kv_directory_children_test.go:45 +0x376
testing.tRunner(0xc0003e2680, 0xf57f48)
	/opt/hostedtoolcache/go/1.17.1/x64/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.17.1/x64/src/testing/testing.go:1306 +0x35a
2021-10-10 21:57:55 -07:00
Chris Lu
3e2acf677c removing tikv to resolve "go mod tidy" problem
tikv is causing "go mod tidy" problem. Need to resolve this before adding tikv back.

go mod tidy
go: finding module for package github.com/coreos/etcd/clientv3/balancer/picker
go: finding module for package cloud.google.com/go/kms/apiv1
go: finding module for package github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
go: finding module for package google.golang.org/grpc/naming
go: finding module for package github.com/coreos/etcd/clientv3/credentials
go: finding module for package github.com/coreos/etcd/clientv3/balancer
go: finding module for package github.com/d4l3k/messagediff
go: found github.com/coreos/etcd/clientv3/balancer in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/picker in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/resolver/endpoint in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/credentials in github.com/coreos/etcd v3.3.26+incompatible
go: found cloud.google.com/go/kms/apiv1 in cloud.google.com/go/kms v1.0.0
go: found github.com/d4l3k/messagediff in github.com/d4l3k/messagediff v1.2.1
go: finding module for package google.golang.org/grpc/naming
github.com/chrislusf/seaweedfs/weed/filer/tikv imports
	github.com/tikv/client-go/v2/tikv imports
	go.etcd.io/etcd/clientv3 tested by
	go.etcd.io/etcd/clientv3.test imports
	github.com/coreos/etcd/integration imports
	github.com/coreos/etcd/proxy/grpcproxy imports
	google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
2021-10-10 19:27:02 -07:00
Chris Lu
17e08a16f4 fix redis3 deletion 2021-10-09 04:54:48 -07:00
Chris Lu
e4830bd93d go fmt 2021-10-07 21:13:31 -07:00
Chris Lu
0a856241fe avoid int bigger than math.MaxInt32
fix https://github.com/chrislusf/seaweedfs/issues/2363
2021-10-07 21:12:57 -07:00
Chris Lu
f3d8232e14 reduce one redis lookup on hot path 2021-10-06 22:01:19 -07:00
Chris Lu
2336a397dc use pipeline to save some time 2021-10-06 20:40:18 -07:00
Chris Lu
371fead8a5 redis3 using redis native sorted set 2021-10-06 18:18:24 -07:00
Chris Lu
8668d49c9d test with real redis
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkNameList-12    	    1789	    760599 ns/op
BenchmarkRedis-12       	   17539	     64122 ns/op
PASS
2021-10-06 01:25:37 -07:00
Chris Lu
6b31f3c97a add benchmark test 2021-10-06 00:37:57 -07:00
Chris Lu
893f0587b1 redis3 adds distributed locking 2021-10-06 00:03:54 -07:00
Chris Lu
947add39e6 clean up *SkipListElementReference loaded from Redis 2021-10-04 02:31:38 -07:00
Chris Lu
2b9aab3442 use 1000 per batch 2021-10-04 01:03:40 -07:00
Chris Lu
366f522a2d add redis3 2021-10-04 01:01:31 -07:00
Chris Lu
b6694279d7 Merge branch 'master' into bptree 2021-10-01 16:55:44 -07:00
chrislusf
3ffbaaa071 use github.com/linxGnu/grocksdb 2021-10-01 03:16:03 +00:00
Chris Lu
040443e2d1 fix possible error case 2021-09-27 23:59:45 -07:00
Chris Lu
9887610b54 log tsNs should be processing time 2021-09-26 11:54:13 -07:00
Chris Lu
2baed2e1e9 avoid possible metadata subscription data loss
Previous implementation append filer logs into one file. So one file is not always sorted, which can lead to miss reading some entries, especially when different filers have different write throughput.
2021-09-25 01:18:44 -07:00
Chris Lu
198fa58e3c Merge branch 'master' into bptree 2021-09-18 14:06:16 -07:00
Chris Lu
49d971e602 filer: redis store needs to clean up batch deleted sub folders 2021-09-18 14:05:16 -07:00
Chris Lu
8f2e4be074 wip 2021-09-18 14:04:30 -07:00
Chris Lu
96514f0f00 fix tests 2021-09-13 02:19:48 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
joshuafc
5654d0d60d CompactionTableSizeMultiplier of leveldb use default value. #2325
To improve performance of leveldb find key in condition of large directory(millions of files) which use uuid as filename.
2021-09-09 10:42:34 +08:00
Chris Lu
574485ec69 better IP v6 support 2021-09-07 19:29:42 -07:00
zhoub
6a7ed1bd0e add bloom filter to leveldb_store to improve fuse performance. 2021-09-07 21:09:10 +08:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
28f45f8fa6 re-org 2021-09-05 14:38:31 -07:00
Chris Lu
df29281536 rename 2021-09-04 05:11:06 -07:00
Chris Lu
c3db389e42 refactor 2021-09-04 05:09:38 -07:00
Chris Lu
e9ce099453 rename 2021-09-04 05:03:51 -07:00
Chris Lu
8707ef00a6 refactor 2021-09-04 05:02:20 -07:00
Chris Lu
8ec357b3d3 go mod 2021-09-03 23:25:33 -07:00
Chris Lu
7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
yulai.li
b17b81529e Add build tags for TiKV filerstore 2021-08-30 15:59:25 +08:00
Rain Li
e5f9ff983d
Merge branch 'master' into filerstore-tikv 2021-08-30 15:52:54 +08:00
Chris Lu
dc481c081c remove unused function 2021-08-27 22:18:54 -07:00
Chris Lu
9242f3aaeb fix build 2021-08-26 17:33:57 -07:00
Chris Lu
05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
yulai.li
c1dc5ab4ac Add deleterange_concurrency to filer configuration file 2021-08-26 18:25:08 +08:00
yulai.li
318757ef8c Change DeleteFolderChildren to DeleteRange api 2021-08-26 17:49:56 +08:00
yulai.li
546efeba8f Fix build bug 2021-08-26 16:20:35 +08:00
yulai.li
2088f28424 init post 2021-08-26 15:20:18 +08:00
Chris Lu
9bcf94b2b1 ensure multi-threaded correctness 2021-08-25 17:28:50 -07:00
Chris Lu
40dc283b2d fix locating data chunks 2021-08-15 23:07:58 -07:00
Chris Lu
72eb6d5b9d ensure no writes to remote storage if content is not changed 2021-08-15 20:23:41 -07:00