mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-01 07:49:02 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,615 Commits 32 Branches 296 Tags 148 MiB
10414fd81c
Commit Graph

5646 Commits

Author SHA1 Message Date
chrislu
689b4ecdcc remove unused collection and replication from upload result 2022-08-20 18:14:57 -07:00
chrislu
e3f40d538d cleaner code 2022-08-20 17:51:30 -07:00
chrislu
9f50b310a0 remove unused code 2022-08-20 17:51:12 -07:00
chrislu
11f99836c3 filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0) 
fix https://github.com/seaweedfs/seaweedfs/issues/3468
 2022-08-19 23:00:56 -07:00
chrislu
fdd8c5d5e0 Filer if-modified-since check doesn't fire on "exact" #3467 
fix https://github.com/seaweedfs/seaweedfs/issues/3467
 2022-08-19 22:21:51 -07:00
chrislu
aac45f3e89 filer: retryable when error is not found 2022-08-19 03:36:15 -07:00
chrislu
58dd880322 minor 2022-08-18 23:51:08 -07:00
Eric Yang
b6f99c4d83
fix naming typo (#3465) 
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-08-18 23:25:32 -07:00
chrislu
5702727db5 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-18 02:39:27 -07:00
chrislu
4d72a1ce6e s3: remove deleted collection during metrics reporting 
fix https://github.com/seaweedfs/seaweedfs/issues/3453
 2022-08-18 02:39:07 -07:00
Guo Lei
3172c33fa6
fix redundant type from array (#3462) 2022-08-18 02:25:25 -07:00
chrislu
42c6e52513 s3: fix regression on HEAD directory operation 2022-08-18 02:13:58 -07:00
chrislu
4573c99ae3 fix tests 2022-08-18 00:15:46 -07:00
chrislu
2b580a7566 also migrate jsonpb 2022-08-17 12:42:03 -07:00
chrislu
eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu
e259052a70 volume: fix metrics for volume request time 2022-08-17 04:29:31 -07:00
qzh
7fcfaf7bc9
fix(filer.sync): offset may be set to 0 (#3451) 
* fix(filer.sync): initializing the offset is related to the path

* fix(filer.sync): the offset maybe to be set to 0.

Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
 2022-08-15 23:43:52 -07:00
qzh
400f0c3e5d
fix(filer.sync): initializing the offset is related to the path (#3450) 
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
 2022-08-15 21:56:47 -07:00
chrislu
fa4d0093e1 3.22 2022-08-15 16:48:23 -07:00
chrislu
93261f5199 Revert "Refactor for Sync method (#3426)" 
This reverts commit 670cb759f8.

with the pr
weed/storage () - (master) > go test -count=1 ./...
ok  	github.com/seaweedfs/seaweedfs/weed/storage	18.486s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map	0.025s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding	0.864s
?   	github.com/seaweedfs/seaweedfs/weed/storage/idx	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle	0.110s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle_map	24.414s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/super_block	0.203s
?   	github.com/seaweedfs/seaweedfs/weed/storage/types	[no test files]
?   	github.com/seaweedfs/seaweedfs/weed/storage/volume_info	[no test files]
weed/storage () - (master) >
weed/storage () - (master) >

without the pr
weed/storage () - (master) >
weed/storage () - (master) > go test -count=1 ./...
ok  	github.com/seaweedfs/seaweedfs/weed/storage	1.617s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map	0.026s
?   	github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding	0.906s
?   	github.com/seaweedfs/seaweedfs/weed/storage/idx	[no test files]
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle	0.202s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/needle_map	24.533s
ok  	github.com/seaweedfs/seaweedfs/weed/storage/super_block	0.280s
?   	github.com/seaweedfs/seaweedfs/weed/storage/types	[no test files]
?   	github.com/seaweedfs/seaweedfs/weed/storage/volume_info	[no test files]
 2022-08-15 15:35:31 -07:00
chrislu
9c588d4010 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-15 10:03:56 -07:00
chrislu
96caf21d09 less verbose log 2022-08-15 10:03:52 -07:00
Andrey Triumfov
31faa6d43d
Remove duplicate slashes in object path to prevent 500 errors (#3442) 2022-08-15 08:19:28 -07:00
chrislu
cb476a53ff remove logs 2022-08-15 01:05:35 -07:00
chrislu
7c029b2183 3.21 2022-08-15 00:32:15 -07:00
Chris Lu
e88392b50f
Fix s3 pagination (#3436) 
* Revert previous changes

* s3: use cursor to track tree traversal

fix https://github.com/seaweedfs/seaweedfs/issues/3166

* special cases for empty prefix and empty directory

* use constants

* address empty folder

* undo local changes

* fix IsTruncated

* adjust counting directories

* fix cases when prefix is a directory

* s3: handle directory object

works for

aws --endpoint-url http://127.0.0.1:8333/ s3api list-objects-v2 --bucket test --prefix "fakedir"
 2022-08-15 00:30:19 -07:00
chrislu
7457c746f0 s3: fix aws s3api head-object 2022-08-14 23:52:35 -07:00
chrislu
789d9dbfe1 webdav: fix nil 
fix https://github.com/seaweedfs/seaweedfs/issues/3440
 2022-08-14 23:33:30 -07:00
Konstantin Lebedev
6199d9b923
rand start garbageThreshold and reset metric ReplicaPlacementMismatch (#3396) 
* rand start garbageThreshold
reset metrics MasterReplicaPlacementMismatch

* vacuum through sleep
 2022-08-12 13:11:09 -07:00
Konstantin Lebedev
4d4cd0948d
avoid infinite loop WaitUntilConnected() (#3431) 
https://github.com/seaweedfs/seaweedfs/issues/3421
 2022-08-11 15:03:26 -07:00
Eric Yang
986daecbfe
add upload response (#3428) 
* add upload response

* fixing comments

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
 2022-08-10 22:43:09 -07:00
chrislu
ae93c966d9 ensure memory is aligned 
fix https://github.com/seaweedfs/seaweedfs/issues/3427
 2022-08-10 22:27:13 -07:00
Rain Li
670cb759f8
Refactor for Sync method (#3426) 2022-08-10 08:21:57 -07:00
chrislu
662ec97602 mount: adjust on du stats reporting 
fix https://github.com/seaweedfs/seaweedfs/issues/3424
 2022-08-09 20:21:01 -07:00
Abirdcfly
b0633716b7
delete minor unreachable code (#3423) 
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
 2022-08-09 01:24:43 -07:00
chrislu
0854171d22 3.20 2022-08-07 14:42:28 -07:00
chrislu
d8e69df295 adjust comments 2022-08-07 11:35:33 -07:00
chrislu
0aeec04c31 quicker to adapt to pattern change 2022-08-07 10:14:01 -07:00
chrislu
928d29af9e fix wrong logic about reader isRandomMode() 2022-08-07 10:13:04 -07:00
chrislu
d3d52b1818 shell: fs.meta.load add quieter mode 
fix https://github.com/seaweedfs/seaweedfs/issues/3414
 2022-08-07 01:42:15 -07:00
chrislu
358ccb5e0e Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-07 01:34:36 -07:00
chrislu
67814a5c79 refactor and fix strings.Split 2022-08-07 01:34:32 -07:00
John W Higgins
3afda0c89c
Allow postgresql to use standard environment variables for connection (#3413) 2022-08-07 00:58:53 -07:00
chrislu
1a4bf0dcb5 filer.sync: parallelize the filer.sync 2022-08-07 00:56:15 -07:00
chrislu
02b38f894b debug 2022-08-07 00:55:49 -07:00
chrislu
0e9478488d filer.sync: fix when excluded paths is empty 2022-08-07 00:55:34 -07:00
chrislu
8a880a139d mount: fix truncate operation 
fix https://github.com/seaweedfs/seaweedfs/issues/2609
 2022-08-06 13:58:45 -07:00
LHHDZ
84ec68e11a
Add download speed limit support (#3408) 2022-08-05 01:16:42 -07:00
chrislu
b278bb24d3 mount: adjust df stats reporting when close to the limit 
fix https://github.com/seaweedfs/seaweedfs/issues/3407
 2022-08-05 00:31:41 -07:00
Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center (#3405) 
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404

* GetDataCenter

* prefer same data center for ReplicationSource

* GetDataCenterId

* remove glog
 2022-08-04 17:35:00 -07:00
Konstantin Lebedev
22181dd018
refactor FilerRequest metrics (#3402) 
* refactor FilerRequest metrics

* avoid double count proxy

* defer to
 2022-08-04 01:44:54 -07:00
chrislu
bd13a7968f remove unused symlink resolving 2022-08-04 01:35:18 -07:00
chrislu
88945d9954 mount: do not follow soft link with xattr 2022-08-04 01:21:15 -07:00
chrislu
90db4b5a44 minor 2022-08-03 00:10:47 -07:00
Chris Lu
fc8035d672
Merge pull request #3394 from kmlebedev/metricsReplicatedWrite 
Detailed metrics VolumeServerRequestHistogram for writing to disk and replication
 2022-08-01 13:31:01 -07:00
Konstantin Lebedev
3ffa4ba91d clean 2022-08-02 01:30:38 +05:00
Konstantin Lebedev
2a5629dd28 align 2022-08-02 01:29:44 +05:00
Konstantin Lebedev
df9cc31a0a refactor 2022-08-02 01:18:26 +05:00
Konstantin Lebedev
0a5d3c3cf6 rename to localWrite and replicatedWrite 2022-08-01 23:09:09 +05:00
chrislu
bd5c9904be Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-01 09:40:05 -07:00
chrislu
e26d1cf4ef fix test 2022-08-01 09:40:01 -07:00
Konstantin Lebedev
fc65122766 rename to LoadAvg_1M 2022-08-01 21:32:21 +05:00
Konstantin Lebedev
ab50f21ad9 Detailed metrics VolumeServerRequestHistogram for writing to disk and replication 2022-08-01 21:16:57 +05:00
Konstantin Lebedev
2f85e31f5c Merge branch 'master' into gentle_vacuum 2022-08-01 20:51:58 +05:00
Konstantin Lebedev
5209ebbeef remove percent 2022-08-01 20:40:38 +05:00
Chris Lu
b59bc607bf
Merge pull request #3338 from kmlebedev/issues/3083 
rollback over onPeerUpdate implementation of automatic clean-up of failed servers in favor of synchronous ping
 2022-08-01 08:23:10 -07:00
Konstantin Lebedev
78cbd8002f revert Sleep 2022-08-01 20:21:23 +05:00
Konstantin Lebedev
cd5c7ad052 move to github.com/seaweedfs/seaweedfs 2022-08-01 16:36:32 +05:00
Konstantin Lebedev
634bc97840 fix conflicts 2022-08-01 14:47:00 +05:00
Konstantin Lebedev
3c75479e2b Merge branch 'master' into gentle_vacuum 
# Conflicts:
#	weed/pb/messaging_pb/messaging.pb.go
#	weed/pb/messaging_pb/messaging_grpc.pb.go
#	weed/pb/s3_pb/s3.pb.go
#	weed/pb/volume_server_pb/volume_server.pb.go
#	weed/server/volume_grpc_vacuum.go
 2022-08-01 14:45:22 +05:00
Konstantin Lebedev
c0d92f61a1 comment 2022-08-01 14:40:42 +05:00
Konstantin Lebedev
1d29f67c02 revert disk stats 2022-08-01 14:29:41 +05:00
Konstantin Lebedev
a98f6d66a3 rollback over onPeerupdate implementation of automatic clean-up of failed servers in favor of synchronous ping 2022-08-01 12:51:41 +05:00
chrislu
036566629a filer.sync: fix synchronization logic in active-active mode 
fix https://github.com/seaweedfs/seaweedfs/issues/3328
 2022-08-01 00:06:18 -07:00
chrislu
303bd067b5 Revert "rename: delete source entry metadata only, skipping hard links" 
This reverts commit 03466f955e.

fix https://github.com/seaweedfs/seaweedfs/issues/3386
 2022-07-31 22:51:41 -07:00
chrislu
03c6f978f4 3.19 2022-07-31 17:16:51 -07:00
chrislu
afef014832 scaffold 2022-07-31 13:23:44 -07:00
Chris Lu
0bb3ba17ae
Merge pull request #3385 from Woellchen/feature/cluster-status-return-healthy 
Add healthy indicator for raft status
 2022-07-30 11:55:30 -07:00
Patrick Schmidt
1a4a36d510
Add healthy indicator for raft status 2022-07-30 19:34:26 +02:00
chrislu
cb096d1f53 minor 2022-07-30 09:52:52 -07:00
chrislu
0423d50106 wip note 2022-07-30 09:36:27 -07:00
chrislu
0f50fcb96a moving 2022-07-29 23:39:11 -07:00
chrislu
13b9a52f80 fix deprecated functions 2022-07-29 01:34:39 -07:00
chrislu
be0ec7bc2f fix test 2022-07-29 01:15:26 -07:00
chrislu
1f161f53dc add back rack 2022-07-29 01:15:19 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev
2f0dda384d vacuum show LA 2022-07-29 11:59:33 +05:00
chrislu
de3afd4199 fix rebasing 2022-07-28 23:25:46 -07:00
chrislu
06cd491abc add message batch 2022-07-28 23:24:38 -07:00
chrislu
7576c244c4 add flatbuffer serde for message 2022-07-28 23:24:38 -07:00
chrislu
a3fae7cd16 add an empty placeholder for mq.topic.list 2022-07-28 23:24:38 -07:00
chrislu
aa4a22ad47 segment serde 2022-07-28 23:24:38 -07:00
chrislu
74f60f246f dynamically connect to a filer 2022-07-28 23:24:38 -07:00
chrislu
9f479aab98 allocate brokers to serve segments 2022-07-28 23:24:38 -07:00
chrislu
bb01b68fa0 refactor 2022-07-28 23:24:38 -07:00
chrislu
94b8c42b2c clean up 2022-07-28 23:24:38 -07:00
chrislu
8060fdcac5 remove old code 2022-07-28 23:24:38 -07:00
chrislu
f25e273e32 display data center and rack in cluster.ps 2022-07-28 23:22:52 -07:00
chrislu
68065128b8 add dc and rack 2022-07-28 23:22:51 -07:00
chrislu
8d31e73ffd mq broker adds data center and rack 2022-07-28 23:22:07 -07:00