mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-29 05:49:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,508 Commits 32 Branches 296 Tags 148 MiB
015dd3a147
Commit Graph

2054 Commits

Author SHA1 Message Date
bukton
fd4576651b add new package mongodb and install lib mongodb 2020-04-18 22:26:57 +07:00
Chris Lu
11f5a6d913 messaging can compile now 2020-04-18 01:12:01 -07:00
Chris Lu
5bea77010f refactor 2020-04-17 22:39:21 -07:00
Chris Lu
2e2537a9ea rename files 2020-04-17 22:35:44 -07:00
Chris Lu
e0f5996560 fix "call of Unmarshal passes non-pointer as second argument" 2020-04-17 10:04:41 -07:00
Chris Lu
826f96b6f3 fix unreachable code 2020-04-17 10:04:28 -07:00
Chris Lu
ffa69b24ae fix travis 2020-04-17 10:01:11 -07:00
Chris Lu
3f3dba5a68 broker: append message logs 2020-04-17 02:29:38 -07:00
Chris Lu
2a45897237 broker: read cipher value from filer 2020-04-17 02:29:00 -07:00
Chris Lu
bda82f61bc filer: able to append to a file 2020-04-17 02:28:09 -07:00
Chris Lu
722b14337e add cipher option to meta data updates 2020-04-17 01:26:27 -07:00
Chris Lu
7d3672c60a simplify 2020-04-17 00:00:48 -07:00
Chris Lu
508f3490a0 update messaging proto 2020-04-16 03:29:57 -07:00
Chris Lu
f5a748d33c refactoring 2020-04-16 02:55:09 -07:00
Chris Lu
ce4b369be2 scaffolding messaging 2020-04-16 02:21:23 -07:00
Reed
aada5b9aa7 making volume stats size info readable. 2020-04-16 10:08:08 +08:00
Chris Lu
9cacaf5eca better detect file mime type 2020-04-14 11:32:31 -07:00
Chris Lu
3f1d79512f only detect when not sure about the types 2020-04-14 11:02:05 -07:00
Chris Lu
d2d1597850 volume: detect gzip effectiveness only when content type is empty 
revert part of 2286eda575
 2020-04-14 10:52:53 -07:00
Chris Lu
23715af402
Merge pull request #1273 from wyxloading/config_race 
Fix data race about config
 2020-04-13 22:24:51 -07:00
Chris Lu
2b5c4fbbf3 tiered caching 
1/4 for small less than 1MB files. 1/4 for 1~4MB files, 1/2 for bigger than 4MB files
 2020-04-13 22:19:27 -07:00
Chris Lu
f282ed444b refactoring 2020-04-13 21:58:10 -07:00
wuyuxiang
b556d3d035 Fix data race about config 2020-04-14 10:22:40 +08:00
Chris Lu
d8f5985e5e add logs 2020-04-13 12:58:45 -07:00
Chris Lu
5c9e91393f 1.74 2020-04-13 10:25:48 -07:00
Chris Lu
7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu
8e23dc078b refactoring 2020-04-12 20:48:21 -07:00
Chris Lu
b062393f3f refactoring 2020-04-12 17:51:31 -07:00
Chris Lu
5361f999ed shell: only one shell is allowed to connect to the cluster 
fix https://github.com/chrislusf/seaweedfs/issues/1266
 2020-04-12 17:48:39 -07:00
Chris Lu
eb16bb0287 go mod cleanup 2020-04-12 16:44:58 -07:00
Chris Lu
d30483d642 re-enable system logs 2020-04-12 14:03:07 -07:00
Chris Lu
6f948e4887 remove configurable topics folder location 2020-04-12 13:07:59 -07:00
Chris Lu
94e35cdb35 mount: fix 
fix bug found by git bisect, but I do not understand why it can cause error!
 2020-04-12 03:34:36 -07:00
Chris Lu
1c2e920fa2 fix compilation error 2020-04-12 03:11:19 -07:00
Chris Lu
b5a713be68 filer store: add redis2 redis_cluster2 as default redis store 2020-04-12 02:50:41 -07:00
Chris Lu
e613695af7 a little optimization 2020-04-12 01:13:57 -07:00
Chris Lu
2a1f396df5 avoid duplicated setting chunks into cache 2020-04-12 01:06:50 -07:00
Chris Lu
b9b7da905e handle nil chunk cache 2020-04-12 01:00:12 -07:00
Chris Lu
211d87cf4c mount: option to disable caching 2020-04-12 00:52:54 -07:00
Chris Lu
1c65656fb4 s3: add option to fsync buckets 2020-04-11 23:37:10 -07:00
Chris Lu
e4af63a721 volume server: accept fsync=true in write requests 2020-04-11 21:39:16 -07:00
Chris Lu
5987810e5e set default chunk size to 16 2020-04-11 21:22:52 -07:00
Chris Lu
df97da25f9 mount: add on disk caching 2020-04-11 21:12:41 -07:00
Chris Lu
c8ca234773 refactoring 2020-04-11 14:27:25 -07:00
Chris Lu
df9d538044 rename function 2020-04-11 14:19:44 -07:00
Chris Lu
d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu
ed54682598 refactoring 2020-04-11 12:43:17 -07:00
Chris Lu
e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu
b7f0ba3800 refactoring 2020-04-11 12:37:41 -07:00
Chris Lu
417125457e add additional buffers for logs 2020-04-11 02:19:48 -07:00
Chris Lu
4b27afe176 fix sqs message reading 2020-04-10 15:27:47 -07:00
Chris Lu
dce2702e1b fix the instruction 2020-04-10 14:50:10 -07:00
Chris Lu
13ab1fdaf0 mount: use file default permission 0666 2020-04-10 02:23:23 -07:00
Chris Lu
1101a42e5c mv : create new folder, move children, and delete old folder 2020-04-10 01:37:03 -07:00
Chris Lu
bcf37346ef add timestamp inside lock 2020-04-10 01:35:59 -07:00
Chris Lu
9fa065f600 typo in logs 2020-04-09 23:43:09 -07:00
Chris Lu
8db2120bee add logs 2020-04-09 23:42:59 -07:00
Chris Lu
43c62a8a95 1.73 2020-04-09 21:44:30 -07:00
Chris Lu
eb39df2704 avoid dead loop 2020-04-09 21:43:26 -07:00
Chris Lu
56ec8c4513 fix to avoid nil file an dirty pages 2020-04-09 21:43:05 -07:00
Chris Lu
59f40e2027 volume: best effort to detect ip address 
fix https://github.com/chrislusf/seaweedfs/issues/1264
 2020-04-09 00:26:24 -07:00
Chris Lu
f6a7e79dc3 weed shell: simplify CLI option for filer 2020-04-08 23:57:15 -07:00
Chris Lu
2329d9e0c1 add volume.fix.replication to default scaffold 2020-04-08 23:36:22 -07:00
Chris Lu
bb78ab9915 wait for master to be aware of the failed volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1268
 2020-04-08 23:12:37 -07:00
Chris Lu
8764bdb9df 1.72 2020-04-08 22:35:14 -07:00
Chris Lu
8a73276602 refactor a bit 2020-04-08 22:33:36 -07:00
Chris Lu
006f78b70b make a copy of request data 
to avoid concurrency issues
 2020-04-08 22:31:19 -07:00
Chris Lu
00b993a234 add util for md5 2020-04-08 22:30:38 -07:00
Chris Lu
ed98223b08 still log, but not persisting the changes 2020-04-08 20:32:57 -07:00
Chris Lu
6630541399 ensure correct file size when opening existing files 2020-04-08 12:50:59 -07:00
Chris Lu
b524a40375 add locking to fs cache 2020-04-08 12:50:34 -07:00
Chris Lu
bd56172b82 simplify file handle management 2020-04-08 12:50:20 -07:00
Chris Lu
dc08e4098f add etag only for PUT or large chunked uploads 2020-04-08 09:13:26 -07:00
Chris Lu
ec2eb8bc48 add If-None-Match and If-Modified-Since 
fix https://github.com/chrislusf/seaweedfs/issues/1269
 2020-04-08 08:12:00 -07:00
Chris Lu
4c498e73de filer replication: add s3 endpoint 2020-04-07 17:49:00 -07:00
Chris Lu
745f5d2a25 fix setting default value 2020-04-07 01:58:48 -07:00
Chris Lu
54debdc6f7 filer: fix configuration settings 2020-04-07 01:30:53 -07:00
Chris Lu
90d576affe disable meta data change event logging for now. 2020-04-07 01:05:24 -07:00
Chris Lu
b51d8f03bb 1.71 2020-04-05 22:04:16 -07:00
Chris Lu
5a5908407d filer: support larger file size 
fix https://github.com/chrislusf/seaweedfs/issues/1257
 2020-04-05 21:48:45 -07:00
Chris Lu
19edd9c091 comment out debug messages 2020-04-05 20:15:17 -07:00
Chris Lu
78ea937606 fix 2020-04-05 17:34:26 -07:00
Chris Lu
53626734d4 iterate through the log buffer 2020-04-05 16:51:30 -07:00
Chris Lu
f71b855db5 adjust caching and notification for recursive directory deletion 2020-04-05 15:03:25 -07:00
Chris Lu
f14b6a09a8 remove duplicated update events 2020-04-05 13:22:15 -07:00
Chris Lu
91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu
2d43f85577 watch entries with common path prefix 2020-04-05 12:51:21 -07:00
Chris Lu
7bc3c93512 add util.PathJoin 2020-04-05 12:40:46 -07:00
Chris Lu
af1f64d244 change from 'weed tail' to 'weed watch' 2020-04-05 12:39:20 -07:00
Chris Lu
b10679fcf0 adjust log file names 2020-04-05 12:38:46 -07:00
Chris Lu
2cb1700ecb set to 4MB meta log buffer size 2020-04-05 12:38:29 -07:00
Chris Lu
e78ee087e9 fix tests 2020-04-05 01:17:35 -07:00
Chris Lu
dcc6b6a3ed go fmt 2020-04-05 00:52:31 -07:00
Chris Lu
bf270d9e8c filer: able to tail meta data changes 2020-04-05 00:51:16 -07:00
Chris Lu
f7b5a1d697 add disk stat to logs 2020-04-04 13:17:23 -07:00
Chris Lu
6e776aaafc loop through all masters 2020-04-03 00:47:48 -07:00
Chris Lu
4a5e381feb fail fast if filer configured wrong masters 
fix https://github.com/chrislusf/seaweedfs/issues/1257
 2020-04-03 00:47:33 -07:00
Chris Lu
aaf5e658cc adjust parameter name 2020-04-03 00:41:05 -07:00
Chris Lu
6c96e51414 re-ordering 2020-04-03 00:40:54 -07:00
Chris Lu
d2f8e50459 make error a little bit more clearer 
fix https://github.com/chrislusf/seaweedfs/issues/1259
 2020-04-03 00:24:09 -07:00
Chris Lu
5f881d2fd5 mount: fail early rather than after mounted the directory 
fix https://github.com/chrislusf/seaweedfs/issues/1258
 2020-04-03 00:21:03 -07:00
Chris Lu
fee3319cbf handle gzip for image resizing 2020-04-02 23:44:24 -07:00
Chris Lu
73bc286377 volume: passing ext info via URL 2020-04-02 21:23:15 -07:00
Chris Lu
cea52a4faf volume copying adds cleaning up on error 
fix https://github.com/chrislusf/seaweedfs/issues/1253
 2020-04-02 15:36:55 -07:00
Chris Lu
c981975627 volume.fix.replication: add test cases for complicated moving 
fix https://github.com/chrislusf/seaweedfs/issues/1253
 2020-04-02 02:16:16 -07:00
Chris Lu
ae3d08883f volume.fsck: fix for erasure encoded volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1254
 2020-04-01 22:10:13 -07:00
Chris Lu
c446438ca5
Merge pull request #1255 from levenlabs/ignore 
Added treat_replication_as_minimums master toml option
 2020-04-01 16:48:58 -07:00
James Hartig
eae3f27c80 Added treat_replication_as_minimums master toml option 2020-04-01 19:08:48 -04:00
Chris Lu
2fc2eb74dd refactoring 2020-03-30 22:57:45 -07:00
Chris Lu
c22094d340 adjust logging 2020-03-30 13:03:43 -07:00
Chris Lu
50a5018b7f writing meta logs is working 2020-03-30 01:19:33 -07:00
Chris Lu
9dc0b1df8f refactoring to compile for windows OS 
windows os does not like to work with fuse
 2020-03-29 21:07:55 -07:00
Chris Lu
be6c2f8511 1.70 2020-03-29 13:40:55 -07:00
Chris Lu
54768d0761 fix: delete a file and then create a directory with the same name 2020-03-29 01:39:48 -07:00
Chris Lu
057722bbf4 return part of the chunk if chunkview is not the full chunk 2020-03-29 00:54:39 -07:00
Chris Lu
af6d9976ad adjust print out 2020-03-28 14:07:25 -07:00
Chris Lu
a75d50bbb8 FUSE: add configurable in memory chunk cache size 2020-03-28 14:07:16 -07:00
Chris Lu
826bc0b7e3 FUSE: add chunk cache for recently accessed file chunks 2020-03-28 13:43:31 -07:00
Chris Lu
4aa82c95e6 refactor 2020-03-28 13:42:35 -07:00
Chris Lu
eedd33dda3 refactoring 2020-03-28 13:41:58 -07:00
Chris Lu
005b4ab3fe 1.69 2020-03-27 04:51:31 -07:00
Chris Lu
d1439c5bd3 fix FUSE read for large files 
FUSE expects ReadAt do not return partial filled buffer with a nil error.
 2020-03-27 04:50:51 -07:00
Chris Lu
f06ca04451 avoid overflow 2020-03-27 04:35:31 -07:00
Chris Lu
e1911760a7 refactoring 2020-03-27 00:30:55 -07:00
Chris Lu
2f5ed29c37 clean up 2020-03-27 00:30:39 -07:00
Chris Lu
2286eda575 sampling whether the data can be gzipped 2020-03-26 23:50:48 -07:00
Chris Lu
5463560912 small test 2020-03-26 23:50:07 -07:00
Chris Lu
34a0d424f8 FUSE mount: report max volume count available disk space 2020-03-26 23:06:21 -07:00
Chris Lu
a5ce359999 refactoring 2020-03-26 22:46:49 -07:00
Chris Lu
0a05052b5c 1.68 2020-03-26 22:22:36 -07:00
Chris Lu
e7e8691720 add some tests 2020-03-26 22:21:52 -07:00
Chris Lu
86398d1efe S3 API: set deletion to recursive 
fix https://github.com/chrislusf/seaweedfs/issues/1251
 2020-03-26 11:05:54 -07:00
Chris Lu
1fef598347 set target Dir object when moving 2020-03-26 10:56:40 -07:00
Chris Lu
9e2e07b238 tweak 2020-03-26 10:56:18 -07:00
Chris Lu
cbce793753 rename: set new name for renamed files 2020-03-26 03:30:23 -07:00
Chris Lu
c763a492f5 refactor 2020-03-26 03:30:02 -07:00
Chris Lu
712720864d error checking for remove 2020-03-26 02:21:30 -07:00
Chris Lu
8d9a76005d add back running output 2020-03-26 02:14:48 -07:00
Chris Lu
dd5b582d05 go fmt 2020-03-26 00:09:01 -07:00
Chris Lu
7f0d87b206 tree structured fs cache 
FsCache for FsNode, wrapping fs.Node
 2020-03-26 00:08:14 -07:00
Chris Lu
2e4fadd10a fix fscache move 2020-03-25 22:19:19 -07:00
Chris Lu
966df2ae27 purging skip EC volumes 2020-03-25 02:41:22 -07:00
Chris Lu
6d87a0c5f5 batch delete orphan data by file keys 2020-03-25 02:21:15 -07:00
Chris Lu
5ebc95b69b refactoring 2020-03-25 02:20:19 -07:00
Chris Lu
d79f0a76b1 format output 2020-03-25 00:56:47 -07:00
Chris Lu
d9b56ed88b shell: volume.fsck add options 2020-03-24 22:38:36 -07:00
Chris Lu
e63a79ade8 better handle lock in case of exception 2020-03-24 18:41:25 -07:00
Chris Lu
4d5554b16f fix compilation on travis 2020-03-24 18:40:52 -07:00
Chris Lu
d83e648c0b auto add prefix "fs." 2020-03-24 02:40:51 -07:00
Chris Lu
a875f67a08 fix pwd printout 2020-03-24 02:40:29 -07:00