Commit Graph

331 Commits

Author SHA1 Message Date
Chris Lu
5c57297bd1 metadata log: read from any timestamp 2020-04-28 00:05:57 -07:00
Chris Lu
725e47dcc3 filer: pass along user name and group name for HDFS 2020-04-27 05:48:48 -07:00
Chris Lu
e24b25de78 async meta caching: can stream updates now 2020-04-21 21:16:13 -07:00
Chris Lu
cb3985be70 go fmt 2020-04-20 17:48:06 -07:00
Chris Lu
ebfab42a50 refactoring 2020-04-20 00:08:47 -07:00
Chris Lu
2955b96ef1 refactor 2020-04-19 23:54:32 -07:00
Chris Lu
ce3cb25cfb working for in memory single log buffer 2020-04-19 23:37:50 -07:00
Chris Lu
f373232227
Merge pull request #1280 from bukton123/filer_mongodb
Add new filer support mongodb
2020-04-19 20:35:19 -07:00
bukton
da37455254 filer mongo add func create index unique directory, name 2020-04-20 08:13:57 +07:00
Chris Lu
4642ed565e fix tests 2020-04-19 10:42:58 -07:00
bukton
d6d245dc89 filer mongodb add option poll size and fixed order by list meta 2020-04-19 17:51:32 +07:00
bukton
082f079ead filer mongo change func update to insert entry 2020-04-19 11:08:10 +07:00
Chris Lu
076c8bd3bc filer master start up with default ip address instead of just localhost 2020-04-18 15:17:27 -07:00
bukton
8c58b0d017 filer mongodb delete folder children 2020-04-19 00:31:49 +07:00
bukton
290c6b7f01 Merge remote-tracking branch 'origin/master' into filer_mongodb
# Conflicts:
#	go.mod
#	go.sum
#	weed/server/filer_server.go
2020-04-19 00:21:45 +07:00
bukton
6234ea441b filer mongodb delete 2020-04-19 00:20:00 +07:00
bukton
24a8192298 filer mongo list 2020-04-19 00:16:50 +07:00
bukton
3c70163798 filer mongodb insert entry and find entry 2020-04-18 23:48:38 +07:00
bukton
fd4576651b add new package mongodb and install lib mongodb 2020-04-18 22:26:57 +07:00
Chris Lu
722b14337e add cipher option to meta data updates 2020-04-17 01:26:27 -07:00
Chris Lu
7d3672c60a simplify 2020-04-17 00:00:48 -07:00
Chris Lu
f282ed444b refactoring 2020-04-13 21:58:10 -07:00
Chris Lu
7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu
d30483d642 re-enable system logs 2020-04-12 14:03:07 -07:00
Chris Lu
6f948e4887 remove configurable topics folder location 2020-04-12 13:07:59 -07:00
Chris Lu
1c2e920fa2 fix compilation error 2020-04-12 03:11:19 -07:00
Chris Lu
b5a713be68 filer store: add redis2 redis_cluster2 as default redis store 2020-04-12 02:50:41 -07:00
Chris Lu
e613695af7 a little optimization 2020-04-12 01:13:57 -07:00
Chris Lu
1c65656fb4 s3: add option to fsync buckets 2020-04-11 23:37:10 -07:00
Chris Lu
d7f3acb2c0 refactor 2020-04-11 12:45:24 -07:00
Chris Lu
ed54682598 refactoring 2020-04-11 12:43:17 -07:00
Chris Lu
e909b55633 refactoring 2020-04-11 12:39:46 -07:00
Chris Lu
b7f0ba3800 refactoring 2020-04-11 12:37:41 -07:00
Chris Lu
bcf37346ef add timestamp inside lock 2020-04-10 01:35:59 -07:00
Chris Lu
ed98223b08 still log, but not persisting the changes 2020-04-08 20:32:57 -07:00
Chris Lu
dc08e4098f add etag only for PUT or large chunked uploads 2020-04-08 09:13:26 -07:00
Chris Lu
ec2eb8bc48 add If-None-Match and If-Modified-Since
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu
90d576affe disable meta data change event logging for now. 2020-04-07 01:05:24 -07:00
Chris Lu
53626734d4 iterate through the log buffer 2020-04-05 16:51:30 -07:00
Chris Lu
f71b855db5 adjust caching and notification for recursive directory deletion 2020-04-05 15:03:25 -07:00
Chris Lu
91da7057b1 refactoring 2020-04-05 13:11:43 -07:00
Chris Lu
b10679fcf0 adjust log file names 2020-04-05 12:38:46 -07:00
Chris Lu
e78ee087e9 fix tests 2020-04-05 01:17:35 -07:00
Chris Lu
bf270d9e8c filer: able to tail meta data changes 2020-04-05 00:51:16 -07:00
Chris Lu
2fc2eb74dd refactoring 2020-03-30 22:57:45 -07:00
Chris Lu
c22094d340 adjust logging 2020-03-30 13:03:43 -07:00
Chris Lu
50a5018b7f writing meta logs is working 2020-03-30 01:19:33 -07:00
Chris Lu
9dc0b1df8f refactoring to compile for windows OS
windows os does not like to work with fuse
2020-03-29 21:07:55 -07:00
Chris Lu
826bc0b7e3 FUSE: add chunk cache for recently accessed file chunks 2020-03-28 13:43:31 -07:00
Chris Lu
4aa82c95e6 refactor 2020-03-28 13:42:35 -07:00
Chris Lu
d1439c5bd3 fix FUSE read for large files
FUSE expects ReadAt do not return partial filled buffer with a nil error.
2020-03-27 04:50:51 -07:00
Chris Lu
f06ca04451 avoid overflow 2020-03-27 04:35:31 -07:00
Chris Lu
2f5ed29c37 clean up 2020-03-27 00:30:39 -07:00
Chris Lu
dd5b582d05 go fmt 2020-03-26 00:09:01 -07:00
Chris Lu
5ebc95b69b refactoring 2020-03-25 02:20:19 -07:00
Chris Lu
c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu
ae2ee379c0 consistent 64bit size 2020-03-22 01:37:46 -07:00
Chris Lu
2bdd936fb6 purge code 2020-03-22 01:33:40 -07:00
Chris Lu
65d2ea9fb0 FUSE mount: stream read data with buffer
fix https://github.com/chrislusf/seaweedfs/issues/1244
2020-03-22 01:00:36 -07:00
Chris Lu
82bfad5b86 refactoring 2020-03-21 23:48:11 -07:00
Chris Lu
6ff9e2835e filer: skip resizing image if width or height larger than original image
fix https://github.com/chrislusf/seaweedfs/issues/1239#issuecomment-602140779
2020-03-21 22:16:00 -07:00
Chris Lu
3505b06023 report deletion error in the log 2020-03-20 23:38:24 -07:00
Chris Lu
308688c8d0 filer: add back image resizing capability 2020-03-20 20:31:11 -07:00
Chris Lu
bec6ec7db6 go fmt 2020-03-17 10:01:55 -07:00
Chris Lu
7b37178716 filer: close stores if interrupted 2020-03-14 20:30:26 -07:00
Chris Lu
e85da50ad4 remove tikv, move its support to "tikv" branch 2020-03-11 20:59:00 -07:00
Chris Lu
ebc739afb5 filer store postgres: skip empty parameters
better fix https://github.com/chrislusf/seaweedfs/issues/1227
2020-03-11 01:13:54 -07:00
Chris Lu
7f3ce7b6d6 adjust tikv build env 2020-03-10 13:36:45 -07:00
Chris Lu
d214cefc2e filer: list diretory pagination make up for the expired entries 2020-03-09 23:28:01 -07:00
Chris Lu
89eb05b50f filer: support TTL for all filer stores 2020-03-09 01:02:01 -07:00
Chris Lu
2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu
afb20de14c breaks dependency loop 2020-03-07 17:01:39 -08:00
Chris Lu
8645283a7b fuse mount: avoid lookup nil entry
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu
13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu
465d18930b filer redis: directory listing follow ttl
fix https://github.com/chrislusf/seaweedfs/issues/1217
2020-03-05 10:35:21 -08:00
Chris Lu
6a8484b4ae master able to list all master clients by type 2020-03-01 22:13:47 -08:00
Chris Lu
555413d9fc weed queue starts 2020-02-27 00:07:13 -08:00
Chris Lu
0156e2975a mount: add mode to run external to SeaweedFS container cluster 2020-02-26 16:46:01 -08:00
Chris Lu
892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
7d10fdf737 fix directory lookup nil 2020-02-25 11:13:06 -08:00
Chris Lu
6ab7368ef2 filer: dynamically create bucket under /buckets folder 2020-02-24 22:28:45 -08:00
Chris Lu
621cdbdf58 filer: avoid possible timeouts for updates and deletions 2020-02-20 15:44:17 -08:00
Chris Lu
83836f4299 filer: recursively batch delete file chunks
fix https://github.com/chrislusf/seaweedfs/issues/1197
2020-02-11 22:54:10 -08:00
Chris Lu
fb19263a71 fix build 2020-01-31 00:59:48 -08:00
Chris Lu
d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu
72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu
19a05ad174 add test cases 2020-01-26 12:47:23 -08:00
Chris Lu
835da19c09 add logging 2020-01-26 00:50:18 -08:00
Chris Lu
8a42aa8221 adjust logging 2020-01-25 00:31:53 -08:00
Chris Lu
28d1b34856 logging 2020-01-24 22:13:06 -08:00
Chris Lu
6e25acc681 add logging 2020-01-24 22:01:51 -08:00
Chris Lu
b6e6ca8595 adjust logs 2020-01-24 20:06:58 -08:00
Chris Lu
6107509c64 adjust logging 2020-01-24 17:26:18 -08:00
Chris Lu
d4e75a0d18 filer: option to create file only if it is new, O_EXCL 2020-01-22 11:42:40 -08:00
Chris Lu
1b0bfbaf59 refactoring 2020-01-19 23:59:46 -08:00
Chris Lu
39edcfde4c filer: set inode
avoid "getcwd: No such file or directory"
2020-01-19 12:06:19 -08:00
Chris Lu
a9b1ee18af default redis reads options
details in https://github.com/chrislusf/seaweedfs/pull/1165
2019-12-21 09:30:51 -08:00
Chris Lu
fcbd04c425 default to false in scaffold.go 2019-12-21 08:31:27 -08:00
Chris Lu
62b1a346ef set default 2019-12-20 20:57:38 -08:00
Dakota Sullivan
a0c9d9c0e7
Fix config property names 2019-12-20 19:23:01 -07:00