mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-15 19:26:42 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,473 Commits 32 Branches 296 Tags 150 MiB
03bb82043c
Commit Graph

4240 Commits

Author SHA1 Message Date
Chris Lu
01336d71eb minor 2021-08-10 13:04:33 -07:00
Chris Lu
1154e23e2d add logs for volume creation 2021-08-10 13:04:25 -07:00
Chris Lu
057ef429ac format 2021-08-10 12:33:29 -07:00
Chris Lu
b63b042afc dedup keeps the largest replica 2021-08-10 12:30:41 -07:00
Chris Lu
0526db12e2 do not treat read only volumes differently 2021-08-10 11:37:12 -07:00
Chris Lu
e50a5b8e28 minor: print disk type 2021-08-10 11:10:09 -07:00
Chris Lu
db6275a0c8 print out balance ratio 2021-08-10 10:43:42 -07:00
Chris Lu
9d85569c55 ensure using local quorum consistency 2021-08-10 05:10:57 -07:00
Chris Lu
85832d02c0 wait for goroutines 2021-08-10 04:13:12 -07:00
Bl1tz23
1c94b3d013 merge master, resolve conflicts 2021-08-10 13:45:24 +03:00
Bl1tz23
e6e57db530 Add liveness\readiness probe for s3 api handler on /status path 2021-08-10 13:42:46 +03:00
Chris Lu
8ff6c9a0c6 output format 2021-08-10 03:25:18 -07:00
Chris Lu
48f448ee09 parallelize tier move 2021-08-10 03:08:29 -07:00
Chris Lu
69a6da7969 avoid fail on tail error 2021-08-10 02:50:28 -07:00
Chris Lu
18228f3044 fix help message 2021-08-10 02:48:41 -07:00
Chris Lu
69655ba8e5 mount: cache on reading remote storage 2021-08-09 22:11:57 -07:00
Chris Lu
a7012d9729 fix 2021-08-09 16:03:03 -07:00
Chris Lu
02f728cb54 fix bug with remote.uncache 2021-08-09 15:24:21 -07:00
Chris Lu
8d3e275735 remote: filer cache remote content on read 2021-08-09 15:16:45 -07:00
Chris Lu
9096f6f4f7 cache: set upper limit of chunk size 2021-08-09 15:08:53 -07:00
Chris Lu
402315f117 go fmt 2021-08-09 14:37:34 -07:00
Chris Lu
a6be2520c9 fix 2021-08-09 14:37:25 -07:00
Chris Lu
713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu
8cfd487608 2.62 2021-08-08 23:33:12 -07:00
Chris Lu
734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu
96ce85f5ae rename 2021-08-08 22:33:31 -07:00
Chris Lu
c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu
df85f7a1eb adjust help message 2021-08-08 22:30:12 -07:00
Chris Lu
c0b12da4ef shell: add filer.remote.unmount 2021-08-08 22:26:37 -07:00
Chris Lu
882a93dacd fix tests 2021-08-08 17:56:26 -07:00
Chris Lu
7412ccdf88 write back remote entry to local entry after uploading to remote 2021-08-08 17:55:03 -07:00
Chris Lu
dcf614a8c3 skip if the remote entry update is because of internal managerial operations 2021-08-08 15:58:10 -07:00
Chris Lu
eed26af266 Merge branch 'master' into add_remote_storage 2021-08-08 15:48:04 -07:00
Chris Lu
4370a4db63 use int64 for volume count in case of negative overflow 2021-08-08 15:19:39 -07:00
Chris Lu
26c222f596 shell: volume.tier.move avoid moving all volumes to one destination 2021-08-08 15:12:39 -07:00
Chris Lu
13e45e1605 filer.remote.sync can work now 2021-08-08 01:21:42 -07:00
Chris Lu
8f5170c138 remove imports 2021-08-07 16:20:17 -07:00
Chris Lu
46b0cb8c86 Merge branch 'master' into add_remote_storage 2021-08-07 15:41:27 -07:00
Chris Lu
de730b079d ChunkStreamReader implenents io.ReaderAt 2021-08-07 15:41:07 -07:00
Chris Lu
59732a0529 refactoring 2021-08-07 15:35:27 -07:00
Chris Lu
ecb234f75a refactor 2021-08-07 14:46:23 -07:00
Chris Lu
270770d7d7 refactor 2021-08-07 14:18:53 -07:00
Chris Lu
679f800caa Merge branch 'master' into add_remote_storage 2021-08-06 20:27:12 -07:00
Chris Lu
0c0f77e2ae skip not found error on deletion 2021-08-06 19:35:47 -07:00
Chris Lu
8828f485c0 print volume deletion error 2021-08-06 19:30:22 -07:00
Chris Lu
d2b23f3d38
Merge pull request #2238 from combineads/change_default_mode 
Change default permissions
 2021-08-05 21:28:06 -07:00
Chris Lu
e23ca3845b
Merge pull request #2237 from combineads/add_mtime 
Add crtime and mtime
 2021-08-05 21:27:13 -07:00
Chris Lu
286e5dd375 Merge branch 'master' into add_remote_storage 2021-08-05 21:07:04 -07:00
Chris Lu
1e22166939 adjust error message 2021-08-05 21:06:55 -07:00
byunghwa.yun
f3dc909b21 Change default permissions 2021-08-06 12:24:35 +09:00
byunghwa.yun
e84fad9acf Add crtime and mtime 2021-08-06 12:13:37 +09:00
Chris Lu
1a5d29520c add tests 2021-08-05 14:49:24 -07:00
Chris Lu
6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu
b9ecf1e3a8 refacotring 2021-08-04 14:56:13 -07:00
Chris Lu
d84c311699 refactoring 2021-08-04 12:30:18 -07:00
Chris Lu
f6a9ad8001 fix tests 2021-08-04 00:31:06 -07:00
Chris Lu
42969c9c62 Merge branch 'master' into add_remote_storage 2021-08-03 21:20:05 -07:00
Chris Lu
8cf0c515bf shell: volume.fix.replication retries even when there is no error #2235 
fix https://github.com/chrislusf/seaweedfs/issues/2235
 2021-08-03 09:32:55 -07:00
Chris Lu
d2ddf1dbdb typo 2021-08-03 09:32:31 -07:00
Chris Lu
3afbf04007 add TLS grpc support for filer meta clients 2021-08-03 01:30:35 -07:00
Chris Lu
fe60c6ef9a minor changes 2021-08-02 11:49:40 -07:00
byunghwa.yun
bdc7730fdb Add autocomplete 2021-08-03 02:56:52 +09:00
Chris Lu
2ae9705442 adjust text 2021-08-01 22:55:19 -07:00
Chris Lu
3739717092 Revert "adds a test" 
This reverts commit f690643b47.
 2021-08-01 22:54:45 -07:00
Chris Lu
f690643b47 adds a test 2021-08-01 22:53:50 -07:00
Chris Lu
6de786185d volume.balance: balance read only volumes first 2021-08-01 22:32:50 -07:00
Chris Lu
3bb640b786 add -force option following #2228 2021-08-01 20:03:05 -07:00
byunghwa.yun
9e839cb5cc Add force option in volume.deleteEmpty command 2021-08-02 11:37:20 +09:00
Chris Lu
56ee1d5ef1 2.61 2021-08-01 15:50:19 -07:00
Chris Lu
81b255df8b Merge branch 'master' into add_remote_storage 2021-08-01 15:44:21 -07:00
Chris Lu
58bc3ecf47 add default quietFor value 2021-08-01 15:36:06 -07:00
Chris Lu
2ca1839d77 shell: add volume.deleteEmpty command 2021-08-01 15:33:45 -07:00
Chris Lu
89933c46d2 s3: skip hidden directories in /buckets folder 2021-08-01 12:28:08 -07:00
Chris Lu
1c7e404abe remove buckets folder option 
the related code still works for old deployments
 2021-08-01 12:23:16 -07:00
Chris Lu
cb1dbd3135 refactor 2021-08-01 11:53:46 -07:00
Chris Lu
1161c70dab Merge branch 'master' into add_remote_storage 2021-08-01 01:10:13 -07:00
Chris Lu
9cc84a910f volume: deletion can skip volume size checking 
fix https://github.com/chrislusf/seaweedfs/issues/2225
 2021-08-01 00:32:51 -07:00
Chris Lu
767edd3c08 rename 2021-07-31 23:52:09 -07:00
Chris Lu
9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu
f5a69a0e44 Merge branch 'master' into add_remote_storage 2021-07-31 09:39:48 -07:00
Chris Lu
6ba65c3382 customizable debug port 2021-07-31 09:18:41 -07:00
byunghwa.yun
5d4438a72f Fix typo 2021-07-31 22:23:52 +09:00
Chris Lu
1ff8285d82 debug from any server 2021-07-31 02:13:21 -07:00
Chris Lu
0d6f45cb46 add debug option to generate full goroutine stack dump 2021-07-31 02:00:01 -07:00
Chris Lu
a3290faf17 shell command to calculate size / number of volumes in a collection 
fix https://github.com/chrislusf/seaweedfs/issues/2224
 2021-07-30 15:49:46 -07:00
Chris Lu
c6f992b2a3 remove dead code 2021-07-30 15:18:01 -07:00
Chris Lu
899963ac20 remote storage location changed to struct 2021-07-29 02:08:55 -07:00
Chris Lu
c090d6bb25 add ReadRemote(), add read remote setup when filer starts 2021-07-28 22:43:12 -07:00
Chris Lu
4deac06da0 log fix 2021-07-28 22:25:04 -07:00
divanikus
5c6270a93a shell: ability to use wildcards for collections, all collections if ommited 2021-07-27 23:53:01 +03:00
Chris Lu
035b0bae29 refactor 2021-07-27 03:32:24 -07:00
Chris Lu
1752eeb538 remote.mount saves the mapping 2021-07-27 03:26:35 -07:00
Chris Lu
4b94b03d90 directory to remote storage mapping 2021-07-27 01:16:28 -07:00
Chris Lu
99b599aa8a remote.mount 2021-07-26 22:53:44 -07:00
Chris Lu
5dede5d38d 2.60 2021-07-25 22:09:09 -07:00
Chris Lu
35f70c51b0 refactor 2021-07-25 03:37:37 -07:00
Chris Lu
72eec84167 shell: add fs.mkdir 2021-07-25 02:37:39 -07:00
byunghwa.yun
2b28a818f1 Fix mysql sql for batch delete 2021-07-25 11:06:14 +09:00
Chris Lu
10464f47d0
Merge pull request #2217 from combineads/add_webp_format 2021-07-24 09:42:41 -07:00
byunghwa.yun
7374b5b473 Add webp extension for resizing 2021-07-24 14:32:31 +09:00
byunghwa.yun
2595f269d1 Add the webp image type 2021-07-24 14:26:40 +09:00
Chris Lu
ac28611817 snowflake sequencer need an unique id 
fix https://github.com/chrislusf/seaweedfs/issues/2213
 2021-07-23 20:54:03 -07:00
Chris Lu
5c14da0f1e filer.copy: fail early if assign request failed 
fix https://github.com/chrislusf/seaweedfs/issues/2216
 2021-07-23 20:01:43 -07:00
Chris Lu
4cc2165061 weed -h subcommand list is not sorted, makes discovery difficult 
fix https://github.com/chrislusf/seaweedfs/issues/2214
 2021-07-23 18:44:53 -07:00
Chris Lu
10fc478557 scaffold -config= should output to stdout 
fix https://github.com/chrislusf/seaweedfs/issues/2212
 2021-07-23 18:41:25 -07:00
Chris Lu
092932af8f shell: faster bucket delete 
avoid waiting for large object metadata
 2021-07-23 11:25:46 -07:00
Chris Lu
bdb632fa62
Merge pull request #2211 from qieqieplus/filer-bloom-filter 
add bloom filter for filer leveldb/rocksdb
 2021-07-22 23:35:54 -07:00
qieqieplus
2b46df13f8 add bloom filter for filer leveldb/rocksdb 2021-07-23 14:05:59 -06:00
Chris Lu
60f5c0a2f5 fix security alert on github.com/dgrijalva/jwt-go 
resolve https://github.com/chrislusf/seaweedfs/security/dependabot/go.mod/github.com%2Fdgrijalva%2Fjwt-go/open
 2021-07-22 10:22:09 -07:00
Chris Lu
182288f860 filer: fix mysql, postgres batch delete error 2021-07-22 08:23:20 -07:00
Chris Lu
84d91f143f mount: hide /etc folder also from mount 2021-07-22 00:40:16 -07:00
Chris Lu
7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu
cc32436d63
Merge pull request #2209 from combineads/fuse_option 
Fix error handling
 2021-07-21 12:37:20 -07:00
Chris Lu
67537ff05b
Merge pull request #2208 from combineads/add_rm_options 2021-07-21 11:08:43 -07:00
byunghwa.yun
e0c7708b03 Fix error handling 2021-07-21 22:48:04 +09:00
byunghwa.yun
41d559f54e Add fs rm options 2021-07-21 21:10:36 +09:00
Chris Lu
70effac0d3 configure and store remote configurations 2021-07-21 02:24:34 -07:00
Chris Lu
3138805b33
Merge pull request #2200 from combineads/add_fs_rm 
weed shell: fs.rm remove file or directory
 2021-07-21 00:35:54 -07:00
Chris Lu
a125c8fbe2
Merge pull request #2204 from nivekuil/tokenpolicy 
cassandra: use LocalQuorum for all queries
 2021-07-20 15:37:31 -07:00
nivekuil
b9a67d46c5 cassandra: use LocalQuorum for all queries 
This changes this filer store from eventual to strong consistency at the cost
of read performance.
 2021-07-20 14:47:39 -07:00
Chris Lu
54c8bc8673 Merge branch 'remote_overlay' 2021-07-19 23:18:35 -07:00
Chris Lu
7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu
93c37cfded
Update filechunk_manifest.go 2021-07-19 18:41:41 -07:00
byunghwa.yun
b23b307e08 Add fs rm 2021-07-20 08:49:28 +09:00
Chris Lu
b938df97a2 remove unused parameter 2021-07-19 02:59:12 -07:00
Chris Lu
450222dd64 add remote to filer.Entry and filer_pb entry, add RemoteConf 2021-07-19 02:47:27 -07:00
Chris Lu
8dc5def435 proto: add remote 2021-07-19 01:12:31 -07:00
Chris Lu
99155e4880 add -retry option for volume.fix.replication 
fix https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:13:46 -07:00
Chris Lu
2f209675ab Added -retry option for volumeServer.evacuate 
related to https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:08:21 -07:00
Chris Lu
fb7a1be1c4 refactor 2021-07-16 11:47:37 -07:00
nivekuil
2faf96f002 cassandra: Use LocalOne instead of One consistency 2021-07-16 04:29:46 -07:00
Chris Lu
4be5b4ff5c [volume.check.disk] skip errors and check next volumes 
fix https://github.com/chrislusf/seaweedfs/issues/2129
 2021-07-16 02:03:32 -07:00
Chris Lu
e95166d739 S 2021-07-15 17:43:38 -07:00
Chris Lu
a45bbc0b75 2.59 2021-07-15 15:52:22 -07:00
Chris Lu
f0042f62dd readable logs 2021-07-15 12:17:48 -07:00
Chris Lu
18c40686d9 s3: multipart upload miss data if file is chunked in 4MB 
fix https://github.com/chrislusf/seaweedfs/issues/2195
 2021-07-15 11:56:28 -07:00
Chris Lu
5a838dbe53 fix compilation 2021-07-13 11:40:21 -07:00
Chris Lu
6103649ffb shell: volume.check.disk adds retries in case the volumes are just moved 
related to https://github.com/chrislusf/seaweedfs/issues/2194
 2021-07-13 11:19:56 -07:00
Chris Lu
49c66e88a0 volume: change all writes to fsync during graceful stopping 
fix https://github.com/chrislusf/seaweedfs/issues/2193
 2021-07-13 01:29:57 -07:00
Chris Lu
01adc567aa shell: volume.fsck deletes a volume is the volume has only orphaned data. 
fix https://github.com/chrislusf/seaweedfs/issues/2190
 2021-07-12 11:22:00 -07:00
Chris Lu
297b41266b 2.58 2021-07-12 01:33:47 -07:00
Chris Lu
b194f91f47 add version to filer configuration response 2021-07-12 01:23:20 -07:00
bingoohuang
9e48bff8ee fix typo in fs.configure help message text 2021-07-12 13:51:21 +08:00
Chris Lu
d013d6d968 shell: volume.fsck "reallyDeleteFromVolume" should send padded file ids to delete 
fix https://github.com/chrislusf/seaweedfs/issues/2188
 2021-07-10 23:16:06 -07:00
Chris Lu
3d624d1e16 rename 2021-07-09 03:19:21 -07:00
Chris Lu
ecce300964 s3 config read via grpc 2021-07-09 02:48:03 -07:00
Chris Lu
da7bd62822 /etc files are stored inside metadata store 2021-07-09 02:33:14 -07:00
Chris Lu
a6d73e0a66
Merge pull request #2185 from bingoohuang/master 
show RemoteVolumes/EcVolumes only if it is not empty
 2021-07-06 00:28:07 -07:00
bingoohuang
ed57a55eae show RemoteVolumes/EcVolumes only if it is not empty 2021-07-06 15:20:18 +08:00
Chris Lu
44b50b2fdf another fix related to 64 bit alignment 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-05 16:30:43 -07:00
Chris Lu
141388367e
Merge pull request #2183 from bingoohuang/master 
extract embed html of master/volume/filer ui to separate files
 2021-07-05 03:22:26 -07:00
bingoohuang
44a2538f67 extract embed html of master/volume/filer ui to separate files 2021-07-05 18:09:44 +08:00
Chris Lu
e2da647fa3
Merge pull request #2182 from qieqieplus/fix-event-notification 
Subscribe meta data: sync empty notification with timestamp
 2021-07-05 02:00:53 -07:00
qieqieplus
233103f6b2 sync empty notification with timestamp 2021-07-05 16:01:16 +08:00
bingoohuang
5dbbe19c8b extract embed toml example to separate files 2021-07-05 11:16:49 +08:00
Chris Lu
5bcc77b46c volume: default readMode to proxy 2021-07-03 15:55:56 -07:00
Chris Lu
f5fa0b08fd 2.57 2021-07-03 15:10:57 -07:00
Chris Lu
a024254ad7 logging 2021-07-03 14:51:01 -07:00
Chris Lu
d39b2689a5 S3 authorization: StreamingSigned enforces access control 
fix https://github.com/chrislusf/seaweedfs/issues/2180
 2021-07-03 14:50:53 -07:00
Chris Lu
fa0dab6029 mount: rename also recursively move file handles 
related to https://github.com/chrislusf/seaweedfs/issues/2169
 2021-07-03 02:59:35 -07:00
Chris Lu
8fe75692ee volume: address "unaligned 64-bit atomic operation" 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-02 13:57:43 -07:00
Chris Lu
2cb8b31ea7
Merge pull request #2179 from nivekuil/tokenpolicy 
Cassandra: Use TokenAwareHostPolicy with fallback to localDC by default
 2021-07-02 13:51:53 -07:00
nivekuil
8425705643 Cassandra: Use TokenAwareHostPolicy by default with fallback 
See https://pkg.go.dev/github.com/gocql/gocql#hdr-Data_center_awareness_and_query_routing
 2021-07-02 13:50:01 -07:00
Chris Lu
d8bda0b229 locate the weed binary 2021-07-02 13:33:17 -07:00
danielflira
586e066897 fix weed fuse parameters parsing 2021-07-02 16:52:52 -03:00
lyg
1a4db87e19 fix s3 metadata error with multipart upload 2021-07-02 11:00:42 +08:00
Chris Lu
c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu
2420c60fc4 log reading adds delay between retries 2021-07-01 14:01:25 -07:00
Chris Lu
067eb15e70 remove debug messages 2021-07-01 01:24:07 -07:00
Chris Lu
b624090398 go fmt 2021-07-01 01:21:14 -07:00
Chris Lu
215b169562 mount: recursively rename locally 2021-07-01 01:19:31 -07:00
Chris Lu
c795183887 return node itself as directory handler 2021-07-01 01:19:31 -07:00
Chris Lu
30dbe98dff Update Makefile 2021-07-01 01:19:31 -07:00
Chris Lu
07f20155fd small optimization 2021-07-01 01:19:31 -07:00
danielflira
a66112c9d2 configurable fusermount path 2021-07-01 00:07:54 -03:00
Kevin Liu
253b4ed898
remote -> proxy in server to match volume flags 
noticed this was missed in https://github.com/chrislusf/seaweedfs/pull/2168
 2021-06-30 13:05:45 -07:00
zhangsong
20d33ae025 add proxy mode to read non-local volumes 2021-06-30 18:33:18 +08:00
zhangsong
7566782c2e add proxy mode to read non-local volumes 2021-06-30 17:28:37 +08:00
陈杨文
6f683e6572
also failed on mips64 
alpine support mips64, will try to build on next version
 2021-06-30 04:36:00 +08:00
Chris Lu
41db292332 skip s390 ppc64le 
due to https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/22703
 2021-06-29 10:06:38 -07:00
Chris Lu
3668d10664 range query for mp4 video play 
fix https://github.com/chrislusf/seaweedfs/issues/2156
 2021-06-29 02:13:29 -07:00
Chris Lu
24e11d1e90 look back when adding to sorted values 
look back when adding to sorted values, before adding it to overflow
 2021-06-28 22:46:49 -07:00
Chris Lu
fc8dd58aea volume: large_volume version has bug when using in memory index 
fix https://github.com/chrislusf/seaweedfs/issues/2162
 2021-06-28 15:48:07 -07:00
Chris Lu
a2979aa051 2.56 2021-06-27 23:33:45 -07:00
Chris Lu
08377fecb8 fixes 2021-06-27 23:32:57 -07:00
Chris Lu
17477b37d5 sleep before re-reading the messages 
If there are no more metadata changes and the client disconnects, it would go into a busy loop without this fix.
 2021-06-27 06:31:04 -07:00
Chris Lu
c764596f96 filer: slow metadata topic read may lose some change events 
fix https://github.com/chrislusf/seaweedfs/issues/2117
 2021-06-27 05:54:16 -07:00
Chris Lu
cc7714fdbe logging changes to debug 2021-06-27 05:54:16 -07:00
thephoenixofthevoid
8d70ba2eaa Fix: Loosing environment variables at StartProcess 2021-06-26 15:20:27 +03:00
Chris Lu
1cac2f2278 shell: volume.fsck checks missing chunks in filer 
fix https://github.com/chrislusf/seaweedfs/issues/2154
 2021-06-24 23:56:24 -07:00
Chris Lu
9dd09bbb33 refactor 2021-06-24 17:22:53 -07:00
Chris Lu
c2e0a75c1f adjust logs 2021-06-24 12:46:00 -07:00
Chris Lu
78b1fb921c adjust log level 2021-06-23 20:59:54 -07:00
Chris Lu
05af54ad10 2.55 
trigger migrated travis build
 2021-06-23 00:41:04 -07:00
Chris Lu
d4b0dcf442
Merge pull request #2147 from jonaz/process_metrics 
Add process metrics of weed itself
 2021-06-22 04:13:17 -07:00
Jonas Falck
829b195084 Add process metrics of weed itself 2021-06-22 13:09:42 +02:00
Chris Lu
d474ce6fe3 master: avoid repeated leader redirection 
fix https://github.com/chrislusf/seaweedfs/issues/2146
 2021-06-21 22:56:07 -07:00
danielflira
f1d207a0fe start weed with mount in background 2021-06-20 02:48:46 -03:00
Chris Lu
b3eb4fecc7 2.54 2021-06-19 03:48:15 -07:00
Chris Lu
f24bb9e688 mount: fix for deletion stopped working since 2.53 
fix https://github.com/chrislusf/seaweedfs/issues/2138

due to 4d0cbd2700
 2021-06-19 03:46:39 -07:00
Chris Lu
88d52adfdd remove unused fields 2021-06-18 15:35:22 -07:00
Chris Lu
56eb522b13 fix stats when a collection is deleted 2021-06-15 21:11:31 -07:00
Chris Lu
055374a50b FUSE: skip flushing if file is deleted 
related to https://github.com/chrislusf/seaweedfs/issues/2110
 2021-06-15 12:45:23 -07:00
Konstantin Lebedev
6aa1a56ec8 avoid crashes Galera Cluster 
https://github.com/chrislusf/seaweedfs/issues/2125
 2021-06-15 18:12:39 +05:00
Chris Lu
ebe971da2e
Merge pull request #2127 from nagy/check-history-error 
Check for history file errors as well
 2021-06-15 03:29:21 -07:00
Chris Lu
7a81caa31e 2.53 2021-06-13 17:12:34 -07:00
Chris Lu
4d0cbd2700 skip cookie checking if from grpc api 2021-06-13 16:16:11 -07:00
Chris Lu
7225cb4ac5 add block and mutex profiling 2021-06-13 16:15:54 -07:00
Chris Lu
28a4a1f8d6 fix for mysql2 postgres2 on fast dropping buckets 2021-06-13 07:31:56 -07:00
Chris Lu
1e76fc994a filer.copy: zero fileSize for directories 2021-06-13 07:09:06 -07:00
Chris Lu
ed6aa13520 minor 2021-06-12 18:36:25 -07:00
Chris Lu
ee6c67682c minor 2021-06-12 02:52:41 -07:00
Chris Lu
9357911a95 remove all bucket metadata 
fix https://github.com/chrislusf/seaweedfs/issues/2118
 2021-06-10 23:37:54 -07:00
Chris Lu
b71c3cfba4 avoid possible empty object 2021-06-10 22:17:53 -07:00
Chris Lu
310e31424e adjust the error output 
fix https://github.com/chrislusf/seaweedfs/issues/2123
 2021-06-10 21:55:13 -07:00
Chris Lu
8b382a8209 refactor 2021-06-10 21:50:21 -07:00
Chris Lu
33b87244ef refactoring 2021-06-10 16:54:36 -07:00
Daniel Nagy
7ce3bee0c0
Check for history file errors as well 2021-06-08 11:14:49 +02:00
Daniel Nagy
0e3adde47f
Return correct exitcode when wheed upload fails 2021-06-07 23:38:14 +02:00
Chris Lu
76f24af79a 2.52 2021-06-07 12:13:23 -07:00
Chris Lu
5e6dfbc25f locks for data racing 2021-06-07 12:04:50 -07:00
Chris Lu
452c6ef183 limits concurrent uploads for one file 2021-06-06 23:05:17 -07:00
Chris Lu
8295e2feb6 skip md5 checking for now because of race condition 2021-06-06 22:16:32 -07:00
Chris Lu
c8dea3dd89 2.51 2021-06-06 21:54:00 -07:00
Chris Lu
19caeb7b02 fix writing the small file 2021-06-06 20:57:03 -07:00
Chris Lu
bb45dea15a filer: parallel data upload 2021-06-06 20:23:36 -07:00
Chris Lu
e00443a940 mount: adjust starting order 
avoid possible nil wfs.Server
 2021-06-06 20:22:42 -07:00
Chris Lu
44f1ba6894 refactor 2021-06-06 18:43:04 -07:00
Chris Lu
6c82326575 use bytes.Buffer to reduce memory allocation and gc 2021-06-06 13:42:36 -07:00
Chris Lu
9cba5cca0b optionally disable concurrent upload limit 2021-06-06 13:13:33 -07:00
Chris Lu
21ad9a4ac2 filer: mongodb avoids E11000 duplicate key error collection: seaweedfs.filemeta index: directory_1_name_1 dup key 2021-06-06 13:12:01 -07:00
Chris Lu
e6ba2f9c37 verbose but cleaner logic to invalidate fuse cache 2021-06-05 13:09:37 -07:00
Chris Lu
fadc1febdb FUSE: invalidate FUSE cached entries 
fix https://github.com/chrislusf/seaweedfs/issues/2108
 2021-06-05 02:23:07 -07:00
Chris Lu
ab606dec2a filer: add path-specific option to enforce readonly 2021-06-04 01:03:41 -07:00
Chris Lu
0a5388744c
Merge pull request #2105 from Woellchen/current_and_parent_dot_directories 
Return artificial . and .. directories
 2021-06-02 12:54:44 -07:00
Patrick Schmidt
77100754e6 Return artificial . and .. directories 2021-06-02 21:28:02 +02:00
Chris Lu
62142ff1d2
Merge pull request #2104 from kmlebedev/sync_volume_on_close 
avoid data loss after restarting a container with a volum server
 2021-06-02 11:47:57 -07:00
Chris Lu
cc34475012 remove file handle locking for setattr 2021-06-02 11:44:12 -07:00
Konstantin Lebedev
99ef280c7c avoid data loss after restarting a container with a volum server 2021-06-02 17:07:19 +05:00
Chris Lu
3db1642392 mount: skip persisting metadata if file is open 2021-06-02 03:40:08 -07:00
Chris Lu
556cc3a4ca mount: avoid exception if disk cache is not initialized 
related to https://github.com/chrislusf/seaweedfs/issues/2102
 2021-05-31 16:42:55 -07:00
Chris Lu
615cb24ba6 shell: use tab print style for completion 2021-05-31 03:29:29 -07:00
Chris Lu
c5de97ddbe keep alive for streaming connections 
fix https://github.com/chrislusf/seaweedfs/issues/2096

related to https://github.com/chrislusf/seaweedfs/issues/1277
 2021-05-31 01:26:24 -07:00
Chris Lu
2b60e2abb1 only disallow streaming signed when no auth enabled 
fix https://github.com/chrislusf/seaweedfs/issues/2101
 2021-05-31 01:03:04 -07:00
Chris Lu
4233ad3f07 2.50 2021-05-30 20:40:30 -07:00
Chris Lu
1456616a77 recreate grpc connections if too many errors 
address https://github.com/chrislusf/seaweedfs/issues/2098
 2021-05-30 00:07:43 -07:00
Chris Lu
fb8036385a s3: save metadata during put-object 
fix https://github.com/chrislusf/seaweedfs/issues/2092
 2021-05-29 14:14:30 -07:00
Chris Lu
2d7b4e5bb6 filer.backup: escape colon from path on windows 
fix https://github.com/chrislusf/seaweedfs/issues/2084
 2021-05-29 06:45:27 -07:00
Chris Lu
ae185b997f
Merge pull request #2099 from danielflira/mount-helper 
fix parameter multiple values
 2021-05-28 21:57:15 -07:00
danielflira
849f36c1ac fix parameter multiple values 2021-05-29 00:37:25 -03:00
Eugeniy Kozlov
6e8bd16819 freespace params fix 2021-05-28 14:19:24 +03:00
Chris Lu
4d55132c7d scaffold add comments 2021-05-27 14:40:25 -07:00
Chris Lu
45bffc92a8 filer.backup: fix cloud sinks when updating entry 2021-05-26 14:53:11 -07:00
Chris Lu
921e0d5008 remove verbose log 2021-05-26 14:43:34 -07:00
Chris Lu
1a70cb9b63 local sink: write and update files 
fix issues with https://github.com/chrislusf/seaweedfs/issues/2084
 2021-05-26 14:42:21 -07:00
Chris Lu
e699d16c85
Merge pull request #2085 from danielflira/mount-helper 
create fuse subcommand to use weed with mount
 2021-05-26 12:19:40 -07:00
danielflira
84488ebb33 replace filer.remote and parts[1] 2021-05-26 12:07:36 -03:00
Chris Lu
ac9bf71544 use jquery 3.6.0 2021-05-26 00:21:11 -07:00
danielflira
877c192af2 create fuse subcommand to use weed with mount 2021-05-25 23:32:35 -03:00
Chris Lu
67dd094b35 sqlite: does not support non-linux/darwin/windows 2021-05-25 17:36:15 -07:00
Chris Lu
dce1f02c9e filer.backup: backup to local directory optionally is incremental 
fixed one issue with https://github.com/chrislusf/seaweedfs/issues/2084
 2021-05-25 17:19:20 -07:00
Konstantin Lebedev
b612d5aebd s3 test get w/ If-Match: bogus ETag 2021-05-24 16:59:44 +05:00
Konstantin Lebedev
69c768870b - object write cache control 
- object write expires
 2021-05-24 15:43:55 +05:00
Konstantin Lebedev
84dce32a57
Merge branch 'master' into head_check_all_chunks 2021-05-24 12:28:19 +05:00
Konstantin Lebedev
0e404cd84b Chunk download stats 2021-05-24 12:14:50 +05:00
Konstantin Lebedev
bb42633522 rm chunk Download stats 2021-05-24 12:08:12 +05:00
Chris Lu
064269bb57 filer: Support sqlite as filer meta store 2021-05-23 23:58:28 -07:00
Chris Lu
b70aeb7585 bucket list only directories 2021-05-23 10:36:22 -07:00
Chris Lu
42fb03a66e 2.49 2021-05-23 00:51:47 -07:00
Chris Lu
3d3fa43542 filer: re-create grpc connections if having transport error 
fix https://github.com/chrislusf/seaweedfs/issues/2070
 2021-05-22 17:46:53 -07:00
Chris Lu
2d0a0733ec filer: bootstrap from peer filer 
fix https://github.com/chrislusf/seaweedfs/issues/1861
 2021-05-22 03:17:02 -07:00
Chris Lu
e5a2bf1287 s3: deprecating filer.options.buckets_fsync 2021-05-22 00:24:23 -07:00
Chris Lu
431684798b s3: add errors if requests are signed by no authentication is setup 
fix https://github.com/chrislusf/seaweedfs/issues/2075
 2021-05-21 14:08:47 -07:00
Chris Lu
2f136a04a1 organize a bit better for temp file directory 2021-05-21 13:02:18 -07:00
Konstantin Lebedev
3325b850cc rm func CheckAllChunkViews 2021-05-21 16:05:59 +05:00
Konstantin Lebedev
26a4f34a57 del checks 
url err logging
stats chunk fetch
 2021-05-21 15:59:12 +05:00
Chris Lu
e8b7d4ad64 simplify a bit 2021-05-21 02:10:44 -07:00
Chris Lu
0b00edfc3b waits in case master lost connection 2021-05-21 02:09:22 -07:00
Chris Lu
f37a4cbd0f randomize initial filer 2021-05-21 01:41:34 -07:00
Chris Lu
5d77840cff adjust help message 2021-05-21 01:38:57 -07:00
Chris Lu
dc1309f084 FUSE mount: support multiple filers 
fix https://github.com/chrislusf/seaweedfs/issues/2015
fix https://github.com/chrislusf/seaweedfs/issues/1531
 2021-05-21 01:28:00 -07:00
Konstantin Lebedev
42f631f549 fix https://github.com/chrislusf/seaweedfs/issues/2046 2021-05-20 13:19:35 +05:00
Chris Lu
30c67e3652 minor 2021-05-19 23:59:39 -07:00
Konstantin Lebedev
03d1199d5f Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader" 
This reverts commit ac71117e
 2021-05-20 11:45:21 +05:00
Chris Lu
87a32bfef4 avoid possible nil when node is disconnected from its parent 
fix https://github.com/chrislusf/seaweedfs/issues/2073
 2021-05-19 10:02:01 -07:00
Chris Lu
1aa7e99a89 skip file not found error when deleting 2021-05-15 09:37:39 -07:00
Chris Lu
45a762223a 2.48 2021-05-14 10:26:42 -07:00
Chris Lu
789465d99e set renamed item to new directory 
related to https://github.com/chrislusf/seaweedfs/issues/2064
 2021-05-13 22:52:19 -07:00
Chris Lu
83c037e093 fix logs 2021-05-13 22:41:19 -07:00
Chris Lu
5a29d284fa ensure proper directory name 
a fix related to https://github.com/chrislusf/seaweedfs/issues/2064
 2021-05-13 10:18:35 -07:00
Chris Lu
3ff307e842 reduce logs 2021-05-13 00:32:37 -07:00
Chris Lu
a48ebd7c73 mount: read file when file is still being written 
a possible fix for https://github.com/chrislusf/seaweedfs/issues/2065
 2021-05-13 00:32:23 -07:00
Chris Lu
26a55bbb5c Adjust error message when bucket name conflicts with existing collections 
fix https://github.com/chrislusf/seaweedfs/issues/2069
 2021-05-12 22:30:39 -07:00
Chris Lu
ceb620a30a directory rename: change directory name after renaming 
fix https://github.com/chrislusf/seaweedfs/issues/2068
 2021-05-12 22:04:47 -07:00
Chris Lu
b430d1b6ee filer.copy: "check.size" before copying files 
fix https://github.com/chrislusf/seaweedfs/issues/2067
 2021-05-12 21:45:39 -07:00
Chris Lu
0f7b43af99 2.47 2021-05-11 10:12:15 -07:00
Chris Lu
d2d36a3f9d master: avoid creating too many volumes 
fix https://github.com/chrislusf/seaweedfs/issues/2062
 2021-05-11 10:05:31 -07:00
Chris Lu
4596e64710 2.46 2021-05-10 21:58:37 -07:00
Chris Lu
e55aa41690 Merge branch 'upload_via_temp_file' 2021-05-10 21:48:18 -07:00
Chris Lu
1737af480a adjust logs 2021-05-10 21:47:51 -07:00
Chris Lu
d06ecc2649 working properly 2021-05-10 21:47:07 -07:00
Chris Lu
fe2edd4b50 2.45 2021-05-10 13:53:03 -07:00
Chris Lu
55d547afec switch back to in memory uploading for now 2021-05-10 13:50:10 -07:00
Chris Lu
9a6aa00e9d avoid nil locations 
fix https://github.com/chrislusf/seaweedfs/issues/2059
 2021-05-10 02:39:52 -07:00
Chris Lu
da0a4e775b Revert "Revert "Merge pull request #2027 from bingoohuang/master"" 
This reverts commit d74cdf0115.
 2021-05-10 00:13:55 -07:00
Chris Lu
aa6949ef25 2.44 2021-05-09 23:28:48 -07:00
Chris Lu
ba77833343 EOF handling 2021-05-09 23:04:24 -07:00
Chris Lu
f4cf30b497 mount: write to tmp file before uploading 2021-05-09 22:56:10 -07:00
Chris Lu
280e7cf289 mount: in case the set attribute is called before persisting the file 2021-05-09 22:55:30 -07:00
Chris Lu
74052064b6 add one missing optimization 2021-05-09 17:25:42 -07:00
Chris Lu
59ace54925 refactor 2021-05-09 17:22:30 -07:00
Chris Lu
50be19d23e refactor 2021-05-09 15:33:01 -07:00
Chris Lu
55e060cf61 refactor 2021-05-09 15:28:54 -07:00
Chris Lu
735e65be17 refactor 2021-05-09 15:22:38 -07:00
Chris Lu
93e84a12f2 refactor 2021-05-09 15:15:18 -07:00
Chris Lu
3942e3b2ef a better fix 2021-05-09 01:42:19 -07:00
Chris Lu
957e1a1bc1 fuse: important: if filer -filer.path is not root, directory listing will fail 2021-05-09 00:32:21 -07:00
Chris Lu
7ca75347ec minor 2021-05-07 21:56:45 -07:00
Chris Lu
8f8738867f add retry to assign volume 
fix https://github.com/chrislusf/seaweedfs/issues/2056
 2021-05-07 07:29:26 -07:00
Chris Lu
007401f3a0 remove duplicated code 2021-05-07 07:14:24 -07:00
Chris Lu
aaad4b578b
Merge pull request #2049 from qieqieplus/async-assign 
ahead of time volume assignment
 2021-05-06 10:00:25 -07:00
qieqieplus
c4d32f6937 ahead of time volume assignment 2021-05-06 18:55:44 +08:00
Chris Lu
55a8f57381 go fmt 2021-05-06 03:37:51 -07:00
Chris Lu
5753749c90 remove verbose logs 2021-05-06 03:34:34 -07:00
Chris Lu
38f411219a mount: skip local chunk cache if opened write only 2021-05-06 03:31:40 -07:00
Chris Lu
c899bdf063 a little optimization 2021-05-06 03:03:00 -07:00
Chris Lu
3eb336e0b0 report error only for the first multipart upload 
the glog.Errorf would always print for s3 multipart uploads
 2021-05-06 01:57:54 -07:00
Chris Lu
76c48ffe27 optional parallel copy ec shards 
fix https://github.com/chrislusf/seaweedfs/issues/2048
 2021-05-06 01:53:35 -07:00
Chris Lu
ac71117ee6 revert PR #1903 avoid http error: superfluous response.WriteHeader 2021-05-05 15:11:39 -07:00
Chris Lu
24efa31e49
Merge pull request #2045 from qieqieplus/fix-vacuum-commit 2021-05-05 07:41:38 -07:00
qieqieplus
ac26080bd2 fix concurrent vacuum & delete panic 2021-05-05 17:54:50 +08:00
Chris Lu
e24ba2aadc filer: delete specific tags 
fix https://github.com/chrislusf/seaweedfs/issues/2041
 2021-05-02 21:53:43 -07:00
Chris Lu
e87e6ef33c s3: return 404 if bucket does not exist 
fix https://github.com/chrislusf/seaweedfs/issues/2039
 2021-05-02 21:30:37 -07:00
Chris Lu
a4bb37a5fe add debug info 2021-05-01 16:10:26 -07:00
Chris Lu
2e56407c6b fix visited checking 2021-05-01 16:09:29 -07:00
Chris Lu
c48ef78670 2.43 2021-05-01 00:39:04 -07:00
Chris Lu
3a86d4dbfd mount: fix directory invalidation 
fix https://github.com/chrislusf/seaweedfs/issues/2038
 2021-04-30 22:51:06 -07:00
Chris Lu
d74cdf0115 Revert "Merge pull request #2027 from bingoohuang/master" 
Need to revert because docker image build failed. The docker apk package only has go 1.15.
 2021-04-30 03:36:15 -07:00
Chris Lu
84312e6799 2.42 2021-04-30 03:14:07 -07:00
Chris Lu
fa0d973113
Merge pull request #2037 from utsl42/reader_at 
make reader_at handle random reads more efficiently for FUSE
 2021-04-28 17:31:46 -07:00
Nathan Hawkins
042de9359c make reader_at handle random reads more efficiently for FUSE 2021-04-28 19:13:37 -04:00
Chris Lu
a26a37dfa3 fix compilation 
fix related to #2032
 2021-04-28 13:36:53 -07:00
Chris Lu
ef94ff6837
Merge pull request #2035 from kmlebedev/fix_chunks_etag 
fix aws style Etag for chunks
 2021-04-28 10:37:18 -07:00
Chris Lu
8ae3ea4b1d
Merge pull request #2034 from kmlebedev/iam_listkeys_filter 
iam ListAccessKeys filtred by username
 2021-04-28 10:34:40 -07:00
Chris Lu
9dca75aea8
Merge pull request #2032 from tobiasmuehl/patch-2 
Detect rar archives by mime type
 2021-04-28 10:31:19 -07:00
Konstantin Lebedev
c2269123d3 fix aws style Etag for chunks 2021-04-28 22:28:05 +05:00
Konstantin Lebedev
39f636c682 iam ListAccessKeys filtred by username 2021-04-28 17:15:22 +05:00
Tobias Mühl
a8864e2abd
Detect rar archives by mime type 
RAR archives might not have .rar extension, see [Wikipedia](https://en.wikipedia.org/wiki/RAR_(file_format))
 2021-04-28 15:54:19 +07:00
Tobias Mühl
12a7e87007
Do not compress brotli archives 2021-04-28 15:51:49 +07:00
Konstantin Lebedev
a48785c7df auth use bucket wild cards 2021-04-27 21:45:40 +05:00