mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-29 14:31:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,578 Commits 32 Branches 296 Tags 148 MiB
77f90ae288
Commit Graph

4330 Commits

Author SHA1 Message Date
Chris Lu
e281f0fa82 refactor 2021-09-02 06:53:21 -07:00
Chris Lu
24fda6f47f remove QR code for mobile upload 2021-09-01 16:59:37 -07:00
Chris Lu
7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu
d1a4e19a3f volume: copy file also copies modification time 
to ensure ttl can work well
 2021-09-01 02:42:57 -07:00
Chris Lu
3bd48c4f29 filer.remote.sync: exit when directory is unmounted 
this will not propagate the deletions back to the cloud
 2021-09-01 01:29:22 -07:00
Chris Lu
3faaa6e360 ensure cached client with updated storage conf 2021-09-01 01:27:45 -07:00
Chris Lu
8e125339d5 line wrap 2021-08-31 23:30:28 -07:00
Chris Lu
68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu
43fd11278e support follow additional path prefixes 2021-08-31 23:23:08 -07:00
Chris Lu
cca588429b shell: fs.meta.save skip saving system logs 2021-08-31 23:05:56 -07:00
Chris Lu
0ee208c1a4 adjust example 2021-08-31 02:04:32 -07:00
Chris Lu
97b7948cd9 adjust error message 2021-08-31 01:07:34 -07:00
Chris Lu
c9205458a1 cloud drive: create mount directory if not exists 2021-08-31 01:07:19 -07:00
Chris Lu
1dfcdc53b7 cloud drive: add storj, filebase 2021-08-30 17:43:14 -07:00
Chris Lu
701cc133f3 cloud drive: s3 add option for v4 signature 2021-08-30 17:28:33 -07:00
Chris Lu
f117f7bf54 adjust comment 2021-08-30 15:56:14 -07:00
Chris Lu
72d4ff6b28
Merge pull request #2280 from blacktear23/filerstore-tikv 
Use TiKV as Filerstore
 2021-08-30 02:21:14 -07:00
Chris Lu
f811fd0903 fix build 2021-08-30 02:03:08 -07:00
yulai.li
b17b81529e Add build tags for TiKV filerstore 2021-08-30 15:59:25 +08:00
Rain Li
e5f9ff983d
Merge branch 'master' into filerstore-tikv 2021-08-30 15:52:54 +08:00
Chris Lu
44d0ac95f6 add cluster id if already set 2021-08-29 22:19:46 -07:00
Chris Lu
18bf556d93 fix comments 2021-08-29 22:19:25 -07:00
Chris Lu
21f37a0a75 remove unused code 2021-08-29 22:19:11 -07:00
Chris Lu
5adfdd7982 change to a new bloom filter library 2021-08-29 21:26:38 -07:00
Chris Lu
05f32376eb add cluster id in filer configuration response 2021-08-29 21:02:10 -07:00
Chris Lu
eacaa44dc2 refactor 2021-08-29 19:13:48 -07:00
Chris Lu
6deee4c0b9 import hdfs 2021-08-29 18:49:43 -07:00
Chris Lu
a31f2907f0 cloud drive: filer.remote.sync supports remove folder 2021-08-29 18:46:28 -07:00
Chris Lu
001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu
05d2774117 refactoring 2021-08-28 22:49:38 -07:00
Chris Lu
975fbc9ee3 add wasabi example 2021-08-28 22:49:38 -07:00
Chris Lu
bec3f63298 2.65 2021-08-28 05:27:33 -07:00
Chris Lu
223c1b3571 adjust invalidation logic 2021-08-28 05:21:01 -07:00
nivekuil
8c523d50ad Invalidate mount meta cache more aggressively 2021-08-28 04:26:37 -07:00
Chris Lu
65a29ad42a add warning on unmount a folder 2021-08-28 02:25:48 -07:00
Chris Lu
a22f37b01c remove tencent region 2021-08-28 02:23:03 -07:00
Chris Lu
a9b41900d8 handle cases when mounted directory does not exist 2021-08-28 01:56:56 -07:00
Chris Lu
0d8936f6cd add aliyun example 2021-08-28 01:09:48 -07:00
Chris Lu
dc481c081c remove unused function 2021-08-27 22:18:54 -07:00
Chris Lu
9242f3aaeb fix build 2021-08-26 17:33:57 -07:00
Chris Lu
49a8dfb976 adjust default concurrent level 2021-08-26 17:05:56 -07:00
Chris Lu
6a0bb7106b cloud drive: parallelize remote storage downloading 2021-08-26 16:16:26 -07:00
Chris Lu
05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
yulai.li
c1dc5ab4ac Add deleterange_concurrency to filer configuration file 2021-08-26 18:25:08 +08:00
yulai.li
318757ef8c Change DeleteFolderChildren to DeleteRange api 2021-08-26 17:49:56 +08:00
yulai.li
de8ef28460 Update filer.toml scaffold to support tikv store 2021-08-26 16:26:27 +08:00
yulai.li
546efeba8f Fix build bug 2021-08-26 16:20:35 +08:00
yulai.li
2088f28424 init post 2021-08-26 15:20:18 +08:00
Chris Lu
c08ac536ed cloud drive: add support for Wasabi 
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
 2021-08-25 17:34:29 -07:00
Chris Lu
9bcf94b2b1 ensure multi-threaded correctness 2021-08-25 17:28:50 -07:00
Chris Lu
a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu
2d805beed4 add help mesages 2021-08-24 23:28:35 -07:00
Chris Lu
c6c97bd83d add default env variables 2021-08-24 23:25:36 -07:00
Chris Lu
a19c728034 cloud drive: add support for Tencent COS 2021-08-24 23:19:45 -07:00
Chris Lu
19a81d25af cloud drive: add support for Aliyun OSS 2021-08-24 23:14:24 -07:00
Chris Lu
47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00
Chris Lu
6bab20d862 cat entry content 2021-08-24 02:38:32 -07:00
Chris Lu
e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu
7c39a18ba5 update azure library 2021-08-24 00:32:35 -07:00
Chris Lu
a78d0227cd adjust package name 2021-08-23 23:19:31 -07:00
Chris Lu
12631a3f5b cloud drive: gcs simplify a little bit 2021-08-23 14:43:01 -07:00
Chris Lu
00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu
f0cc130849 do not force path style for better compatibility 2021-08-23 03:09:41 -07:00
Chris Lu
2836a58d87 cloud drive: S3 supports storage class 2021-08-23 02:18:59 -07:00
Chris Lu
95e2b83ca5 fix format 2021-08-23 00:49:59 -07:00
Chris Lu
ff7dc3b44c 2.64 2021-08-23 00:39:15 -07:00
Chris Lu
ca35a77e85 adjust help message 2021-08-23 00:37:55 -07:00
Chris Lu
258063de26 cloud drive: add google cloud storage 2021-08-23 00:29:27 -07:00
Chris Lu
df1d6133a8 bptree does not work well for auto-increasing keys 2021-08-22 18:19:26 -07:00
Chris Lu
3b2c39f1be mount: set name when mount path equals mount folder 
fix https://github.com/chrislusf/seaweedfs/issues/2275#issuecomment-903255876
 2021-08-22 04:59:09 -07:00
Chris Lu
51c8f2518f change key type to ItemKey 2021-08-21 15:54:42 -07:00
Chris Lu
b3e49d2758 change value type to ItemValue 2021-08-21 15:52:17 -07:00
Chris Lu
38c8470d1d add back non_dedup 2021-08-21 15:13:13 -07:00
Chris Lu
849f185a20 add memory kv store 2021-08-21 15:00:44 -07:00
Chris Lu
5f6cc9a814 make proto node 2021-08-21 13:36:52 -07:00
Chris Lu
2158d4fe4d adjust help message 2021-08-21 02:17:10 -07:00
Chris Lu
172da83449 bpnode use get prev and next 2021-08-20 18:50:16 -07:00
Chris Lu
01661ec6a7 move to getter setter file 2021-08-20 18:37:34 -07:00
Chris Lu
0c360eb6b2 add getter and setter for root of tree and map 2021-08-20 18:34:50 -07:00
Chris Lu
1b1ab331f6
Merge pull request #2274 from qieqieplus/gzip-pool 2021-08-20 04:22:41 -07:00
Chris Lu
88d68cad87 remove dedup 2021-08-20 04:14:52 -07:00
qieqieplus
7720533f84 reduce gzip allocation 2021-08-20 18:38:18 +08:00
Chris Lu
2d237da74a remove size since each put/get will have to update the root node 2021-08-20 01:19:11 -07:00
Chris Lu
ec72547c8d started by copying from https://sourcegraph.com/github.com/timtadh/data-structures@master/-/tree/tree/bptree 2021-08-20 01:12:52 -07:00
Chris Lu
05fc7db755 filer.copy: large files pack file ids into chunks 2021-08-18 23:46:54 -07:00
mauroparente
c63713774b
Fix image resize 
https://github.com/chrislusf/seaweedfs/issues/2262
 2021-08-18 23:15:35 +02:00
Chris Lu
570d2eccec skip body if not allowed by http status 2021-08-18 00:56:35 -07:00
Chris Lu
ccb3df41f8 elide secret key 2021-08-17 11:27:08 -07:00
Konstantin Lebedev
865b06b7fe The IEEE assembler optimizations has been submitted and will be part of the Go 1.6 standard library. 
https://github.com/klauspost/crc32
 2021-08-17 13:06:48 +05:00
Chris Lu
e2aa3cf63b fix go test 2021-08-15 23:20:46 -07:00
Chris Lu
1f35d32be0 2.63 2021-08-15 23:14:59 -07:00
Chris Lu
40dc283b2d fix locating data chunks 2021-08-15 23:07:58 -07:00
Chris Lu
72eb6d5b9d ensure no writes to remote storage if content is not changed 2021-08-15 20:23:41 -07:00
Chris Lu
5a7c40510f format output 2021-08-15 20:07:13 -07:00
Chris Lu
bb94930196 add some delays if error 2021-08-15 20:06:47 -07:00
Chris Lu
5d5a21ba2d adjust log format 2021-08-15 19:46:45 -07:00
Chris Lu
c45c12aedc metadata updates 2021-08-15 19:27:30 -07:00
Chris Lu
507411bdf0 fix https://github.com/chrislusf/seaweedfs/issues/2260 
fix https://github.com/chrislusf/seaweedfs/issues/2260
 2021-08-15 19:16:00 -07:00
Chris Lu
c3ffd457ef fix compilation error 2021-08-15 12:40:22 -07:00
Chris Lu
49b5e47bd1 retry forever with filer.remote.sync, and some refactoring 2021-08-15 12:38:26 -07:00
Chris Lu
fda2fc47b1 add RetryForever 2021-08-15 12:37:35 -07:00
Chris Lu
a539d64896 refactor 2021-08-15 12:09:54 -07:00
Chris Lu
8f7d2d317f readerAt need to use the right offset 
fix https://github.com/chrislusf/seaweedfs/issues/2259
 2021-08-15 11:55:58 -07:00
Chris Lu
0b2a92d371 skip if http.StatusNoContent 2021-08-15 11:27:49 -07:00
Chris Lu
ec989b0377 remove println 2021-08-15 02:10:27 -07:00
Chris Lu
9462f5129a shell: add "remote.meta.sync" 2021-08-15 01:53:46 -07:00
Chris Lu
3ada61c875 rename 2021-08-14 21:50:35 -07:00
Chris Lu
c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu
cb53802752 adjust help message 2021-08-14 15:55:53 -07:00
Chris Lu
889b143fa7 adjust modification detection logic 2021-08-14 15:44:47 -07:00
Chris Lu
f365af81c2 parallelize remote content fetching 2021-08-14 15:41:37 -07:00
Chris Lu
53e66980b2 add comments 2021-08-14 15:16:10 -07:00
Chris Lu
9921801e0c Revert "use default or path-specific setting for cache replication level" 
This reverts commit ba6923b223.
 2021-08-14 15:14:26 -07:00
Chris Lu
ba6923b223 use default or path-specific setting for cache replication level 2021-08-14 15:14:01 -07:00
Chris Lu
708debca14 remote.cache and uncache: more flexible options to select files to cache or uncache 2021-08-14 15:11:55 -07:00
Chris Lu
08258a819d fix mistake 2021-08-14 05:10:30 -07:00
Chris Lu
4909bd9684 gRpc connection error on filer when no volume left #2243 
fix https://github.com/chrislusf/seaweedfs/issues/2243

grpc do not cache connections only when connection problem happens.
Normal error results should not close the shared grpc connection.
 2021-08-14 05:06:44 -07:00
Chris Lu
8126ab4b5d rename 2021-08-14 05:03:45 -07:00
Chris Lu
0db2517994 go fmt 2021-08-14 02:55:44 -07:00
Chris Lu
7937db52e1 Filer locationPrefix configure does not exec replication #2257 
fix https://github.com/chrislusf/seaweedfs/issues/2257
 2021-08-14 02:54:13 -07:00
Chris Lu
0c66b173a4 fix 2021-08-13 11:31:43 -07:00
Chris Lu
e02a317d3d adjust retry logic in case some data is partially written 2021-08-13 11:30:38 -07:00
Chris Lu
2d519c6cb6 adjust the retry logic 2021-08-13 11:13:30 -07:00
Chris Lu
b961fcd338 filer: stream read from volume server, reduce memory usage 2021-08-13 11:00:11 -07:00
Chris Lu
f4decf02df volume copying: clean up stale volume data files 
fix https://github.com/chrislusf/seaweedfs/issues/2250
 2021-08-13 03:24:21 -07:00
Chris Lu
0f7d4556d8 shell: volume.tier.move makes up changes if volume move failed 2021-08-13 03:09:28 -07:00
Chris Lu
333cdce485 add verbose message 2021-08-13 02:57:14 -07:00
Chris Lu
78e8ddf910 Only when tailing volume, the zero-ed cookie should skip checking. 
This only happens when checkCookie == false and fsync == false.
 2021-08-13 02:09:35 -07:00
Chris Lu
a8617c1a39 tail volume: fix zero cookie problem from batch deletion 2021-08-13 01:54:35 -07:00
Chris Lu
5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu
6238644c35 remove gateway command 2021-08-12 20:52:04 -07:00
Chris Lu
d1d1fc772c move some volume lookup operations to grpc 
jwt related lookup will come in next commit
 2021-08-12 20:33:00 -07:00
Chris Lu
5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu
5469019852 adjust data type 2021-08-12 17:54:34 -07:00
Chris Lu
bfac55e6c0 avoid integer overflow 
fix https://github.com/chrislusf/seaweedfs/issues/2254
 2021-08-11 22:22:49 -07:00
Chris Lu
5516fa8e80
Merge pull request #2251 from kmlebedev/rertyUpdateIamIdentity 
Retry save and update IAM identity
 2021-08-11 09:21:03 -07:00
Konstantin Lebedev
f0afd35eec Retry save and update IAM identity 
https://github.com/chrislusf/seaweedfs/issues/2242
 2021-08-11 19:29:04 +05:00
byunghwa.yun
775dfbae85 Synchronize number of open files 2021-08-11 23:14:56 +09:00
Konstantin Lebedev
ec09966fd3 Retry save and update IAM identity 
https://github.com/chrislusf/seaweedfs/issues/2242
 2021-08-11 17:53:48 +05:00
Chris Lu
f2cd753bf9 fix avoid lock error 
fix https://github.com/chrislusf/seaweedfs/issues/2247
 2021-08-10 14:34:13 -07:00
Chris Lu
01336d71eb minor 2021-08-10 13:04:33 -07:00
Chris Lu
1154e23e2d add logs for volume creation 2021-08-10 13:04:25 -07:00
Chris Lu
057ef429ac format 2021-08-10 12:33:29 -07:00
Chris Lu
b63b042afc dedup keeps the largest replica 2021-08-10 12:30:41 -07:00
Chris Lu
0526db12e2 do not treat read only volumes differently 2021-08-10 11:37:12 -07:00
Chris Lu
e50a5b8e28 minor: print disk type 2021-08-10 11:10:09 -07:00
Chris Lu
db6275a0c8 print out balance ratio 2021-08-10 10:43:42 -07:00
Chris Lu
9d85569c55 ensure using local quorum consistency 2021-08-10 05:10:57 -07:00
Chris Lu
85832d02c0 wait for goroutines 2021-08-10 04:13:12 -07:00
Bl1tz23
1c94b3d013 merge master, resolve conflicts 2021-08-10 13:45:24 +03:00
Bl1tz23
e6e57db530 Add liveness\readiness probe for s3 api handler on /status path 2021-08-10 13:42:46 +03:00
Chris Lu
8ff6c9a0c6 output format 2021-08-10 03:25:18 -07:00
Chris Lu
48f448ee09 parallelize tier move 2021-08-10 03:08:29 -07:00
Chris Lu
69a6da7969 avoid fail on tail error 2021-08-10 02:50:28 -07:00
Chris Lu
18228f3044 fix help message 2021-08-10 02:48:41 -07:00
Chris Lu
69655ba8e5 mount: cache on reading remote storage 2021-08-09 22:11:57 -07:00
Chris Lu
a7012d9729 fix 2021-08-09 16:03:03 -07:00
Chris Lu
02f728cb54 fix bug with remote.uncache 2021-08-09 15:24:21 -07:00
Chris Lu
8d3e275735 remote: filer cache remote content on read 2021-08-09 15:16:45 -07:00
Chris Lu
9096f6f4f7 cache: set upper limit of chunk size 2021-08-09 15:08:53 -07:00
Chris Lu
402315f117 go fmt 2021-08-09 14:37:34 -07:00
Chris Lu
a6be2520c9 fix 2021-08-09 14:37:25 -07:00
Chris Lu
713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu
8cfd487608 2.62 2021-08-08 23:33:12 -07:00
Chris Lu
734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu
96ce85f5ae rename 2021-08-08 22:33:31 -07:00
Chris Lu
c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu
df85f7a1eb adjust help message 2021-08-08 22:30:12 -07:00
Chris Lu
c0b12da4ef shell: add filer.remote.unmount 2021-08-08 22:26:37 -07:00
Chris Lu
882a93dacd fix tests 2021-08-08 17:56:26 -07:00
Chris Lu
7412ccdf88 write back remote entry to local entry after uploading to remote 2021-08-08 17:55:03 -07:00
Chris Lu
dcf614a8c3 skip if the remote entry update is because of internal managerial operations 2021-08-08 15:58:10 -07:00
Chris Lu
eed26af266 Merge branch 'master' into add_remote_storage 2021-08-08 15:48:04 -07:00
Chris Lu
4370a4db63 use int64 for volume count in case of negative overflow 2021-08-08 15:19:39 -07:00
Chris Lu
26c222f596 shell: volume.tier.move avoid moving all volumes to one destination 2021-08-08 15:12:39 -07:00
Chris Lu
13e45e1605 filer.remote.sync can work now 2021-08-08 01:21:42 -07:00
Chris Lu
8f5170c138 remove imports 2021-08-07 16:20:17 -07:00
Chris Lu
46b0cb8c86 Merge branch 'master' into add_remote_storage 2021-08-07 15:41:27 -07:00
Chris Lu
de730b079d ChunkStreamReader implenents io.ReaderAt 2021-08-07 15:41:07 -07:00
Chris Lu
59732a0529 refactoring 2021-08-07 15:35:27 -07:00
Chris Lu
ecb234f75a refactor 2021-08-07 14:46:23 -07:00
Chris Lu
270770d7d7 refactor 2021-08-07 14:18:53 -07:00
Chris Lu
679f800caa Merge branch 'master' into add_remote_storage 2021-08-06 20:27:12 -07:00
Chris Lu
0c0f77e2ae skip not found error on deletion 2021-08-06 19:35:47 -07:00
Chris Lu
8828f485c0 print volume deletion error 2021-08-06 19:30:22 -07:00
Chris Lu
d2b23f3d38
Merge pull request #2238 from combineads/change_default_mode 
Change default permissions
 2021-08-05 21:28:06 -07:00
Chris Lu
e23ca3845b
Merge pull request #2237 from combineads/add_mtime 
Add crtime and mtime
 2021-08-05 21:27:13 -07:00
Chris Lu
286e5dd375 Merge branch 'master' into add_remote_storage 2021-08-05 21:07:04 -07:00
Chris Lu
1e22166939 adjust error message 2021-08-05 21:06:55 -07:00
byunghwa.yun
f3dc909b21 Change default permissions 2021-08-06 12:24:35 +09:00
byunghwa.yun
e84fad9acf Add crtime and mtime 2021-08-06 12:13:37 +09:00
Chris Lu
1a5d29520c add tests 2021-08-05 14:49:24 -07:00
Chris Lu
6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu
b9ecf1e3a8 refacotring 2021-08-04 14:56:13 -07:00
Chris Lu
d84c311699 refactoring 2021-08-04 12:30:18 -07:00
Chris Lu
f6a9ad8001 fix tests 2021-08-04 00:31:06 -07:00
Chris Lu
42969c9c62 Merge branch 'master' into add_remote_storage 2021-08-03 21:20:05 -07:00
Chris Lu
8cf0c515bf shell: volume.fix.replication retries even when there is no error #2235 
fix https://github.com/chrislusf/seaweedfs/issues/2235
 2021-08-03 09:32:55 -07:00
Chris Lu
d2ddf1dbdb typo 2021-08-03 09:32:31 -07:00
Chris Lu
3afbf04007 add TLS grpc support for filer meta clients 2021-08-03 01:30:35 -07:00
Chris Lu
fe60c6ef9a minor changes 2021-08-02 11:49:40 -07:00
byunghwa.yun
bdc7730fdb Add autocomplete 2021-08-03 02:56:52 +09:00
Chris Lu
2ae9705442 adjust text 2021-08-01 22:55:19 -07:00
Chris Lu
3739717092 Revert "adds a test" 
This reverts commit f690643b47.
 2021-08-01 22:54:45 -07:00
Chris Lu
f690643b47 adds a test 2021-08-01 22:53:50 -07:00
Chris Lu
6de786185d volume.balance: balance read only volumes first 2021-08-01 22:32:50 -07:00
Chris Lu
3bb640b786 add -force option following #2228 2021-08-01 20:03:05 -07:00
byunghwa.yun
9e839cb5cc Add force option in volume.deleteEmpty command 2021-08-02 11:37:20 +09:00
Chris Lu
56ee1d5ef1 2.61 2021-08-01 15:50:19 -07:00
Chris Lu
81b255df8b Merge branch 'master' into add_remote_storage 2021-08-01 15:44:21 -07:00
Chris Lu
58bc3ecf47 add default quietFor value 2021-08-01 15:36:06 -07:00
Chris Lu
2ca1839d77 shell: add volume.deleteEmpty command 2021-08-01 15:33:45 -07:00
Chris Lu
89933c46d2 s3: skip hidden directories in /buckets folder 2021-08-01 12:28:08 -07:00
Chris Lu
1c7e404abe remove buckets folder option 
the related code still works for old deployments
 2021-08-01 12:23:16 -07:00
Chris Lu
cb1dbd3135 refactor 2021-08-01 11:53:46 -07:00
Chris Lu
1161c70dab Merge branch 'master' into add_remote_storage 2021-08-01 01:10:13 -07:00
Chris Lu
9cc84a910f volume: deletion can skip volume size checking 
fix https://github.com/chrislusf/seaweedfs/issues/2225
 2021-08-01 00:32:51 -07:00
Chris Lu
767edd3c08 rename 2021-07-31 23:52:09 -07:00
Chris Lu
9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu
f5a69a0e44 Merge branch 'master' into add_remote_storage 2021-07-31 09:39:48 -07:00
Chris Lu
6ba65c3382 customizable debug port 2021-07-31 09:18:41 -07:00
byunghwa.yun
5d4438a72f Fix typo 2021-07-31 22:23:52 +09:00
Chris Lu
1ff8285d82 debug from any server 2021-07-31 02:13:21 -07:00
Chris Lu
0d6f45cb46 add debug option to generate full goroutine stack dump 2021-07-31 02:00:01 -07:00
Chris Lu
a3290faf17 shell command to calculate size / number of volumes in a collection 
fix https://github.com/chrislusf/seaweedfs/issues/2224
 2021-07-30 15:49:46 -07:00
Chris Lu
c6f992b2a3 remove dead code 2021-07-30 15:18:01 -07:00
Chris Lu
899963ac20 remote storage location changed to struct 2021-07-29 02:08:55 -07:00
Chris Lu
c090d6bb25 add ReadRemote(), add read remote setup when filer starts 2021-07-28 22:43:12 -07:00
Chris Lu
4deac06da0 log fix 2021-07-28 22:25:04 -07:00
divanikus
5c6270a93a shell: ability to use wildcards for collections, all collections if ommited 2021-07-27 23:53:01 +03:00
Chris Lu
035b0bae29 refactor 2021-07-27 03:32:24 -07:00
Chris Lu
1752eeb538 remote.mount saves the mapping 2021-07-27 03:26:35 -07:00
Chris Lu
4b94b03d90 directory to remote storage mapping 2021-07-27 01:16:28 -07:00
Chris Lu
99b599aa8a remote.mount 2021-07-26 22:53:44 -07:00
Chris Lu
5dede5d38d 2.60 2021-07-25 22:09:09 -07:00
Chris Lu
35f70c51b0 refactor 2021-07-25 03:37:37 -07:00
Chris Lu
72eec84167 shell: add fs.mkdir 2021-07-25 02:37:39 -07:00
byunghwa.yun
2b28a818f1 Fix mysql sql for batch delete 2021-07-25 11:06:14 +09:00
Chris Lu
10464f47d0
Merge pull request #2217 from combineads/add_webp_format 2021-07-24 09:42:41 -07:00
byunghwa.yun
7374b5b473 Add webp extension for resizing 2021-07-24 14:32:31 +09:00
byunghwa.yun
2595f269d1 Add the webp image type 2021-07-24 14:26:40 +09:00
Chris Lu
ac28611817 snowflake sequencer need an unique id 
fix https://github.com/chrislusf/seaweedfs/issues/2213
 2021-07-23 20:54:03 -07:00
Chris Lu
5c14da0f1e filer.copy: fail early if assign request failed 
fix https://github.com/chrislusf/seaweedfs/issues/2216
 2021-07-23 20:01:43 -07:00
Chris Lu
4cc2165061 weed -h subcommand list is not sorted, makes discovery difficult 
fix https://github.com/chrislusf/seaweedfs/issues/2214
 2021-07-23 18:44:53 -07:00
Chris Lu
10fc478557 scaffold -config= should output to stdout 
fix https://github.com/chrislusf/seaweedfs/issues/2212
 2021-07-23 18:41:25 -07:00
Chris Lu
092932af8f shell: faster bucket delete 
avoid waiting for large object metadata
 2021-07-23 11:25:46 -07:00
Chris Lu
bdb632fa62
Merge pull request #2211 from qieqieplus/filer-bloom-filter 
add bloom filter for filer leveldb/rocksdb
 2021-07-22 23:35:54 -07:00
qieqieplus
2b46df13f8 add bloom filter for filer leveldb/rocksdb 2021-07-23 14:05:59 -06:00
Chris Lu
60f5c0a2f5 fix security alert on github.com/dgrijalva/jwt-go 
resolve https://github.com/chrislusf/seaweedfs/security/dependabot/go.mod/github.com%2Fdgrijalva%2Fjwt-go/open
 2021-07-22 10:22:09 -07:00
Chris Lu
182288f860 filer: fix mysql, postgres batch delete error 2021-07-22 08:23:20 -07:00
Chris Lu
84d91f143f mount: hide /etc folder also from mount 2021-07-22 00:40:16 -07:00
Chris Lu
7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu
cc32436d63
Merge pull request #2209 from combineads/fuse_option 
Fix error handling
 2021-07-21 12:37:20 -07:00
Chris Lu
67537ff05b
Merge pull request #2208 from combineads/add_rm_options 2021-07-21 11:08:43 -07:00
byunghwa.yun
e0c7708b03 Fix error handling 2021-07-21 22:48:04 +09:00
byunghwa.yun
41d559f54e Add fs rm options 2021-07-21 21:10:36 +09:00
Chris Lu
70effac0d3 configure and store remote configurations 2021-07-21 02:24:34 -07:00
Chris Lu
3138805b33
Merge pull request #2200 from combineads/add_fs_rm 
weed shell: fs.rm remove file or directory
 2021-07-21 00:35:54 -07:00
Chris Lu
a125c8fbe2
Merge pull request #2204 from nivekuil/tokenpolicy 
cassandra: use LocalQuorum for all queries
 2021-07-20 15:37:31 -07:00
nivekuil
b9a67d46c5 cassandra: use LocalQuorum for all queries 
This changes this filer store from eventual to strong consistency at the cost
of read performance.
 2021-07-20 14:47:39 -07:00
Chris Lu
54c8bc8673 Merge branch 'remote_overlay' 2021-07-19 23:18:35 -07:00
Chris Lu
7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu
93c37cfded
Update filechunk_manifest.go 2021-07-19 18:41:41 -07:00
byunghwa.yun
b23b307e08 Add fs rm 2021-07-20 08:49:28 +09:00
Chris Lu
b938df97a2 remove unused parameter 2021-07-19 02:59:12 -07:00
Chris Lu
450222dd64 add remote to filer.Entry and filer_pb entry, add RemoteConf 2021-07-19 02:47:27 -07:00
Chris Lu
8dc5def435 proto: add remote 2021-07-19 01:12:31 -07:00
Chris Lu
99155e4880 add -retry option for volume.fix.replication 
fix https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:13:46 -07:00
Chris Lu
2f209675ab Added -retry option for volumeServer.evacuate 
related to https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:08:21 -07:00
Chris Lu
fb7a1be1c4 refactor 2021-07-16 11:47:37 -07:00
nivekuil
2faf96f002 cassandra: Use LocalOne instead of One consistency 2021-07-16 04:29:46 -07:00
Chris Lu
4be5b4ff5c [volume.check.disk] skip errors and check next volumes 
fix https://github.com/chrislusf/seaweedfs/issues/2129
 2021-07-16 02:03:32 -07:00
Chris Lu
e95166d739 S 2021-07-15 17:43:38 -07:00
Chris Lu
a45bbc0b75 2.59 2021-07-15 15:52:22 -07:00
Chris Lu
f0042f62dd readable logs 2021-07-15 12:17:48 -07:00
Chris Lu
18c40686d9 s3: multipart upload miss data if file is chunked in 4MB 
fix https://github.com/chrislusf/seaweedfs/issues/2195
 2021-07-15 11:56:28 -07:00
Chris Lu
5a838dbe53 fix compilation 2021-07-13 11:40:21 -07:00
Chris Lu
6103649ffb shell: volume.check.disk adds retries in case the volumes are just moved 
related to https://github.com/chrislusf/seaweedfs/issues/2194
 2021-07-13 11:19:56 -07:00
Chris Lu
49c66e88a0 volume: change all writes to fsync during graceful stopping 
fix https://github.com/chrislusf/seaweedfs/issues/2193
 2021-07-13 01:29:57 -07:00
Chris Lu
01adc567aa shell: volume.fsck deletes a volume is the volume has only orphaned data. 
fix https://github.com/chrislusf/seaweedfs/issues/2190
 2021-07-12 11:22:00 -07:00
Chris Lu
297b41266b 2.58 2021-07-12 01:33:47 -07:00
Chris Lu
b194f91f47 add version to filer configuration response 2021-07-12 01:23:20 -07:00
bingoohuang
9e48bff8ee fix typo in fs.configure help message text 2021-07-12 13:51:21 +08:00
Chris Lu
d013d6d968 shell: volume.fsck "reallyDeleteFromVolume" should send padded file ids to delete 
fix https://github.com/chrislusf/seaweedfs/issues/2188
 2021-07-10 23:16:06 -07:00
Chris Lu
3d624d1e16 rename 2021-07-09 03:19:21 -07:00
Chris Lu
ecce300964 s3 config read via grpc 2021-07-09 02:48:03 -07:00
Chris Lu
da7bd62822 /etc files are stored inside metadata store 2021-07-09 02:33:14 -07:00
Chris Lu
a6d73e0a66
Merge pull request #2185 from bingoohuang/master 
show RemoteVolumes/EcVolumes only if it is not empty
 2021-07-06 00:28:07 -07:00
bingoohuang
ed57a55eae show RemoteVolumes/EcVolumes only if it is not empty 2021-07-06 15:20:18 +08:00
Chris Lu
44b50b2fdf another fix related to 64 bit alignment 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-05 16:30:43 -07:00
Chris Lu
141388367e
Merge pull request #2183 from bingoohuang/master 
extract embed html of master/volume/filer ui to separate files
 2021-07-05 03:22:26 -07:00
bingoohuang
44a2538f67 extract embed html of master/volume/filer ui to separate files 2021-07-05 18:09:44 +08:00
Chris Lu
e2da647fa3
Merge pull request #2182 from qieqieplus/fix-event-notification 
Subscribe meta data: sync empty notification with timestamp
 2021-07-05 02:00:53 -07:00
qieqieplus
233103f6b2 sync empty notification with timestamp 2021-07-05 16:01:16 +08:00
bingoohuang
5dbbe19c8b extract embed toml example to separate files 2021-07-05 11:16:49 +08:00
Chris Lu
5bcc77b46c volume: default readMode to proxy 2021-07-03 15:55:56 -07:00
Chris Lu
f5fa0b08fd 2.57 2021-07-03 15:10:57 -07:00
Chris Lu
a024254ad7 logging 2021-07-03 14:51:01 -07:00
Chris Lu
d39b2689a5 S3 authorization: StreamingSigned enforces access control 
fix https://github.com/chrislusf/seaweedfs/issues/2180
 2021-07-03 14:50:53 -07:00
Chris Lu
fa0dab6029 mount: rename also recursively move file handles 
related to https://github.com/chrislusf/seaweedfs/issues/2169
 2021-07-03 02:59:35 -07:00