mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-25 11:39:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,837 Commits 32 Branches 296 Tags 147 MiB
20d33ae025
Commit Graph

722 Commits

Author SHA1 Message Date
Chris Lu
d475c89fcc go fmt 2021-01-28 15:23:46 -08:00
Chris Lu
19295600f9 mount: change option name to volumeServerAccess, with publicUrl and filerProxy modes 2021-01-28 15:23:16 -08:00
Chris Lu
9a06c35da4 replicate: incremental sink only contains new and updated files 
address da08402ba2
 2021-01-28 02:39:22 -08:00
Chris Lu
da08402ba2 replicate: use creation time for local incremental file sink 
related to https://github.com/chrislusf/seaweedfs/pull/1762
 2021-01-28 02:17:41 -08:00
Chris Lu
822f1ade9d
Merge pull request #1762 from kmlebedev/backupsink 
replication to create time date directory
 2021-01-28 02:03:33 -08:00
Konstantin Lebedev
be1062b7fc rename in scaffold 2021-01-28 14:59:20 +05:00
Konstantin Lebedev
02fdc0a333 rename backup to local_incremental and use mtime 2021-01-28 14:56:13 +05:00
Chris Lu
5f72c388bc
Merge pull request #1764 from danlsgiga/handle-umask 
Use provided umask on mount
 2021-01-28 01:09:20 -08:00
Daniel Santos
d38613c49a match upstream 2021-01-27 20:56:41 -07:00
Konstantin Lebedev
6b54ff9912 replication to create time date directory 2021-01-27 15:01:33 +05:00
Chris Lu
0488c84bea adjust help message 2021-01-27 00:17:52 -08:00
Konstantin Lebedev
3634811408 replication to todays date directory 2021-01-27 12:45:58 +05:00
Chris Lu
5e07afb0f0
Merge pull request #1759 from kmlebedev/sink.local 
replication to local disk storage
 2021-01-26 22:31:54 -08:00
Daniel Santos
0c64d60fdb Fix log message with correct mode 2021-01-26 21:45:52 -07:00
Daniel Santos
0aaaa9b470 Fix var assignment 2021-01-26 21:06:08 -07:00
Daniel Santos
5a761bb7a4 Enforce umask correctly 2021-01-26 20:42:10 -07:00
Chris Lu
ad2a20c8a5 notification add ack and nack 2021-01-26 11:08:44 -08:00
Konstantin Lebedev
612b7975a1 replication to local disk storage 2021-01-26 22:50:25 +05:00
Chris Lu
3a1d3d3413 mount: properly invalidate kernel node cache entry 
fix https://github.com/chrislusf/seaweedfs/issues/1752
 2021-01-26 02:50:53 -08:00
Chris Lu
cc1f3907ff filer.sync: default filerProxy to true 2021-01-25 10:14:42 -08:00
Chris Lu
62ebb917e0 mount: avoid the need to specify different cache directory for different mount 
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/11
 2021-01-25 09:10:20 -08:00
Chris Lu
00707ec00f mount: outsideContainerClusterMode proxy through filer 
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
 2021-01-24 19:01:58 -08:00
Chris Lu
6ca10725b8 Revert "mount: when outside cluster network, use filer as proxy to access volume servers" 
This reverts commit 096e088d7b.
 2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b mount: when outside cluster network, use filer as proxy to access volume servers 2021-01-24 01:41:38 -08:00
Chris Lu
80b8692688 filer.sync: replicate outside of either cluster, only need to see filers 2021-01-24 00:01:44 -08:00
Chris Lu
024c186a48 filer.meta.tail: output json format for easier parsing 2021-01-21 22:39:25 -08:00
Chris Lu
84f05787f8 mount: limit background requests and congestion threshold 2021-01-20 23:30:14 -08:00
Chris Lu
1a3f859c58 filer store: postgres configuration adjust text 2021-01-19 18:12:07 -08:00
Chris Lu
b434f7e4e0 filer store: postgres adjust default config, add optioanl schema 2021-01-19 18:10:36 -08:00
Chris Lu
d5add83e85 filer store: add postgres2 2021-01-19 18:07:29 -08:00
Chris Lu
52a8f1470e filer store: add mysql2 2021-01-19 17:21:50 -08:00
Chris Lu
96354208c5 scaffold: add YugabyteDB 2021-01-19 12:35:14 -08:00
Chris Lu
f13c082583 add memsql to comments 2021-01-19 11:31:55 -08:00
LazyDBA247-Anyvision
8eed763b97 better mysql connection pool management 
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime)
to enable refresh of connections.
 2021-01-14 08:14:21 +02:00
Chris Lu
3fb2ed9093 filer.meta.tail: optionally submit metadata changes to ElasticSearch 2021-01-13 03:31:37 -08:00
Chris Lu
c5df2577f5 rename 2021-01-13 00:31:19 -08:00
Chris Lu
90c5077610 rename "weed watch" to "weed filer.meta.tail" 2021-01-12 18:48:01 -08:00
Chris Lu
0ef43a23a7 go fmt 2021-01-12 02:30:12 -08:00
Chris Lu
629c9962e7 filer: add leveldb3 
support dedicated leveldb instance for each bucket
 2021-01-12 02:29:44 -08:00
Chris Lu
cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu
2c7148cd55 redis cluster: do not enable route by latency by default 2021-01-11 02:43:53 -08:00
Chris Lu
6e12a3a490 refactoring 2021-01-11 02:08:26 -08:00
Chris Lu
62ee484d12 refactoring 2021-01-11 00:03:13 -08:00
Chris Lu
1efb51ba84 filer: change to saveToFilerLimit from cacheToFilerLimit 
short circuit saving small files to volume server
 2021-01-10 23:14:46 -08:00
Chris Lu
d3d3f2fb9b mount: default to 128 concurrent writers 2021-01-09 22:53:37 -08:00
Chris Lu
30efaa66c6 server: add option to disable master 2021-01-07 02:35:47 -08:00
Chris Lu
2b76854641 add "weed filer.cat" to read files directly from volume servers 2021-01-06 04:22:00 -08:00
Chris Lu
5f538c5f9d Merge branch 'add_rocksdb' 2021-01-03 00:48:56 -08:00
Chris Lu
a04d8927a8 add scaffold 2021-01-03 00:44:52 -08:00
Chris Lu
4250a3a675 upload: add read default collection from master 
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753576071
 2021-01-02 23:43:01 -08:00
Chris Lu
adef4ddc87 mount: load security keys 
fix https://github.com/chrislusf/seaweedfs/issues/1714
 2020-12-29 11:51:38 -08:00
Chris Lu
89977123b6 shell: add shell.toml support 
fix https://github.com/chrislusf/seaweedfs/issues/1664
 2020-12-28 15:07:18 -08:00
Chris Lu
90df52d001 s3: add command option "allowEmptyFolder" to filer and server 2020-12-27 21:09:06 -08:00
Chris Lu
561a2ff0bc s3: add option for "alllowEmptyFolder" 2020-12-27 10:50:27 -08:00
Chris Lu
75c6edba9e filer: hbase add enabled flag 2020-12-24 00:19:16 -08:00
Chris Lu
c3d1b3b5aa hook up 2020-12-23 21:49:01 -08:00
Chris Lu
1c7e1295dc Merge branch 'master' into support_ssd_volume 2020-12-22 17:44:52 -08:00
Chris Lu
a1009e8044 filer: add -defaultStoreDir so that filer.toml can be skipped 
fix https://github.com/chrislusf/seaweedfs/issues/1659
 2020-12-22 17:33:40 -08:00
Chris Lu
90d785a15f filer: redis, redis cluster, cassandra support super large directory 2020-12-22 02:26:05 -08:00
Chris Lu
41c0f3ad24 filer: support path-specific filer store 2020-12-19 01:27:09 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
23014b6810 Merge branch 'master' into support_ssd_volume 2020-12-16 08:26:51 -08:00
Chris Lu
3c7f9633eb show error a little bit more obviously 
fix https://github.com/chrislusf/seaweedfs/issues/1678
 2020-12-14 09:39:33 -08:00
Chris Lu
f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942 refactor 2020-12-13 22:29:52 -08:00
Chris Lu
8baba93fce rename parameter name to "disk" 2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6 rename parameter name to "disk" 2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
06bb7bf6c0 adding /usr/local/etc/seaweedfs for bsd style config search path 
related to https://reviews.freebsd.org/D27391
 2020-12-07 16:46:48 -08:00
Chris Lu
263eb29e9f filer: add option to cache small files to filer store 2020-12-04 22:39:43 -08:00
Chris Lu
6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu
983fd90010 marked as WIP 2020-11-26 15:08:52 -08:00
Chris Lu
b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu
ef3e068c51 deprecating "filer.options.buckets_fsync" in filer.toml 2020-11-16 16:56:49 -08:00
Chris Lu
827930e249
Merge pull request #1610 from kmlebedev/prefer_read_in_this_dc 
prefer to read from volumes in this data center
 2020-11-11 22:09:41 -08:00
Konstantin Lebedev
0880aff224 update dc flag desc 2020-11-12 10:07:52 +05:00
Konstantin Lebedev
fc7baef5bb fiil serverUrls sorted by data center 2020-11-12 02:13:33 +05:00
Chris Lu
745ee8d8f3 avoid unnecessary error message 
fix https://github.com/chrislusf/seaweedfs/issues/1611
 2020-11-11 13:00:05 -08:00
Chris Lu
8750cac090 move to util.RetryWaitTime 2020-11-01 02:36:43 -08:00
Chris Lu
de86945aeb go fmt 2020-10-31 16:45:38 -07:00
Chris Lu
8826601be1 mount: optional limit for the number of concurrent writers 2020-10-30 21:22:20 -07:00
Chris Lu
7ecbb4b3c9 clean up 2020-10-30 10:46:31 -07:00
Kenny
9cc6575ab5
Update filer_copy.go 
修改 weed filer.copy 上传目录时文件名变子目录的问题
 2020-10-30 18:16:49 +08:00
Chris Lu
47047516d6 Revert "increase default volume file size limit to 1024" 
This reverts commit 0983060a

I am confused why this was changed to 1024 by myself. 1GB should be too large for most cases.
 2020-10-29 15:46:29 -07:00
Chris Lu
c6d9974fe7 server: Add option to enable/disable volume server 
fix https://github.com/chrislusf/seaweedfs/issues/1572
 2020-10-29 00:24:18 -07:00
Chris Lu
e219c57849 passing full path when assign volume locations 2020-10-25 15:46:29 -07:00
Chris Lu
f375b93aef renaming 2020-10-25 15:32:43 -07:00
Chris Lu
63b0fb54f7 minor 2020-10-25 14:15:53 -07:00
Chris Lu
147d4d23b6 filer.copy: automatically use the right bucket 2020-10-25 11:21:33 -07:00
Chris Lu
da36abf033 go fmt 2020-10-24 20:12:04 -07:00
Chris Lu
b7b1eb51b1 default to 2 for better performance in common computer hardware 2020-10-24 20:08:26 -07:00
Chris Lu
1b17f71939 adjust election timeout to 10 seconds 2020-10-23 23:06:44 -07:00
Chris Lu
0cea84e7ab make it easy with setting up multiple masters in a clcuster 2020-10-23 18:18:46 -07:00
Konstantin Lebedev
a9a7005687 set desc of option 2020-10-22 11:23:00 +05:00
Chris Lu
6c92fa349d filer: support default rack 
fix https://github.com/chrislusf/seaweedfs/issues/1546
 2020-10-20 17:41:39 -07:00
Chris Lu
ee1fc6558a refactor 2020-10-16 11:02:44 -07:00
Chris Lu
3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu
7704469d29 mount: outsideContainerClusterMode changed to use volume server publicUrl 2020-10-11 20:42:15 -07:00
Chris Lu
d155f907c2 mount: configurable read wait time 2020-10-10 20:09:43 -07:00
Chris Lu
78859e35fc s3: adjust config examples 2020-10-08 10:12:09 -07:00
Chris Lu
a8624c2e4f read from alternative replica 
related to https://github.com/chrislusf/seaweedfs/issues/1512
 2020-10-07 22:49:04 -07:00
Chris Lu
da4edf3651 master: check peers for existing leader before starting a leader election 
fix https://github.com/chrislusf/seaweedfs/issues/1509
 2020-10-07 01:25:39 -07:00
Chris Lu
c543762e23 volume: auto expands -max setting 
I have tried to run weed volume but got errors...
cmd:
weed volume -max=0 -mserver="127.0.0.1:9333" -port=8080 -dir="/var/www/d1/sfs,/var/www/d2/sfs,/var/www/d3/sfs"
log:
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d1/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d2/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d3/sfs Permission: -rwxr-xr-x
F1006 15:09:26 17040 volume.go:142] 3 directories by -dir, but only 1 max is set by -max
What am I doing wrong? Thx.
 2020-10-06 09:05:30 -07:00
Chris Lu
2c20ef72ae fix starting problem 2020-10-03 13:56:38 -07:00
Chris Lu
d77e3c96e9 FUSE: use the mount directory permissions 2020-10-03 13:37:35 -07:00
Устюжанин Антон Александрович
6fbfe73683 fix: resumeState 2020-10-03 22:16:47 +05:00
Устюжанин Антон Александрович
dc31b19469 fix: restore raft state 2020-10-03 14:03:41 +05:00
Устюжанин Антон Александрович
8c82fb7e5f fix: restore raft state 2020-10-02 23:01:20 +05:00
Chris Lu
79ab10e300 adjust help message 2020-09-30 09:32:00 -07:00
Chris Lu
55cb68c09c filer: replication follows master config if not specified 2020-09-30 09:15:55 -07:00
Chris Lu
e6552b5e1e filer: able to start s3 together 2020-09-27 23:00:43 -07:00
Chris Lu
8516517c48 filer store: Cassandra supports username/password 2020-09-24 21:31:06 -07:00
Chris Lu
043b063136
Merge pull request #1482 from hilimd/master 
Fix: s3 delete object
 2020-09-24 18:21:34 -07:00
limd
48c578410f mount: rollback default value 2020-09-25 09:18:52 +08:00
Chris Lu
4856bce0ee adjust for metrics port 2020-09-24 10:21:23 -07:00
Chris Lu
d2d3aec3e1 consolidate to one metricsPort in "weed server" mode 2020-09-24 09:55:02 -07:00
Chris Lu
2e9099369e
Merge pull request #1489 from kmlebedev/promhttp 
Promhttp
 2020-09-24 09:50:48 -07:00
Konstantin Lebedev
98e9de6e11 fix style 2020-09-24 17:48:39 +05:00
Konstantin Lebedev
324e44d4b3 add start metrics server 2020-09-24 17:45:39 +05:00
limd
cbca14edc5 mount: fix k8s pvc and os mount directory permission bug 2020-09-24 18:07:16 +08:00
limd
59e91e9c7e mount: fix k8s pvc mount directory permission 2020-09-24 10:25:45 +08:00
limd
9b21ec27e4 mount: auto created dir set corrct umask fix bug 2020-09-23 18:35:37 +08:00
Chris Lu
f7a0ccb595 mount: auto created dir set corrct umask 2020-09-23 02:31:19 -07:00
James Hartig
658fc2e5b6 Allow option to enable volume pprof on server 2020-09-21 22:43:10 -04:00
Chris Lu
d013d09a9b adjust logging 2020-09-20 15:38:59 -07:00
Chris Lu
2c21eb1971 volume: get metrics configuration from master 
fix https://github.com/chrislusf/seaweedfs/issues/1354
 2020-09-19 00:03:00 -07:00
Chris Lu
6544e60bea s3 add metrics 
empty for now
 2020-09-17 06:56:15 -07:00
Chris Lu
feca07bf96 Filer: ip bind 
fix https://github.com/chrislusf/seaweedfs/issues/1470
 2020-09-16 10:26:08 -07:00
Chris Lu
684a875876 update help message 2020-09-15 00:40:38 -07:00
Chris Lu
103fafe00b weed export: print out [start,stop) content range 2020-09-14 22:57:23 -07:00
Chris Lu
5d6753fb98 shell: add volumeServer.leave command 2020-09-13 21:25:51 -07:00
Chris Lu
1af95c5b76 refactoring 2020-09-13 12:41:26 -07:00
Chris Lu
1a7afe7e6a adjust default value 
related to https://github.com/chrislusf/seaweedfs/issues/1453
 2020-09-12 12:46:42 -07:00
Chris Lu
f2723c1bc8 do not idx file format 
revert c9ab8d05fa
 2020-09-12 12:42:36 -07:00
Chris Lu
446e476a11 go fmt 2020-09-12 04:08:03 -07:00
Chris Lu
cd9b89ba55 reduce default wait time to 10s before shutting down 2020-09-12 04:05:33 -07:00
Chris Lu
1b8094ef75 weed export: export deleted files 2020-09-11 02:05:14 -07:00
Chris Lu
c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
Chris Lu
0d9b858cfb
Merge pull request #1455 from taozix/master 
add more basic elastic options.
 2020-09-10 01:31:17 -07:00
ruitao.liu
719dc43af1 modify elastic urls from string to array. 2020-09-10 16:24:09 +08:00
ruitao.liu
3f7fbfddca add more basic elastic options. 2020-09-10 14:22:07 +08:00
Chris Lu
47b3f932e4 watch: adjust output format 2020-09-09 22:34:48 -07:00
Chris Lu
7f69acd1f2 sync pprof 2020-09-09 11:33:52 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
ruitao.liu
83080b5e03 ES backended filer support kv ops. 2020-09-04 15:40:13 +08:00
Chris Lu
46f65a84a6 filer: elastic7 adjust default value 2020-09-03 20:12:38 -07:00
ruitao.liu
a93d27d1e8 new filer option to es v7. 2020-09-03 16:34:58 +08:00