Chris Lu
f8446b42ab
this can compile now!!!
2021-02-16 02:47:02 -08:00
LazyDBA247-Anyvision
7f458d5e78
better postgres connection pool management
...
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime )
to enable refresh of stale connections.
2021-02-15 07:45:09 +02:00
Chris Lu
0bc3a1f9e8
disk type only supports hdd and ssd, not ready for random tags yet
2021-02-14 11:38:43 -08:00
LazyDBA247-Anyvision
51b4963e2e
postgres2 & memsql2
...
add escape (quote identifiers) for the dynamic sql
so tables (collections) with special characters will work.
2021-02-14 13:14:36 +02:00
Chris Lu
ef76365ec2
adjust help message
2021-02-13 15:47:08 -08:00
Chris Lu
4bd8a692d8
disk type can be generic tags
2021-02-13 13:50:14 -08:00
Chris Lu
7d9dc3c6a2
use fasthttp lib to read
2021-02-12 05:59:52 -08:00
Chris Lu
821c46edf1
Merge branch 'master' into support_ssd_volume
2021-02-09 11:37:07 -08:00
bingoohuang
7256902fb0
fix typo offset.ToAcutalOffset to offset.ToActualOffset
2021-02-07 12:11:51 +08:00
Chris Lu
d67ccb66c0
webdav: can start together with "weed server" or "weed filer"
2021-01-31 22:16:52 -08:00
Chris Lu
d475c89fcc
go fmt
2021-01-28 15:23:46 -08:00
Chris Lu
19295600f9
mount: change option name to volumeServerAccess, with publicUrl and filerProxy modes
2021-01-28 15:23:16 -08:00
Chris Lu
9a06c35da4
replicate: incremental sink only contains new and updated files
...
address da08402ba2
2021-01-28 02:39:22 -08:00
Chris Lu
da08402ba2
replicate: use creation time for local incremental file sink
...
related to https://github.com/chrislusf/seaweedfs/pull/1762
2021-01-28 02:17:41 -08:00
Chris Lu
822f1ade9d
Merge pull request #1762 from kmlebedev/backupsink
...
replication to create time date directory
2021-01-28 02:03:33 -08:00
Konstantin Lebedev
be1062b7fc
rename in scaffold
2021-01-28 14:59:20 +05:00
Konstantin Lebedev
02fdc0a333
rename backup to local_incremental and use mtime
2021-01-28 14:56:13 +05:00
Chris Lu
5f72c388bc
Merge pull request #1764 from danlsgiga/handle-umask
...
Use provided umask on mount
2021-01-28 01:09:20 -08:00
Daniel Santos
d38613c49a
match upstream
2021-01-27 20:56:41 -07:00
Konstantin Lebedev
6b54ff9912
replication to create time date directory
2021-01-27 15:01:33 +05:00
Chris Lu
0488c84bea
adjust help message
2021-01-27 00:17:52 -08:00
Konstantin Lebedev
3634811408
replication to todays date directory
2021-01-27 12:45:58 +05:00
Chris Lu
5e07afb0f0
Merge pull request #1759 from kmlebedev/sink.local
...
replication to local disk storage
2021-01-26 22:31:54 -08:00
Daniel Santos
0c64d60fdb
Fix log message with correct mode
2021-01-26 21:45:52 -07:00
Daniel Santos
0aaaa9b470
Fix var assignment
2021-01-26 21:06:08 -07:00
Daniel Santos
5a761bb7a4
Enforce umask correctly
2021-01-26 20:42:10 -07:00
Chris Lu
ad2a20c8a5
notification add ack and nack
2021-01-26 11:08:44 -08:00
Konstantin Lebedev
612b7975a1
replication to local disk storage
2021-01-26 22:50:25 +05:00
Chris Lu
3a1d3d3413
mount: properly invalidate kernel node cache entry
...
fix https://github.com/chrislusf/seaweedfs/issues/1752
2021-01-26 02:50:53 -08:00
Chris Lu
cc1f3907ff
filer.sync: default filerProxy to true
2021-01-25 10:14:42 -08:00
Chris Lu
62ebb917e0
mount: avoid the need to specify different cache directory for different mount
...
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/11
2021-01-25 09:10:20 -08:00
Chris Lu
00707ec00f
mount: outsideContainerClusterMode proxy through filer
...
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
6ca10725b8
Revert "mount: when outside cluster network, use filer as proxy to access volume servers"
...
This reverts commit 096e088d7b
.
2021-01-24 03:15:19 -08:00
Chris Lu
096e088d7b
mount: when outside cluster network, use filer as proxy to access volume servers
2021-01-24 01:41:38 -08:00
Chris Lu
80b8692688
filer.sync: replicate outside of either cluster, only need to see filers
2021-01-24 00:01:44 -08:00
Chris Lu
024c186a48
filer.meta.tail: output json format for easier parsing
2021-01-21 22:39:25 -08:00
Chris Lu
84f05787f8
mount: limit background requests and congestion threshold
2021-01-20 23:30:14 -08:00
Chris Lu
1a3f859c58
filer store: postgres configuration adjust text
2021-01-19 18:12:07 -08:00
Chris Lu
b434f7e4e0
filer store: postgres adjust default config, add optioanl schema
2021-01-19 18:10:36 -08:00
Chris Lu
d5add83e85
filer store: add postgres2
2021-01-19 18:07:29 -08:00
Chris Lu
52a8f1470e
filer store: add mysql2
2021-01-19 17:21:50 -08:00
Chris Lu
96354208c5
scaffold: add YugabyteDB
2021-01-19 12:35:14 -08:00
Chris Lu
f13c082583
add memsql to comments
2021-01-19 11:31:55 -08:00
LazyDBA247-Anyvision
8eed763b97
better mysql connection pool management
...
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime )
to enable refresh of connections.
2021-01-14 08:14:21 +02:00
Chris Lu
3fb2ed9093
filer.meta.tail: optionally submit metadata changes to ElasticSearch
2021-01-13 03:31:37 -08:00
Chris Lu
c5df2577f5
rename
2021-01-13 00:31:19 -08:00
Chris Lu
90c5077610
rename "weed watch" to "weed filer.meta.tail"
2021-01-12 18:48:01 -08:00
Chris Lu
0ef43a23a7
go fmt
2021-01-12 02:30:12 -08:00
Chris Lu
629c9962e7
filer: add leveldb3
...
support dedicated leveldb instance for each bucket
2021-01-12 02:29:44 -08:00
Chris Lu
cfb9342a15
avoid concurrent map updates to viper
2021-01-12 02:28:13 -08:00
Chris Lu
2c7148cd55
redis cluster: do not enable route by latency by default
2021-01-11 02:43:53 -08:00
Chris Lu
6e12a3a490
refactoring
2021-01-11 02:08:26 -08:00
Chris Lu
62ee484d12
refactoring
2021-01-11 00:03:13 -08:00
Chris Lu
1efb51ba84
filer: change to saveToFilerLimit from cacheToFilerLimit
...
short circuit saving small files to volume server
2021-01-10 23:14:46 -08:00
Chris Lu
d3d3f2fb9b
mount: default to 128 concurrent writers
2021-01-09 22:53:37 -08:00
Chris Lu
30efaa66c6
server: add option to disable master
2021-01-07 02:35:47 -08:00
Chris Lu
2b76854641
add "weed filer.cat" to read files directly from volume servers
2021-01-06 04:22:00 -08:00
Chris Lu
5f538c5f9d
Merge branch 'add_rocksdb'
2021-01-03 00:48:56 -08:00
Chris Lu
a04d8927a8
add scaffold
2021-01-03 00:44:52 -08:00
Chris Lu
4250a3a675
upload: add read default collection from master
...
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753576071
2021-01-02 23:43:01 -08:00
Chris Lu
adef4ddc87
mount: load security keys
...
fix https://github.com/chrislusf/seaweedfs/issues/1714
2020-12-29 11:51:38 -08:00
Chris Lu
89977123b6
shell: add shell.toml support
...
fix https://github.com/chrislusf/seaweedfs/issues/1664
2020-12-28 15:07:18 -08:00
Chris Lu
90df52d001
s3: add command option "allowEmptyFolder" to filer and server
2020-12-27 21:09:06 -08:00
Chris Lu
561a2ff0bc
s3: add option for "alllowEmptyFolder"
2020-12-27 10:50:27 -08:00
Chris Lu
75c6edba9e
filer: hbase add enabled flag
2020-12-24 00:19:16 -08:00
Chris Lu
c3d1b3b5aa
hook up
2020-12-23 21:49:01 -08:00
Chris Lu
1c7e1295dc
Merge branch 'master' into support_ssd_volume
2020-12-22 17:44:52 -08:00
Chris Lu
a1009e8044
filer: add -defaultStoreDir so that filer.toml can be skipped
...
fix https://github.com/chrislusf/seaweedfs/issues/1659
2020-12-22 17:33:40 -08:00
Chris Lu
90d785a15f
filer: redis, redis cluster, cassandra support super large directory
2020-12-22 02:26:05 -08:00
Chris Lu
41c0f3ad24
filer: support path-specific filer store
2020-12-19 01:27:09 -08:00
Chris Lu
1bf22c0b5b
go fmt
2020-12-16 09:14:05 -08:00
Chris Lu
23014b6810
Merge branch 'master' into support_ssd_volume
2020-12-16 08:26:51 -08:00
Chris Lu
3c7f9633eb
show error a little bit more obviously
...
fix https://github.com/chrislusf/seaweedfs/issues/1678
2020-12-14 09:39:33 -08:00
Chris Lu
f6a419c26c
disk type configurable for each folder
2020-12-13 22:49:56 -08:00
Chris Lu
7c52a35942
refactor
2020-12-13 22:29:52 -08:00
Chris Lu
8baba93fce
rename parameter name to "disk"
2020-12-13 12:06:21 -08:00
Chris Lu
51eadaf2b6
rename parameter name to "disk"
2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2
rename from volumeType to diskType
2020-12-13 11:59:32 -08:00
Chris Lu
d156c74ec0
volume server set volume type and heartbeat to the master
2020-12-13 03:11:24 -08:00
Chris Lu
e9cd798bd3
adding volume type
2020-12-13 00:58:58 -08:00
Chris Lu
06bb7bf6c0
adding /usr/local/etc/seaweedfs for bsd style config search path
...
related to https://reviews.freebsd.org/D27391
2020-12-07 16:46:48 -08:00
Chris Lu
263eb29e9f
filer: add option to cache small files to filer store
2020-12-04 22:39:43 -08:00
Chris Lu
6d30b21b10
volume: add "-dir.idx" option for separate index storage
...
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
983fd90010
marked as WIP
2020-11-26 15:08:52 -08:00
Chris Lu
b11449f955
pass in dir.idx parameter to DiskLocation
2020-11-26 14:59:03 -08:00
Chris Lu
ef3e068c51
deprecating "filer.options.buckets_fsync" in filer.toml
2020-11-16 16:56:49 -08:00
Chris Lu
827930e249
Merge pull request #1610 from kmlebedev/prefer_read_in_this_dc
...
prefer to read from volumes in this data center
2020-11-11 22:09:41 -08:00
Konstantin Lebedev
0880aff224
update dc flag desc
2020-11-12 10:07:52 +05:00
Konstantin Lebedev
fc7baef5bb
fiil serverUrls sorted by data center
2020-11-12 02:13:33 +05:00
Chris Lu
745ee8d8f3
avoid unnecessary error message
...
fix https://github.com/chrislusf/seaweedfs/issues/1611
2020-11-11 13:00:05 -08:00
Chris Lu
8750cac090
move to util.RetryWaitTime
2020-11-01 02:36:43 -08:00
Chris Lu
de86945aeb
go fmt
2020-10-31 16:45:38 -07:00
Chris Lu
8826601be1
mount: optional limit for the number of concurrent writers
2020-10-30 21:22:20 -07:00
Chris Lu
7ecbb4b3c9
clean up
2020-10-30 10:46:31 -07:00
Kenny
9cc6575ab5
Update filer_copy.go
...
修改 weed filer.copy 上传目录时文件名变子目录的问题
2020-10-30 18:16:49 +08:00
Chris Lu
47047516d6
Revert "increase default volume file size limit to 1024"
...
This reverts commit 0983060a
I am confused why this was changed to 1024 by myself. 1GB should be too large for most cases.
2020-10-29 15:46:29 -07:00
Chris Lu
c6d9974fe7
server: Add option to enable/disable volume server
...
fix https://github.com/chrislusf/seaweedfs/issues/1572
2020-10-29 00:24:18 -07:00
Chris Lu
e219c57849
passing full path when assign volume locations
2020-10-25 15:46:29 -07:00
Chris Lu
f375b93aef
renaming
2020-10-25 15:32:43 -07:00
Chris Lu
63b0fb54f7
minor
2020-10-25 14:15:53 -07:00
Chris Lu
147d4d23b6
filer.copy: automatically use the right bucket
2020-10-25 11:21:33 -07:00
Chris Lu
da36abf033
go fmt
2020-10-24 20:12:04 -07:00
Chris Lu
b7b1eb51b1
default to 2 for better performance in common computer hardware
2020-10-24 20:08:26 -07:00
Chris Lu
1b17f71939
adjust election timeout to 10 seconds
2020-10-23 23:06:44 -07:00
Chris Lu
0cea84e7ab
make it easy with setting up multiple masters in a clcuster
2020-10-23 18:18:46 -07:00
Konstantin Lebedev
a9a7005687
set desc of option
2020-10-22 11:23:00 +05:00
Chris Lu
6c92fa349d
filer: support default rack
...
fix https://github.com/chrislusf/seaweedfs/issues/1546
2020-10-20 17:41:39 -07:00
Chris Lu
ee1fc6558a
refactor
2020-10-16 11:02:44 -07:00
Chris Lu
3f7d1d1bf1
Only wait on retryable requests
2020-10-13 00:29:46 -07:00
Chris Lu
7704469d29
mount: outsideContainerClusterMode changed to use volume server publicUrl
2020-10-11 20:42:15 -07:00
Chris Lu
d155f907c2
mount: configurable read wait time
2020-10-10 20:09:43 -07:00
Chris Lu
78859e35fc
s3: adjust config examples
2020-10-08 10:12:09 -07:00
Chris Lu
a8624c2e4f
read from alternative replica
...
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu
da4edf3651
master: check peers for existing leader before starting a leader election
...
fix https://github.com/chrislusf/seaweedfs/issues/1509
2020-10-07 01:25:39 -07:00
Chris Lu
c543762e23
volume: auto expands -max setting
...
I have tried to run weed volume but got errors...
cmd:
weed volume -max=0 -mserver="127.0.0.1:9333" -port=8080 -dir="/var/www/d1/sfs,/var/www/d2/sfs,/var/www/d3/sfs"
log:
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d1/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d2/sfs Permission: -rwxr-xr-x
I1006 15:09:26 17040 file_util.go:23] Folder /var/www/d3/sfs Permission: -rwxr-xr-x
F1006 15:09:26 17040 volume.go:142] 3 directories by -dir, but only 1 max is set by -max
What am I doing wrong? Thx.
2020-10-06 09:05:30 -07:00
Chris Lu
2c20ef72ae
fix starting problem
2020-10-03 13:56:38 -07:00
Chris Lu
d77e3c96e9
FUSE: use the mount directory permissions
2020-10-03 13:37:35 -07:00
Устюжанин Антон Александрович
6fbfe73683
fix: resumeState
2020-10-03 22:16:47 +05:00
Устюжанин Антон Александрович
dc31b19469
fix: restore raft state
2020-10-03 14:03:41 +05:00
Устюжанин Антон Александрович
8c82fb7e5f
fix: restore raft state
2020-10-02 23:01:20 +05:00
Chris Lu
79ab10e300
adjust help message
2020-09-30 09:32:00 -07:00
Chris Lu
55cb68c09c
filer: replication follows master config if not specified
2020-09-30 09:15:55 -07:00
Chris Lu
e6552b5e1e
filer: able to start s3 together
2020-09-27 23:00:43 -07:00
Chris Lu
8516517c48
filer store: Cassandra supports username/password
2020-09-24 21:31:06 -07:00
Chris Lu
043b063136
Merge pull request #1482 from hilimd/master
...
Fix: s3 delete object
2020-09-24 18:21:34 -07:00
limd
48c578410f
mount: rollback default value
2020-09-25 09:18:52 +08:00
Chris Lu
4856bce0ee
adjust for metrics port
2020-09-24 10:21:23 -07:00
Chris Lu
d2d3aec3e1
consolidate to one metricsPort in "weed server" mode
2020-09-24 09:55:02 -07:00
Chris Lu
2e9099369e
Merge pull request #1489 from kmlebedev/promhttp
...
Promhttp
2020-09-24 09:50:48 -07:00
Konstantin Lebedev
98e9de6e11
fix style
2020-09-24 17:48:39 +05:00
Konstantin Lebedev
324e44d4b3
add start metrics server
2020-09-24 17:45:39 +05:00
limd
cbca14edc5
mount: fix k8s pvc and os mount directory permission bug
2020-09-24 18:07:16 +08:00
limd
59e91e9c7e
mount: fix k8s pvc mount directory permission
2020-09-24 10:25:45 +08:00
limd
9b21ec27e4
mount: auto created dir set corrct umask fix bug
2020-09-23 18:35:37 +08:00
Chris Lu
f7a0ccb595
mount: auto created dir set corrct umask
2020-09-23 02:31:19 -07:00
James Hartig
658fc2e5b6
Allow option to enable volume pprof on server
2020-09-21 22:43:10 -04:00
Chris Lu
d013d09a9b
adjust logging
2020-09-20 15:38:59 -07:00
Chris Lu
2c21eb1971
volume: get metrics configuration from master
...
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-19 00:03:00 -07:00
Chris Lu
6544e60bea
s3 add metrics
...
empty for now
2020-09-17 06:56:15 -07:00
Chris Lu
feca07bf96
Filer: ip bind
...
fix https://github.com/chrislusf/seaweedfs/issues/1470
2020-09-16 10:26:08 -07:00
Chris Lu
684a875876
update help message
2020-09-15 00:40:38 -07:00
Chris Lu
103fafe00b
weed export: print out [start,stop) content range
2020-09-14 22:57:23 -07:00
Chris Lu
5d6753fb98
shell: add volumeServer.leave command
2020-09-13 21:25:51 -07:00
Chris Lu
1af95c5b76
refactoring
2020-09-13 12:41:26 -07:00
Chris Lu
1a7afe7e6a
adjust default value
...
related to https://github.com/chrislusf/seaweedfs/issues/1453
2020-09-12 12:46:42 -07:00
Chris Lu
f2723c1bc8
do not idx file format
...
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
446e476a11
go fmt
2020-09-12 04:08:03 -07:00
Chris Lu
cd9b89ba55
reduce default wait time to 10s before shutting down
2020-09-12 04:05:33 -07:00
Chris Lu
1b8094ef75
weed export: export deleted files
2020-09-11 02:05:14 -07:00
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
2020-09-10 14:42:52 -07:00