chrislu
87ccef72db
cluster.ps can list brokers
2022-07-28 23:22:07 -07:00
chrislu
21b6b07dd8
renaming
2022-07-28 23:22:06 -07:00
chrislu
8ca7d1ef17
rename to SeaweedMQ
2022-07-28 23:22:06 -07:00
chrislu
ca836568ac
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-07-27 12:12:43 -07:00
chrislu
3828b8ce87
"github.com/chrislusf/raft" => "github.com/seaweedfs/raft"
2022-07-27 12:12:40 -07:00
Konstantin Lebedev
7e09a548a6
exclude directories to sync on filer
2022-07-27 19:22:57 +05:00
chrislu
64f3d6fb6e
metadata subscription uses client epoch
2022-07-23 10:50:28 -07:00
duanhongyi
1ceab96aba
filer tikv support tls
2022-07-08 14:23:06 +08:00
chrislu
b7de5c6c43
shell: remove unused filer parameter from shell.toml
2022-07-07 15:01:23 -07:00
Konstantin Lebedev
bcbdc4cb37
use const multipart uploads folder
...
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
guol-fnst
74c9f39d48
remove hdfs, add tikv for full version update
2022-06-28 12:35:07 +08:00
chrislu
4a65159250
fix reading time
2022-06-27 12:40:47 -07:00
yulai.li
af23e63e3f
Improve filer command help, add supported filer store list
2022-06-27 12:09:16 +08:00
yulai.li
46e0b629e5
Update tikv client version and add one PC support
2022-06-26 22:43:37 +08:00
chrislu
492da3dbce
master: put metadata under instance specific folder
2022-06-20 19:04:49 -07:00
chrislu
2295766e2c
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-20 14:16:18 -07:00
chrislu
1d0c53ea56
remote storage: stop supporting hdfs as a remote storage
2022-06-20 14:15:59 -07:00
Stephan
1eb7826909
Fix link to osxfuse github page
2022-06-20 22:36:07 +02:00
Chris Lu
d28f8bb88f
Merge pull request #3192 from guol-fnst/weed_update
...
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-16 22:58:29 -07:00
guol-fnst
c7c20881f5
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-17 12:47:04 +08:00
Chris Lu
5f8d6b5f13
Merge pull request #3184 from guol-fnst/weed_update
...
Weed update
2022-06-16 10:02:45 -07:00
guol-fnst
cd078471c4
fix path bug on windows
...
fix usage
2022-06-16 17:24:56 +08:00
guol-fnst
995e221e97
remove -output, add -dir,-name for more flexibility
2022-06-16 16:07:32 +08:00
Chris Lu
82f3bcc65e
Merge pull request #3081 from paochiang/volume_upload_limit_fix
...
exclude replication from the concurrentUploadLimitMB
2022-06-15 23:16:44 -07:00
guol-fnst
4bc8792643
support specific version of weed
2022-06-16 11:40:23 +08:00
guol-fnst
1dbb925d29
remove unused function
...
remove log when weed is up to date
add check whether weed is full version
2022-06-16 11:40:20 +08:00
Chris Lu
4eeeb5f50d
Merge pull request #3180 from blacktear23/issue-3149
2022-06-15 00:55:20 -07:00
yulai.li
358c0c0b37
Fix: provide filer.ui.deleteDir options for server command
2022-06-15 15:53:31 +08:00
Guo Lei
3df84cabf6
Merge branch 'chrislusf:master' into weed_update
2022-06-15 15:28:40 +08:00
guol-fnst
695e8c7555
fix minor bug
2022-06-15 15:26:04 +08:00
zhihao.qu
4d0d1848c6
fix(filer.sync): modify clientName format : from -> to
2022-06-15 13:33:20 +08:00
yulai.li
b90d213a87
Change ui.deleteDir default to true
2022-06-15 11:37:37 +08:00
zhihao.qu
42d04c581b
feat(filer.sync): add metricsServer in filer.sync.
...
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
2022-06-15 11:33:18 +08:00
yulai.li
28cda854eb
Add filer command line parameter to let Filer UI show/hide directory delete button
2022-06-15 00:30:49 +08:00
zhihao.qu
14d82c3dea
feat(filer.sync): add offset to path.
2022-06-14 19:46:02 +08:00
guol-fnst
78fc9cdfa0
add "copied from https://github.com/restic/restic/tree/master/internal/selfupdate "
2022-06-14 19:10:54 +08:00
guol-fnst
322dbcf372
add update feature
2022-06-14 19:01:57 +08:00
zhihao.qu
cd5cca36a4
feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges
2022-06-09 10:53:19 +08:00
chrislu
8a1166c021
auto remove previous leftover socket
2022-06-08 00:32:09 -07:00
chrislu
633a4aed18
filer: fix customizable local socket file name
...
fix https://github.com/chrislusf/seaweedfs/issues/3147
2022-06-08 00:15:28 -07:00
chrislu
78be3b0f6c
fix compilation
2022-06-06 10:09:21 -07:00
chrislu
4fd5f96598
filer: remove replication, collection, disk_type info from entry metadata
...
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
ningfd
f32142f6f5
add disableXAttr in mount option
2022-06-06 14:09:01 +08:00
creeew
02ae102731
fix filer.sync missing source srv uploaded files to target when target down
2022-06-02 01:28:47 +08:00
chrislu
490e0850bf
filer.meta.tail add one example of using untilTimeAgo
2022-05-30 20:24:59 -07:00
chrislu
968ca95b49
filer.meta.tail: support untilTimeAgo for a range
2022-05-30 16:16:23 -07:00
chrislu
a2b101a737
subscribe metadata between a range
2022-05-30 15:04:19 -07:00
liubaojiang
076e48a676
add inflight upload data wait timeout
2022-05-21 10:38:08 +08:00
Chris Lu
8efe55f981
Merge pull request #3059 from guol-fnst/avoid_dup_vol
...
avoid loading duplicated volume directory
2022-05-17 23:48:57 -07:00
guol-fnst
076595fbdd
just exit in case of duplicated volume directories were loaded
2022-05-17 15:41:49 +08:00
Nat Makarevitch
b72f0634d7
minor (typos...), done while reading around
2022-05-16 22:11:33 +08:00
chrislu
29198720f2
s3: add grpc server to accept configuration changes
2022-05-15 00:43:37 -07:00
chrislu
735038b2c1
backup do not need to use preallocation
...
fix https://github.com/chrislusf/seaweedfs/issues/3044
2022-05-13 13:46:52 -07:00
Konstantin Lebedev
f127b326bf
add options to scaffold
2022-05-03 22:54:31 +05:00
Konstantin Lebedev
d8925b4e83
Merge branch 'new_master' into ydb
...
# Conflicts:
# go.mod
# go.sum
2022-05-03 00:13:57 +05:00
chrislu
94635e9b5c
filer: add filer group
2022-05-01 21:59:16 -07:00
Konstantin Lebedev
50c4f62ed4
ydb-go-sdk move to v3
2022-05-02 02:07:47 +05:00
chrislu
192983b464
s3 backend support customizing storage class
2022-04-30 17:36:40 -07:00
chrislu
3885374edf
conditionally build elastic, gocdk to reduce binary size
2022-04-21 01:10:46 -07:00
chrislu
6994e88192
fix typo
2022-04-21 00:02:18 -07:00
a
1d6a9e66b6
Merge branch 'master' into a
2022-04-20 14:01:42 -07:00
chrislu
a69f0b62c2
start filer local socket only on non windows
...
fix https://github.com/chrislusf/seaweedfs/issues/2930
2022-04-18 00:08:31 -07:00
leyou240
89eb87c1d1
Merge branch 'master' into slices.SortFunc
2022-04-18 10:39:29 +08:00
justin
3551ca2fcf
enhancement: replace sort.Slice with slices.SortFunc to reduce reflection
2022-04-18 10:35:43 +08:00
chrislu
1384529eb7
Fix filer.backup deletes files in backup folder in incremental mode
...
fix https://github.com/chrislusf/seaweedfs/issues/2919
2022-04-14 13:35:01 -07:00
a
846858fb43
merge master
2022-04-12 14:42:29 -07:00
a
41d396edc4
Merge branch 'master' into a
2022-04-07 08:22:59 -07:00
Konstantin Lebedev
f5246b748d
Merge branch 'new_master' into hashicorp_raft
...
# Conflicts:
# weed/pb/master_pb/master.pb.go
2022-04-07 18:50:27 +05:00
chrislu
86ed27f602
mount: remove leftover socket file
2022-04-07 00:33:13 -07:00
eddy-gfx
ec53eec94f
Merge branch 'master' into a
2022-04-06 18:45:55 -05:00
Konstantin Lebedev
17c6e8e39f
Merge branch 'new_master' into hashicorp_raft
...
# Conflicts:
# go.mod
# go.sum
2022-04-05 13:29:46 +05:00
Konstantin Lebedev
622297f1a7
add stats raft handler
2022-04-04 19:16:06 +05:00
Konstantin Lebedev
14dd971890
hashicorp raft with state machine
2022-04-04 17:51:51 +05:00
Konstantin Lebedev
c514710b7b
initial add hashicorp raft
2022-04-04 13:50:56 +05:00
chrislu
6a2bcd03aa
configure mount quota
2022-04-02 21:34:26 -07:00
chrislu
958f880b70
mount: add grpc method to adjust quota
2022-04-02 15:14:37 -07:00
chrislu
daca2d22a5
use original server address string as map key
2022-04-01 17:34:42 -07:00
a
7e92517571
change user and pass to username and password
2022-04-01 14:09:25 -05:00
Konstantin Lebedev
b6a2b43efe
allowDeleteBucketNotEmpty
2022-03-30 22:46:13 +05:00
a
bc603e534f
diff
2022-03-28 16:50:28 +00:00
chrislu
21e0898631
refactor: change masters from a slice to a map
2022-03-26 13:33:17 -07:00
chrislu
4ba7127ab1
refactor
2022-03-26 13:13:19 -07:00
Konstantin Lebedev
849a2c1f68
fix cmdFiler
2022-03-23 14:30:46 +05:00
Konstantin Lebedev
105bcfdbd4
set iam ip from filer
2022-03-23 13:45:51 +05:00
elee
423ce57cde
prefix search, bucket implemented
2022-03-17 21:12:25 -05:00
Berck Nash
7ee38fa3a4
The fixes for https://github.com/chrislusf/seaweedfs/issues/1937 had a few problems:
...
(1) The help file says that in the absence of a ipBind being specified, that it will bind to the "ip" specified. Instead, it bound to localhost which broke the default configuration. This change implements the documented behavior instead.
(2) The new IAM filer ip address has no default. This instantiates it to the same as the filer IP. I'm not sure if there should be a corresponding iam.ip or iam.ipBind option added to the filer command?
2022-03-17 15:30:23 -06:00
elee
921535001a
arangodb adapter
2022-03-17 04:49:26 -05:00
Berck Nash
9b14f0c81a
Add mTLS support for both master and volume http server.
2022-03-16 09:52:17 -06:00
chrislu
3639cad69c
master, filer, s3: also listen to "localhost" in addition to specific ip address
...
related to https://github.com/chrislusf/seaweedfs/issues/1937
2022-03-15 22:28:18 -07:00
chrislu
3a6eb8ca5f
default bind to one ip address
...
fix https://github.com/chrislusf/seaweedfs/issues/1937
2022-03-11 14:02:39 -08:00
banjiaojuhao
b9ff7723dd
[bugfix] filer: nil pointer dereference
2022-03-07 23:26:25 +08:00
chrislu
da3d330616
s3 and filer transport using unix domain socket instead of tcp
2022-03-07 02:00:14 -08:00
chrislu
c7e8ac18f0
mount: quota for one mounted collection
...
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
2022-03-06 02:44:40 -08:00
chrislu
f51e20028a
mount: avoid comma in mount options
...
fix https://github.com/chrislusf/seaweedfs/issues/2719
2022-03-03 03:42:29 -08:00
chrislu
18543c6e8b
minor
2022-02-27 23:11:09 -08:00
chrislu
c3792c8352
remove dead code
2022-02-27 03:03:19 -08:00
chrislu
aa9eef81e6
retire mount v1
2022-02-27 02:57:27 -08:00
chrislu
6e1ab97988
use debug option to see operations
2022-02-27 02:02:30 -08:00
chrislu
2112d99140
mount2: add back readonly mode
2022-02-27 01:13:32 -08:00
chrislu
fc7a4957ea
fix mount2 options
2022-02-25 21:22:44 -08:00
chrislu
202a29d014
refactoring
2022-02-25 01:17:26 -08:00
Tuan Vuong
d2ec62656d
initialize master address in iam options
2022-02-23 12:01:54 +07:00
banjiaojuhao
b5ec346700
FilerStore: add redis_lua
2022-02-15 20:54:57 +08:00
chrislu
4e72863ba5
mount2 add debug mode
2022-02-15 00:26:30 -08:00
chrislu
4e181db21a
mount: default disable cache
...
* Prevent cases as https://github.com/seaweedfs/seaweedfs-csi-driver/issues/43
* Improve read write benchmarks
* Improve AI training performance. Most of the files are just read once.
2022-02-14 20:42:33 -08:00
chrislu
377bf31445
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-02-14 13:48:51 -08:00
chrislu
e8420aaed7
fix building for windows freebsd
2022-02-14 13:48:48 -08:00
Chris Lu
d3ee621fce
Merge pull request #2661 from garenchan/ck-dev1
2022-02-14 10:08:37 -08:00
Konstantin Lebedev
526094d2da
StopTimeout 30 sec
2022-02-14 21:42:27 +05:00
Konstantin Lebedev
275e9a4e86
reduce to default http server KillTimeout and StopTimeout
2022-02-14 21:38:24 +05:00
garenchan
bd032eabe7
[UPDATE] Make heartbeat interval and election timeout of masters configurable.
2022-02-14 21:09:07 +08:00
chrislu
05724a68d4
skip other OS
2022-02-14 02:59:51 -08:00
chrislu
b9c2bff931
clean up
2022-02-14 02:14:26 -08:00
chrislu
dbeeda8123
listen for metadata updates
2022-02-14 01:09:31 -08:00
chrislu
180445f5a8
change to use fuse file system
2022-02-11 21:35:09 -08:00
chrislu
f87da798a4
to be re-written following fuse virtual file system
2022-02-11 03:09:30 -08:00
chrislu
b6143de52a
mount with name
2022-02-10 22:43:55 -08:00
chrislu
7a0c35674c
clean up previously mounted folder
2022-02-10 20:46:53 -08:00
chrislu
c3f9d9fa2e
initial setup
2022-02-10 20:32:13 -08:00
chrislu
85c1615b43
filer read empty file may cause OOM in some cases
...
fix https://github.com/chrislusf/seaweedfs/issues/2641
2022-02-07 23:08:54 -08:00
chrislu
affe3c2c12
change to util.WriteFile
2022-02-04 21:32:27 -08:00
chrislu
84c9bc4389
edge case: old entry was not replicated to remote storage
2022-01-30 20:23:24 -08:00
Chris Lu
7c66f3b5fb
Merge pull request #2602 from kmlebedev/master_metrics
...
master metricsHttpPort
2022-01-20 09:26:25 -08:00
chrislu
b3e526ba95
url should be always using forward slash
2022-01-19 22:16:26 -08:00
Konstantin Lebedev
77c98b657e
master metricsHttpPort
2022-01-19 21:43:22 +05:00
guol-fnst
da9540e666
add gocql timeout setting
2022-01-18 15:21:13 +08:00
chrislu
fc0628c038
working
2022-01-17 01:53:56 -08:00
chrislu
2bfeb5d1c8
add filer to iam option
2022-01-15 03:37:52 -08:00
chrislu
b17c426e99
weed server: optionally start IAM service
...
related to https://github.com/chrislusf/seaweedfs/issues/2560
2022-01-13 22:49:49 -08:00
chrislu
8907e6a40a
add more help messages
2022-01-13 13:03:04 -08:00
chrislu
826a7b307e
master: remove hard coded filer settings in master.toml
...
fix https://github.com/chrislusf/seaweedfs/issues/2529
2022-01-12 01:11:25 -08:00
Kyle Sanderson
9e012001be
filer.copy: don't crash when volume creation fails
...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1d58247]
goroutine 7482 [running]:
github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks.func1(0x2)
/go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:488 +0x2a7
created by github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks
/go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:455 +0x225
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1d58247]
goroutine 7480 [running]:
github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks.func1(0x0)
/go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:488 +0x2a7
created by github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks
/go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:455 +0x225
2022-01-11 22:22:39 -08:00
chrislu
1a7d5b5b5e
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-01-11 12:24:56 -08:00
chrislu
41daecfdca
Update mount_std.go
2022-01-11 12:23:12 -08:00
Chris Lu
abe5da7d2c
Merge pull request #2575 from Radtoo/fix_paths2
...
Fix paths2
2022-01-11 12:04:30 -08:00
chrislu
6a12520a96
fix logging
2022-01-10 01:00:11 -08:00
Radtoo
389002f195
Using positional arguments rather than option flag to enable better shell usage
2022-01-08 16:52:12 +01:00
Radtoo
fba1efb77a
Now works with a single file too
...
Parsing removed from doFixOneVolume
Needle init removed from runFix
2022-01-08 16:31:53 +01:00
chrislu
110d5a5233
support fixing a collection of volumes, or volumes under one directory
2022-01-07 14:52:16 -08:00
Chris Lu
42c849e0df
Merge branch 'master' into metadata_follow_with_client_id
2022-01-02 01:07:30 -08:00
Chris Lu
9b94177380
Merge pull request #2543 from skurfuerst/seaweedfs-158
...
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
2022-01-01 22:34:13 -08:00
Sebastian Kurfuerst
1cd3b6b4e1
BUGFIX: security.toml contained wrong keys
2021-12-31 22:05:41 +01:00
Sebastian Kurfuerst
10404c4275
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
...
- one JWT for reading and one for writing, analogous to how the JWT
between Master and Volume Server works
- I did not implement IP `whiteList` parameter on the filer
Additionally, because http_util.DownloadFile now sets the JWT,
the `download` command should now work when `jwt.signing.read` is
configured. By looking at the code, I think this case did not work
before.
## Docs to be adjusted after a release
Page `Amazon-S3-API`:
```
# Authentication with Filer
You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as
explained in [Security-Configuration](Security-Configuration) -
controlled by the `grpc.*` configuration in `security.toml`.
Starting with version XX, it is also possible to authenticate the HTTP
operations between the S3-API-Proxy and the Filer (especially
uploading new files). This is configured by setting
`filer_jwt.signing.key` and `filer_jwt.signing.read.key` in
`security.toml`.
With both configurations (gRPC and JWT), it is possible to have Filer
and S3 communicate in fully authenticated fashion; so Filer will reject
any unauthenticated communication.
```
Page `Security Overview`:
```
The following items are not covered, yet:
- master server http REST services
Starting with version XX, the Filer HTTP REST services can be secured
with a JWT, by setting `filer_jwt.signing.key` and
`filer_jwt.signing.read.key` in `security.toml`.
...
Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer.
Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).**
...
# Securing Filer HTTP with JWT
To enable JWT-based access control for the Filer,
1. generate `security.toml` file by `weed scaffold -config=security`
2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string
3. copy the same `security.toml` file to the filers and all S3 proxies.
If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`.
If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`.
The S3 API Gateway reads the above JWT keys and sends authenticated
HTTP requests to the filer.
```
Page `Security Configuration`:
```
(update scaffold file)
...
[filer_jwt.signing]
key = "blahblahblahblah"
[filer_jwt.signing.read]
key = "blahblahblahblah"
```
Resolves : #158
2021-12-30 14:45:27 +01:00
chrislu
5c87fcc6d2
add client id for all metadata listening clients
2021-12-30 00:23:57 -08:00
chrislu
fb434318e3
dynamically adjust connection timeout
...
better fix for https://github.com/chrislusf/seaweedfs/issues/2541
2021-12-29 22:44:39 -08:00
chrislu
5788bf2270
s3: increase timeout limit
...
https://github.com/chrislusf/seaweedfs/issues/2541
2021-12-29 22:21:02 -08:00
chrislu
498661e3bb
mount: remove limits on number of parallel requests
2021-12-28 17:41:01 -08:00
chrislu
0da2dfd640
fuse: change to direct io mode
...
before and after:
chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 4.534068 secs (236816430 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 0.01s user 3.86s system 84% cpu 4.561 total
chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 3.824072 secs (280784948 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 0.01s user 3.22s system 83% cpu 3.857 total
2021-12-28 12:22:56 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Sebastian Kurfürst
6db49100d6
BUGFIX: add access.ui setting to scaffolded security.toml
...
... The property is read here: b70cb3e0b2/weed/server/volume_server.go (L69)
2021-12-24 13:59:04 +01:00
chrislu
85c526c583
s3: bind to a specific IP
...
fix https://github.com/chrislusf/seaweedfs/issues/2516
2021-12-17 11:34:37 -08:00
chrislu
bf4d7affc0
gateway to remote object store: skip replicating multipart upload part files
...
fix https://github.com/chrislusf/seaweedfs/issues/2509
2021-12-14 19:48:31 -08:00
chrislu
316f326464
add more help message
2021-12-13 13:14:36 -08:00
kmlebedev
4f98553ba9
audit log SignatureVersion
2021-12-10 19:40:32 +05:00
Konstantin Lebedev
98251fe16a
non blocking audit log
2021-12-09 19:47:16 +05:00
Chris Lu
2ba08afed1
Merge pull request #2498 from kmlebedev/s3_audit_log
2021-12-07 09:35:48 -08:00
Konstantin Lebedev
10678cde81
audit log config
2021-12-07 18:20:52 +05:00
Konstantin Lebedev
4ec8715f20
audit log
2021-12-07 12:15:48 +05:00
chrislu
c146c76d10
avoid creating the same bucket with a different randomized name
...
related to https://github.com/chrislusf/seaweedfs/issues/2492
2021-12-05 13:06:41 -08:00
chrislu
42d97a3442
adjust randomized bucket name
2021-12-05 12:36:58 -08:00
Chris Lu
689f5513a9
redis3 supports sentinel
2021-11-29 01:09:51 -08:00
Chris Lu
3d7390302d
add s3.clean.uploads -timeAgo=24h
2021-11-29 00:49:49 -08:00
Chris Lu
ad16221a35
adjust error log
2021-11-28 22:06:17 -08:00
Chris Lu
cf1586a34d
add logs for writing to remote file
2021-11-27 22:09:23 -08:00
limd
8805c04128
fix redis2 sentinel config example
2021-11-25 19:20:02 +08:00
limd
ec03f22cc3
Merge remote-tracking branch 'origin/master'
2021-11-25 16:07:14 +08:00
limd
220797bd71
support redis sentinel
2021-11-25 15:57:03 +08:00
Chris Lu
6c27845be0
add retries when writing to remote s3
...
fix https://github.com/chrislusf/seaweedfs/issues/2465
2021-11-22 21:48:04 -08:00
Chris Lu
1f75f1f9dc
filer: fix mysql2 SQL template
2021-11-11 22:28:28 -08:00
Chris Lu
3abbaccb70
filer: fix mysql command to upsert
2021-11-11 22:27:13 -08:00
Chris Lu
c4e22b5a9a
filer: deprecate "-peers" option
2021-11-06 14:36:45 -07:00
Chris Lu
5ea86ef1da
Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()"
...
This reverts commit af71ae11aa
2021-11-05 17:52:15 -07:00
Chris Lu
af71ae11aa
master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()
2021-11-03 01:09:48 -07:00
Chris Lu
ab97b17e62
better printout
2021-11-02 23:45:47 -07:00
Chris Lu
5160eb08f7
shell: optionally read filer address from master
2021-11-02 23:38:45 -07:00
Chris Lu
edbf6d297b
filer.meta.tail: add example to send metadata to elastic search
2021-10-31 18:01:33 -07:00
Chris Lu
3be3c17f59
volume vacuum: avoid timeout with streaming progress report
...
fix https://github.com/chrislusf/seaweedfs/issues/2396
2021-10-24 01:55:34 -07:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
3e2acf677c
removing tikv to resolve "go mod tidy" problem
...
tikv is causing "go mod tidy" problem. Need to resolve this before adding tikv back.
go mod tidy
go: finding module for package github.com/coreos/etcd/clientv3/balancer/picker
go: finding module for package cloud.google.com/go/kms/apiv1
go: finding module for package github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
go: finding module for package google.golang.org/grpc/naming
go: finding module for package github.com/coreos/etcd/clientv3/credentials
go: finding module for package github.com/coreos/etcd/clientv3/balancer
go: finding module for package github.com/d4l3k/messagediff
go: found github.com/coreos/etcd/clientv3/balancer in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/picker in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/resolver/endpoint in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/credentials in github.com/coreos/etcd v3.3.26+incompatible
go: found cloud.google.com/go/kms/apiv1 in cloud.google.com/go/kms v1.0.0
go: found github.com/d4l3k/messagediff in github.com/d4l3k/messagediff v1.2.1
go: finding module for package google.golang.org/grpc/naming
github.com/chrislusf/seaweedfs/weed/filer/tikv imports
github.com/tikv/client-go/v2/tikv imports
go.etcd.io/etcd/clientv3 tested by
go.etcd.io/etcd/clientv3.test imports
github.com/coreos/etcd/integration imports
github.com/coreos/etcd/proxy/grpcproxy imports
google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
2021-10-10 19:27:02 -07:00
Chris Lu
bf218cd59d
removing etcd sequencer
...
causing go mod tidy problem. If anyone wants this, please help to resolve this first.
github.com/chrislusf/seaweedfs/weed/sequence imports
go.etcd.io/etcd/client tested by
go.etcd.io/etcd/client.test imports
github.com/coreos/etcd/integration imports
github.com/coreos/etcd/proxy/grpcproxy imports
google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
2021-10-10 19:10:46 -07:00
Chris Lu
6a030547a2
server: remove peer check if not starting master
...
more fix https://github.com/chrislusf/seaweedfs/issues/2352
2021-10-04 03:27:10 -07:00
Chris Lu
62c2732fd1
add back logic to check master peers
...
fix https://github.com/chrislusf/seaweedfs/issues/2352
2021-10-04 02:51:26 -07:00
Chris Lu
d4bb16e20e
Merge pull request #2354 from chrislusf/bptree
...
Add Redis3
2021-10-04 02:43:15 -07:00
Chris Lu
04662126bb
add redis3
2021-10-04 01:04:27 -07:00
Chris Lu
366f522a2d
add redis3
2021-10-04 01:01:31 -07:00
Chris Lu
b297849147
typo
2021-10-01 23:24:54 -07:00
Chris Lu
af207bbaf0
retry both assign volume and uploading data
...
fix https://github.com/chrislusf/seaweedfs/issues/2351
2021-10-01 23:23:39 -07:00
Chris Lu
4a1d4d7462
s3: default to allow empty folder for better performance
2021-09-26 22:34:14 -07:00
Chris Lu
a814f3f0a8
adjust metadata tail output
2021-09-25 01:04:51 -07:00
Chris Lu
52fe86df45
use default 10000 for grpc port
2021-09-20 14:05:59 -07:00
Chris Lu
84fdda85e7
go fmt
2021-09-19 12:06:15 -07:00
Chris Lu
94a01fcfcb
filer.remote.gateway: add options to include or exclude new bucket names to mirror
2021-09-15 23:04:16 -07:00
Chris Lu
b5f4910412
adjust help messages
2021-09-15 22:53:10 -07:00
Chris Lu
f0907eb83c
add command filre.remote.gateway
2021-09-15 22:48:04 -07:00
Chris Lu
63da4bbb54
separate filer.remote.gateway command to avoid confusion
2021-09-15 22:47:17 -07:00
Chris Lu
2789d10342
go fmt
2021-09-14 10:37:06 -07:00
Chris Lu
e5fc35ed0c
change server address from string to a type
2021-09-12 22:47:52 -07:00
Chris Lu
2c9d4c8f43
custom grpc port: filer
2021-09-12 02:28:37 -07:00
Chris Lu
e690a2be16
custom grpc port: volume server
2021-09-12 02:25:15 -07:00
Chris Lu
232ad2fe65
custom grpc port: master follower
2021-09-12 02:25:00 -07:00
