Commit Graph

9943 Commits

Author SHA1 Message Date
Mesar Hameed
a90b777ff4
Feat: etcd filer store keys should have customizable prefix (#4484)
An etcd cluster is not necessarily only dedicated to seaweedfs.
This security enhancement adds a customizable key_prefix option to the etcd filer store.
This will allow an etcd cluster administrator to limit the seaweedfs etcd user to only read/write a subset of keys under the
key_prefix, instead of all keys on the etcd cluster.
2023-05-18 23:08:56 -07:00
SmsS4
17e91d2917
Use filerGroup for s3 buckets collection prefix (#4465)
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
2023-05-16 09:39:43 -07:00
chrislu
b7f011f777 fix test 2023-05-15 17:46:56 -07:00
Mesar Hameed
65484e80ef
Feat: support username/password authentication for etcd filer store s… (#4477)
Feat: support username/password authentication for etcd filer store seaweedfs/seaweedfs#4262

Co-authored-by: Mesar Hameed <mesar.hameed@gmail.com>
2023-05-15 17:45:14 -07:00
dependabot[bot]
64b2fbe0f1
build(deps): bump go.mongodb.org/mongo-driver from 1.11.4 to 1.11.6 (#4475)
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.11.4 to 1.11.6.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.11.4...v1.11.6)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:46:06 -07:00
dependabot[bot]
2c347bec0c
build(deps): bump golang.org/x/tools from 0.8.0 to 0.9.1 (#4474)
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.8.0 to 0.9.1.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.8.0...v0.9.1)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:45:49 -07:00
dependabot[bot]
d77c787ec6
build(deps): bump cloud.google.com/go/pubsub from 1.30.0 to 1.30.1 (#4473)
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.30.0 to 1.30.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.30.0...pubsub/v1.30.1)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:45:33 -07:00
dependabot[bot]
7034fd155f
build(deps): bump actions/setup-go from 4.0.0 to 4.0.1 (#4470)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](4d34df0c23...fac708d667)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:45:23 -07:00
dependabot[bot]
7d9f9ecda5
build(deps): bump go.etcd.io/etcd/client/v3 from 3.5.8 to 3.5.9 (#4471)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.8 to 3.5.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.8...v3.5.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:25:45 -07:00
dependabot[bot]
79d42eede4
build(deps): bump google.golang.org/api from 0.114.0 to 0.122.0 (#4472)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.114.0 to 0.122.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.114.0...v0.122.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-15 09:24:53 -07:00
chrislu
9204ee2d2d 3.50 2023-05-11 08:03:11 -07:00
chrislu
2e351aa967 Revert "build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.43.0 to 3.46.0 (#4456)"
This reverts commit d232590299.
2023-05-09 23:18:46 -07:00
chrislu
b0062c23cb Update go.sum 2023-05-09 23:11:40 -07:00
chrislu
51d9066e90 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-05-09 22:54:26 -07:00
chrislu
d999f1f0e2 update raft version
fix #4460
2023-05-09 22:54:23 -07:00
dependabot[bot]
e624a1c234
build(deps): bump github.com/aws/aws-sdk-go from 1.44.253 to 1.44.258 (#4457)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.253 to 1.44.258.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG_PENDING.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.253...v1.44.258)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 09:32:26 -07:00
dependabot[bot]
d232590299
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.43.0 to 3.46.0 (#4456)
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.43.0 to 3.46.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.43.0...v3.46.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 09:32:18 -07:00
dependabot[bot]
953c7d648f
build(deps): bump github.com/lib/pq from 1.10.8 to 1.10.9 (#4455)
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.8 to 1.10.9.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.8...v1.10.9)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 09:32:10 -07:00
dependabot[bot]
3297d59360
build(deps): bump github.com/hanwen/go-fuse/v2 from 2.2.0 to 2.3.0 (#4454)
Bumps [github.com/hanwen/go-fuse/v2](https://github.com/hanwen/go-fuse) from 2.2.0 to 2.3.0.
- [Commits](https://github.com/hanwen/go-fuse/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: github.com/hanwen/go-fuse/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 09:32:00 -07:00
dependabot[bot]
1a97e1ddd8
build(deps): bump google.golang.org/grpc from 1.54.0 to 1.55.0 (#4453)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.54.0 to 1.55.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.54.0...v1.55.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 09:30:45 -07:00
chrislu
25b237b83c reset wait time on a successful retry 2023-05-07 23:39:33 -07:00
chrislu
59f55c1a66 3.49 2023-05-07 14:00:19 -07:00
chrislu
b2a3860290 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-05-07 13:33:47 -07:00
chrislu
4511edc871 update raft 2023-05-07 13:33:44 -07:00
themarkchen
7592d013fe
fix shell volume.balance bug (#4447) 2023-05-04 07:35:37 -07:00
dependabot[bot]
6fdff0bb18
build(deps): bump github.com/aws/aws-sdk-go from 1.44.248 to 1.44.253 (#4443)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.248 to 1.44.253.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.248...v1.44.253)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:14:41 -07:00
dependabot[bot]
c2129d5555
build(deps): bump github.com/go-sql-driver/mysql from 1.7.0 to 1.7.1 (#4440)
Bumps [github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/go-sql-driver/mysql/releases)
- [Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.7.0...v1.7.1)

---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:14:33 -07:00
dependabot[bot]
fdf1064a92
build(deps): bump github.com/gocql/gocql from 1.3.2 to 1.4.0 (#4441)
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql) from 1.3.2 to 1.4.0.
- [Release notes](https://github.com/gocql/gocql/releases)
- [Changelog](https://github.com/gocql/gocql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gocql/gocql/compare/v1.3.2...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/gocql/gocql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:14:24 -07:00
dependabot[bot]
dc283e3bd9
build(deps): bump github.com/hashicorp/raft from 1.4.0 to 1.5.0 (#4442)
Bumps [github.com/hashicorp/raft](https://github.com/hashicorp/raft) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/hashicorp/raft/releases)
- [Changelog](https://github.com/hashicorp/raft/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/raft/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/raft
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:14:16 -07:00
dependabot[bot]
00e243d98e
build(deps): bump modernc.org/sqlite from 1.22.0 to 1.22.1 (#4444)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.22.0 to 1.22.1.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.22.0...v1.22.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:14:06 -07:00
dependabot[bot]
b6dd4b8477
build(deps): bump wangyoucao577/go-release-action from 1.37 to 1.38 (#4439)
Bumps [wangyoucao577/go-release-action](https://github.com/wangyoucao577/go-release-action) from 1.37 to 1.38.
- [Release notes](https://github.com/wangyoucao577/go-release-action/releases)
- [Commits](b98909985b...0aa3e4818b)

---
updated-dependencies:
- dependency-name: wangyoucao577/go-release-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 09:13:56 -07:00
chrislu
42766bccc8 3.48 2023-04-30 22:20:18 -07:00
zemul
e9fda774f4
[Filer] post add param:saveInside (#4434)
* fix:mount deadlock

* feat: filer http upload to metadata

* feat: /etc save inside

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-29 08:31:05 -07:00
wusong
4867aa03ab
Fix: http rename move dir to subdir (#4432)
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-04-26 20:54:36 -07:00
Konstantin Lebedev
d75a7b7f62
allow deleting only older empty dir without recursion (#4430) 2023-04-25 08:31:14 -07:00
dependabot[bot]
fafff5896e
build(deps): bump go.etcd.io/etcd/client/v3 from 3.5.7 to 3.5.8 (#4428)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.7 to 3.5.8.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Changelog](https://github.com/etcd-io/etcd/blob/main/Dockerfile-release.ppc64le)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.7...v3.5.8)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:37:32 -07:00
dependabot[bot]
a36967982d
build(deps): bump github.com/aws/aws-sdk-go from 1.44.244 to 1.44.248 (#4424)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.244 to 1.44.248.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.244...v1.44.248)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:27:40 -07:00
dependabot[bot]
3e714a53cf
build(deps): bump golang.org/x/image from 0.6.0 to 0.7.0 (#4427)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/image/releases)
- [Commits](https://github.com/golang/image/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:27:25 -07:00
dependabot[bot]
5a46005ddf
build(deps): bump go.mongodb.org/mongo-driver from 1.11.3 to 1.11.4 (#4426)
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.11.3 to 1.11.4.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.11.3...v1.11.4)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:27:17 -07:00
dependabot[bot]
54440f0d63
build(deps): bump modernc.org/sqlite from 1.21.1 to 1.22.0 (#4425)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.21.1 to 1.22.0.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.21.1...v1.22.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:27:09 -07:00
dependabot[bot]
fab0ab5440
build(deps): bump docker/metadata-action from 4.3.0 to 4.4.0 (#4423)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4.3.0 to 4.4.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](507c2f2dc5...c4ee3adeed)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:26:49 -07:00
zemul
f352616b7d
Volume range read use sync.pool (#4422) 2023-04-24 07:00:48 -07:00
chrislu
18686b7375 3.47 2023-04-23 22:45:41 -07:00
zemul
68f1cc34a5
[mount]Fix copy file range (#4407)
* fix:mount deadlock

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-23 22:34:38 -07:00
chrislu
e1b92bc701 reset local file content if remote file is updated
fix https://github.com/seaweedfs/seaweedfs/issues/4289
2023-04-22 22:32:21 -07:00
chrislu
ff7b6d779e avoid overwriting variables
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-22 11:49:59 -07:00
wusong
2e240704ab
Writables inconsistency (#4417)
fix: inconsistent read and write permissions between master and volume server

Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-21 00:14:41 -07:00
wusong
19245dde50
mount: add retry for read only case (#4416)
* mount: add retry for read only case

Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>

* add new util retry function for mount

Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>

* change error list param

Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>

---------

Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-21 00:09:31 -07:00
SmsS4
4131874fa8
use env for flags wtih dot in name (#4415) 2023-04-20 09:33:27 -07:00
Damiano Albani
9f55c7c90e
Rclone storage backend (#4402)
* Add Rclone storage backend

* Support templating the name of files stored via Rclone

* Enable Rclone accounting

* Remove redundant type conversion

* Provide progress information for Rclone download/upload operations

* Log error when Rclone can't instantiate filesystem

* Remove filename templating functionality for Rclone storage

To (maybe) be later reintroduced as a generic functionality for all
storage backends.

* Remove S3 specific check

* Move Rclone config initialisation to init() method
2023-04-18 14:12:12 -07:00