Commit Graph

10385 Commits

Author SHA1 Message Date
chrislu
0edc5aad9a 3.65 2024-04-14 20:15:46 -07:00
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part (#5476)
* try fix s3  test_multipart_get_part

* add passed s3 tests

* fix SeaweedFSUploadId

* rm spaces

* convert part request to range

* add passed s3 tests of multipart
2024-04-14 10:41:32 -07:00
yudrywet
8833745fbc
chore: fix some typos in comments (#5497) 2024-04-14 07:42:40 -07:00
chrislu
7a72677f98 add test 2024-04-12 01:31:23 -07:00
chrislu
d5b2cee34a log possible racing condition
fix https://github.com/seaweedfs/seaweedfs/issues/5412
2024-04-11 23:44:54 -07:00
chrislu
1410c062f8 go mod tidy 2024-04-11 19:50:25 -07:00
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk (#5458) 2024-04-11 04:47:21 -07:00
skycope
6dae685f9c
fix append disorder (#5492) 2024-04-10 19:21:44 -07:00
dependabot[bot]
f08f95ac80
chore(deps): bump github.com/shirou/gopsutil/v3 from 3.24.1 to 3.24.3 (#5481)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.24.1 to 3.24.3.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.24.1...v3.24.3)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:17:05 -07:00
dependabot[bot]
370c6586f0
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.36.0 to 0.37.0 (#5482)
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud) from 0.36.0 to 0.37.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.36.0...v0.37.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:16:55 -07:00
dependabot[bot]
7aef1b88c9
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.7 to 1.27.11 (#5483)
chore(deps): bump github.com/aws/aws-sdk-go-v2/config

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.27.7 to 1.27.11.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.27.7...config/v1.27.11)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:16:43 -07:00
dependabot[bot]
576a4376f4
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.56.1 to 3.63.0 (#5484)
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.56.1 to 3.63.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.56.1...v3.63.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:16:33 -07:00
dependabot[bot]
40ef5d05a3
chore(deps): bump golang.org/x/sys from 0.18.0 to 0.19.0 (#5485)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.18.0 to 0.19.0.
- [Commits](https://github.com/golang/sys/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:16:25 -07:00
dependabot[bot]
da42173d16
chore(deps): bump docker/setup-buildx-action from 3.2.0 to 3.3.0 (#5486)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](2b51285047...d70bba72b1)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-08 10:16:15 -07:00
steve.wei
67ead9b18f
fix(volume.fix.replication): adjust volume count, not free volume count (#5479) 2024-04-08 07:30:04 -07:00
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server (#5480) 2024-04-08 07:27:00 -07:00
clonefetch
9e07a87fcb
chore: fix function names in comment (#5478) 2024-04-08 07:19:02 -07:00
Konstantin Lebedev
3e25ed1b11
[s3] add s3 pass test_multipart_upload_size_too_small (#5475)
* add s3 pass test_multipart_upload_size_too_small

* refactor metric names

* return ErrNoSuchUpload if empty parts

* fix test
2024-04-07 11:52:35 -07:00
Konstantin Lebedev
35cba720a5
[s3] add s3 pass test_multipart_upload (#5474)
add s3 pass test_multipart_upload
2024-04-07 11:51:22 -07:00
Konstantin Lebedev
d42a04cceb
[s3] fix s3 test_multipart_resend_first_finishes_last (#5471)
* try fix s3 test
https://github.com/seaweedfs/seaweedfs/pull/5466

* add error handler metrics

* refactor

* refactor multipartExt

* delete bad entry parts
2024-04-06 10:56:39 -07:00
Nikita Borzykh
7aa25c1137
fix: correctly form partNumber from new entry.Name format in "listObjectParts" (#5470) 2024-04-04 09:16:16 -07:00
mergwyn
881c922972
helm chart: make s3 ingress annotations consistent (#5463)
Make s3 ingress annotations consistent
2024-04-03 09:09:34 -07:00
mergwyn
b4def8f000
Correct type for volume logs and idx values (#5462)
Correct type for volume logs and idx values (#1)

Empty values now correctly set to {}
2024-04-03 08:17:41 -07:00
skycope
b19c9847c6
fix completed multiupload lost data (#5460)
If there are putObjectPart requests with the same uploadId during
completeMultiPart, it can result in data loss. putObjectPart requests
might be due to timeout retries.

Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-04-02 20:18:42 -07:00
Konstantin Lebedev
d5d8b8e2ae
fix panic at isAllWritable (#5457)
fix panic
https://github.com/seaweedfs/seaweedfs/issues/5456
2024-04-02 09:06:19 -07:00
Henco Appel
2a88da4de7
fix: always close volume file (#4530) (#5459)
If sync fails then close is never called. We should always be calling
close on the file.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
2024-04-02 08:31:40 -07:00
Nikita Borzykh
df9031e19a
fix: ensure complete gRPC stream consumption in doListFilerEntries to prevent context canceled errors. (#5441) 2024-04-01 15:55:16 -07:00
dependabot[bot]
94008a84a4
chore(deps): bump github.com/schollz/progressbar/v3 from 3.14.1 to 3.14.2 (#5453) 2024-04-01 09:56:07 -07:00
dependabot[bot]
2fc37db77e
chore(deps): bump go.etcd.io/etcd/client/pkg/v3 from 3.5.12 to 3.5.13 (#5452) 2024-04-01 09:55:28 -07:00
dependabot[bot]
53fdda82f0
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.26.0 to 1.26.1 (#5451) 2024-04-01 09:55:11 -07:00
dependabot[bot]
fab5f66a95
chore(deps): bump github.com/linxGnu/grocksdb from 1.8.12 to 1.8.14 (#5450) 2024-04-01 09:02:26 -07:00
dependabot[bot]
c18a4c6911
chore(deps): bump github.com/go-sql-driver/mysql from 1.8.0 to 1.8.1 (#5449) 2024-04-01 09:01:57 -07:00
dependabot[bot]
d41c1c2956
chore(deps): bump github.com/viant/ptrie from 0.3.0 to 0.3.1 (#5447) 2024-04-01 09:01:36 -07:00
dependabot[bot]
d6149f87a3
chore(deps): bump github.com/cenkalti/backoff/v4 from 4.2.1 to 4.3.0 (#5446) 2024-04-01 09:01:18 -07:00
dependabot[bot]
ef06d6f074
chore(deps): bump google.golang.org/api from 0.169.0 to 0.172.0 (#5448) 2024-04-01 08:59:56 -07:00
dependabot[bot]
2e5aa8d03b
chore(deps): bump actions/dependency-review-action from 4.2.4 to 4.2.5 (#5445) 2024-04-01 08:27:33 -07:00
Nikita Borzykh
829c0da381
fix: adjust condition in prefixFilterEntries prevent infinite loop (#5440) 2024-03-30 14:47:00 -07:00
Konstantin Lebedev
f9ec92dd83
[tests] add action s3 tests (#5427) 2024-03-29 07:57:27 -07:00
Konstantin Lebedev
9c1e0f5811
[master] grow volumes if no writable volumes in current dataCenter (#5434)
* grow volumes if no writable volumes in current dataCenter
https://github.com/seaweedfs/seaweedfs/issues/3886

* fix tests with volume grow

* automatic volume grow one volume

* add ErrorChunkAssign metrics
2024-03-29 00:38:27 -07:00
Konstantin Lebedev
df40908e57
fix panic 5435 (#5436) 2024-03-28 16:17:59 -07:00
chrislu
6aa804b368 lock instead of RLock, to prevent racing condition
https://github.com/seaweedfs/seaweedfs/discussions/5432#discussioncomment-8933608
2024-03-27 16:05:11 -07:00
chrislu
36fec34c47 print only adapted url
fix https://github.com/seaweedfs/seaweedfs/issues/5424
2024-03-25 12:50:43 -07:00
dependabot[bot]
8b3756b815
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.25.3 to 1.26.0 (#5422)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.25.3 to 1.26.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.25.3...v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 10:10:35 -07:00
dependabot[bot]
14775a42f0
chore(deps): bump github.com/hashicorp/raft from 1.6.0 to 1.6.1 (#5420)
Bumps [github.com/hashicorp/raft](https://github.com/hashicorp/raft) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/hashicorp/raft/releases)
- [Changelog](https://github.com/hashicorp/raft/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/raft/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/raft
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 10:10:24 -07:00
dependabot[bot]
d48043be09
chore(deps): bump gocloud.dev from 0.36.0 to 0.37.0 (#5423)
Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.36.0 to 0.37.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.36.0...v0.37.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 09:52:04 -07:00
dependabot[bot]
f3599c9252
chore(deps): bump actions/dependency-review-action from 4.1.3 to 4.2.4 (#5418)
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 4.1.3 to 4.2.4.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](9129d7d40b...733dd5d4a5)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-25 08:27:45 -07:00
chrislu
aed6fdffd2 fix nil volume data backend
fix https://github.com/seaweedfs/seaweedfs/issues/5417
2024-03-25 08:19:31 -07:00
Nikita Borzykh
9ca871af53
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries in etcd meta storage backend (#5416)
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries
2024-03-24 14:08:54 -07:00
Nikita Korolev
2dd2bb3e16
fix filer address parsing (#5415) 2024-03-24 11:20:33 -07:00
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage (#5403) 2024-03-24 08:36:21 -07:00