Lisandro Pin
9fbc4ea417
Rework shell.EcBalance()'s waitgroup code into a standalone type. ( #6373 )
...
Rework `shell.EcBalance()`'s waitgroup with errors code into a standalone type.
We'll re-use this for other EC jobs - for example, volume creation. Also fixes
potential concurrency issues when collecting error results.
2024-12-17 09:39:51 -08:00
zemul
e77e50886e
mount metacache add ttl ( #6360 )
...
* fix:mount deadlock
* fix
* feat: metaCache ttl
* Update weed/command/mount.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* fix InodeEntry
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-12-16 20:19:32 -08:00
dependabot[bot]
b2f26804a0
chore(deps): bump docker/setup-buildx-action from 3.7.1 to 3.8.0 ( #6369 )
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 3.7.1 to 3.8.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](c47758b77c...6524bf65af
2024-12-16 14:09:26 -08:00
dependabot[bot]
52cf759a96
chore(deps): bump golang.org/x/crypto from 0.30.0 to 0.31.0 ( #6370 )
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.30.0 to 0.31.0.
- [Commits](https://github.com/golang/crypto/compare/v0.30.0...v0.31.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 14:09:12 -08:00
dependabot[bot]
93da4fe4d5
chore(deps): bump wangyoucao577/go-release-action from 1.52 to 1.53 ( #6368 )
...
Bumps [wangyoucao577/go-release-action](https://github.com/wangyoucao577/go-release-action ) from 1.52 to 1.53.
- [Release notes](https://github.com/wangyoucao577/go-release-action/releases )
- [Commits](2aa2977ad6...481a2c1a0f
2024-12-16 14:08:49 -08:00
dependabot[bot]
fa2d28c998
chore(deps): bump helm/kind-action from 1.10.0 to 1.11.0 ( #6367 )
...
Bumps [helm/kind-action](https://github.com/helm/kind-action ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/helm/kind-action/releases )
- [Commits](https://github.com/helm/kind-action/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: helm/kind-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:29:07 -08:00
dependabot[bot]
641fbc7a80
chore(deps): bump cloud.google.com/go/storage from 1.45.0 to 1.48.0 ( #6366 )
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.45.0 to 1.48.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.45.0...spanner/v1.48.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:28:56 -08:00
dependabot[bot]
ab7b81bbad
chore(deps): bump github.com/schollz/progressbar/v3 from 3.16.0 to 3.17.1 ( #6365 )
...
chore(deps): bump github.com/schollz/progressbar/v3
Bumps [github.com/schollz/progressbar/v3](https://github.com/schollz/progressbar ) from 3.16.0 to 3.17.1.
- [Release notes](https://github.com/schollz/progressbar/releases )
- [Commits](https://github.com/schollz/progressbar/compare/v3.16.0...v3.17.1 )
---
updated-dependencies:
- dependency-name: github.com/schollz/progressbar/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:28:43 -08:00
dependabot[bot]
cbc3afe7d1
chore(deps): bump go.mongodb.org/mongo-driver from 1.16.0 to 1.17.1 ( #6364 )
...
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver ) from 1.16.0 to 1.17.1.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases )
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.16.0...v1.17.1 )
---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:28:32 -08:00
dependabot[bot]
b4bcedf064
chore(deps): bump github.com/getsentry/sentry-go from 0.29.1 to 0.30.0 ( #6363 )
...
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go ) from 0.29.1 to 0.30.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases )
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md )
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.29.1...v0.30.0 )
---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:28:22 -08:00
dependabot[bot]
7a92b2cbb9
chore(deps): bump github.com/fclairamb/ftpserverlib from 0.24.1 to 0.25.0 ( #6362 )
...
chore(deps): bump github.com/fclairamb/ftpserverlib
Bumps [github.com/fclairamb/ftpserverlib](https://github.com/fclairamb/ftpserverlib ) from 0.24.1 to 0.25.0.
- [Release notes](https://github.com/fclairamb/ftpserverlib/releases )
- [Commits](https://github.com/fclairamb/ftpserverlib/compare/v0.24.1...v0.25.0 )
---
updated-dependencies:
- dependency-name: github.com/fclairamb/ftpserverlib
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-16 12:28:11 -08:00
coffeecloudgit
c531e2eb0e
Update disk_location_ec.go ( #6359 )
...
Add lock for l.ecVolumes
2024-12-15 22:05:47 -08:00
coffeecloudgit
7eb8257126
Update store.go ( #6358 )
...
Add lock for location.ecVolumes And single delete
2024-12-15 22:05:22 -08:00
Lisandro Pin
9b48ce0613
Parallelize EC shards balancing within racks ( #6354 )
...
Parallelize EC shards balancing within racks.
2024-12-15 13:36:23 -08:00
Lisandro Pin
926cfea3dc
Parallelize EC shards balancing across racks. ( #6352 )
2024-12-13 06:05:32 -08:00
Lisandro Pin
b81def5e5c
Parallelize EC balancing for racks. ( #6351 )
2024-12-13 05:33:53 -08:00
zouyixiong
d6f3e1970d
fix: filer may crash by bucketLastActiveTsNs concurrency access. ( #6350 )
2024-12-13 05:30:21 -08:00
Konstantin Lebedev
0a4b1909a2
[shell] only apply the balancing for writable volumes ( #6346 )
2024-12-13 01:10:00 -08:00
zouyixiong
9987a65e8a
fix: record and delete bucket metrics after inactive ( #6349 )
2024-12-12 20:34:02 -08:00
Lisandro Pin
b0210df081
Begin implementing EC balancing parallelization support. ( #6342 )
...
* Begin implementing EC balancing parallelization support.
Impacts both `ec.encode` and `ec.balance`,
* Nit: improve type naming.
* Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof.
* Nit: unify naming for `ecBalancer` wait group methods with the rest of the module.
* Fix concurrency bug.
* Fix whitespace after Gitlab automerge.
* Delete stray TODO.
2024-12-12 09:14:44 -08:00
Lisandro Pin
23ffbb083c
Limit EC re-balancing for ec.encode to relevant collections when a volume ID argument is provided. ( #6347 )
...
Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID is provided.
2024-12-12 08:41:33 -08:00
Lisandro Pin
6320036c56
Delete legacy balancing code for ec.encode. ( #6344 )
2024-12-12 07:42:03 -08:00
Konstantin Lebedev
700b95304b
[shell] volume.list show only writable volumes ( #6338 )
...
* show only writable volumes
* fix import
2024-12-11 09:06:15 -08:00
Konstantin Lebedev
c37281735e
volume.list avoid output empty data center and rack and disk info ( #6341 )
2024-12-11 09:03:16 -08:00
Lisandro Pin
8c82c037b9
Unify the re-balancing logic for ec.encode with ec.balance. ( #6339 )
...
Among others, this enables recent changes related to topology aware
re-balancing at EC encoding time.
2024-12-10 13:30:13 -08:00
Konstantin Lebedev
ff1392f7f4
[shell] use constant for hdd of type ( #6337 )
...
use constant for hdd of type
2024-12-10 08:43:59 -08:00
dependabot[bot]
4f6c989309
chore(deps): bump golang.org/x/net from 0.31.0 to 0.32.0 ( #6330 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.31.0 to 0.32.0.
- [Commits](https://github.com/golang/net/compare/v0.31.0...v0.32.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 09:59:05 -08:00
dependabot[bot]
af4c51aee6
chore(deps): bump cloud.google.com/go/pubsub from 1.45.1 to 1.45.3 ( #6328 )
...
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.45.1 to 1.45.3.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.45.1...pubsub/v1.45.3 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 09:04:45 -08:00
dependabot[bot]
3cd2f06279
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.69.0 to 1.71.0 ( #6329 )
...
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.69.0 to 1.71.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.69.0...service/s3/v1.71.0 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 08:46:16 -08:00
dependabot[bot]
e438fdc16c
chore(deps): bump github.com/hanwen/go-fuse/v2 from 2.7.1 to 2.7.2 ( #6331 )
...
Bumps [github.com/hanwen/go-fuse/v2](https://github.com/hanwen/go-fuse ) from 2.7.1 to 2.7.2.
- [Commits](https://github.com/hanwen/go-fuse/compare/v2.7.1...v2.7.2 )
---
updated-dependencies:
- dependency-name: github.com/hanwen/go-fuse/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 08:46:00 -08:00
dependabot[bot]
3680bbac56
chore(deps): bump golang.org/x/sys from 0.27.0 to 0.28.0 ( #6332 )
...
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.27.0 to 0.28.0.
- [Commits](https://github.com/golang/sys/compare/v0.27.0...v0.28.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 08:45:26 -08:00
dependabot[bot]
67461ccd27
chore(deps): bump docker/build-push-action from 6.9.0 to 6.10.0 ( #6333 )
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 6.9.0 to 6.10.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](4f58ea7922...48aba3b46d
2024-12-09 08:45:13 -08:00
Lisandro Pin
522a25790a
Remove average constraints when selecting nodes/racks to balance EC shards into. ( #6325 )
2024-12-06 09:00:06 -08:00
vadimartynov
914ff0e718
Replaced https.client.enabled with false in scaffold/security ( #6320 )
2024-12-05 10:39:28 -08:00
chrislu
f3dde99796
adjust error message
2024-12-05 09:33:50 -08:00
Lisandro Pin
34cdbdd279
Share common parameters for EC re-balancing functions under a single struct. ( #6319 )
...
TODO cleanup for https://github.com/seaweedfs/seaweedfs/discussions/6179 .
2024-12-05 09:00:46 -08:00
Lisandro Pin
edef485333
Account for replication placement settings when balancing EC shards within the same rack. ( #6317 )
...
* Account for replication placement settings when balancing EC shards within racks.
* Update help contents for `ec.balance`.
* Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
2024-12-04 10:47:51 -08:00
Lisandro Pin
351efa134d
Account for replication placement settings when balancing EC shards across racks. ( #6316 )
2024-12-04 09:00:55 -08:00
Konstantin Lebedev
b65eb2ec45
[security] reload whiteList on http seerver ( #6302 )
...
* reload whiteList
* white_list add to scaffold
2024-12-02 10:38:10 -08:00
Lisandro Pin
b2ba7d7408
Resolve replica placement for EC volumes from master server defaults. ( #6303 )
2024-12-02 08:44:07 -08:00
Lisandro Pin
9a741a61b1
Display details upon failures to re-balance EC shards racks. ( #6299 )
2024-11-28 08:42:41 -08:00
Lisandro Pin
559a1fd0f4
Improve EC shards rebalancing logic across nodes ( #6297 )
...
* Improve EC shards rebalancing logic across nodes.
- Favor target nodes with less preexisting shards, to ensure a fair distribution.
- Randomize selection when multiple possible target nodes are available.
- Add logic to account for replication settings when selecting target nodes (currently disabled).
* Fix minor test typo.
* Clarify internal error messages for `pickEcNodeToBalanceShardsInto()`.
2024-11-27 11:51:57 -08:00
steve.wei
88fa64a01a
change comment for expect header ( #6289 )
...
* remove case of expect
* Set the default value of the Expect header for compatibility.
2024-11-26 08:35:41 -08:00
Konstantin Lebedev
e2e97db917
[master] avoid timeout when assigning for main request with filter by DC or rack ( #6291 )
...
* avoid timeout when assigning for main request with filter by DC or rack
https://github.com/seaweedfs/seaweedfs/issues/6290
* use constant NoWritableVolumes
2024-11-26 08:33:31 -08:00
Konstantin Lebedev
fec88e64eb
[master] update LastLeaderChangeTime for hashicorp raft ( #6292 )
2024-11-26 08:02:45 -08:00
Trim21
d43fa07f06
use readable bytes size string in shell output ( #6288 )
2024-11-25 17:25:17 -08:00
dependabot[bot]
f965af51b7
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.3 to 1.69.0 ( #6287 )
...
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3
Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2 ) from 1.66.3 to 1.69.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases )
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.66.3...service/s3/v1.69.0 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-25 14:35:21 -08:00
dependabot[bot]
11042ddb71
chore(deps): bump docker/metadata-action from 5.5.1 to 5.6.1 ( #6281 )
...
Bumps [docker/metadata-action](https://github.com/docker/metadata-action ) from 5.5.1 to 5.6.1.
- [Release notes](https://github.com/docker/metadata-action/releases )
- [Commits](8e5442c4ef...369eb591f4
2024-11-25 14:27:46 -08:00
dependabot[bot]
e4abb6cbea
chore(deps): bump github.com/hanwen/go-fuse/v2 from 2.6.1 to 2.7.1 ( #6283 )
...
Bumps [github.com/hanwen/go-fuse/v2](https://github.com/hanwen/go-fuse ) from 2.6.1 to 2.7.1.
- [Commits](https://github.com/hanwen/go-fuse/compare/v2.6.1...v2.7.1 )
---
updated-dependencies:
- dependency-name: github.com/hanwen/go-fuse/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-25 14:26:51 -08:00
dependabot[bot]
19d01c02c6
chore(deps): bump github.com/stretchr/testify from 1.9.0 to 1.10.0 ( #6284 )
...
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify ) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/stretchr/testify/releases )
- [Commits](https://github.com/stretchr/testify/compare/v1.9.0...v1.10.0 )
---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-25 14:26:44 -08:00
