Lisandro Pin
ca499de1cb
Improve EC shards rebalancing logic across racks ( #6270 )
...
Improve EC shards rebalancing logic across racks.
- Favor target shards with less preexisting shards, to ensure a fair distribution.
- Randomize selection when multiple possible target shards are available.
- Add logic to account for replication settings when selecting target shards (currently disabled).
2024-11-21 08:46:24 -08:00
Lisandro Pin
0d5393641e
Unify usage of shell.EcNode.dc as DataCenterId. ( #6258 )
2024-11-19 06:33:18 -08:00
Lisandro Pin
f2db746690
Introduce logic to resolve volume replica placement within EC rebalancing. ( #6254 )
...
* Rename `command_ec_encode_test.go` to `command_ec_common_test.go`.
All tests defined in this file are now for `command_ec_common.go`.
* Minor code cleanups.
- Fix broken `ec.balance` test.
- Rework integer ceiling division to not use floats, which can introduce precision errors.
* Introduce logic to resolve volume replica placement within EC rebalancing.
This will be used to make rebalancing logic topology-aware.
* Give shell.EcNode.dc a dedicated DataCenterId type.
2024-11-18 18:05:06 -08:00
Lisandro Pin
efdebf712e
Refactor ec.balance
logic into a weeed/shell/command_ec_common.go
… ( #6195 )
...
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function.
This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'.
* s/Balance()/EcBalance()/g
2024-11-04 17:56:20 -08:00
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
...
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
8cb42c39ad
Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs "
...
This reverts commit 2e5aa06026
, reversing
changes made to 4d414f54a2
.
2023-09-18 16:12:50 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ( #4850 )
...
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* API changes
* go mod
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2023-09-18 14:43:05 -07:00
chrislu
f9383aa726
refactor to change capacity data type
2022-10-09 18:58:10 -07:00
Ryan Russell
bd2dc6d641
refactor(shell): Decending
-> Descending
( #3675 )
...
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-14 12:06:48 -07:00
chrislu
676e27c589
shell: stop long running jobs if lock is lost
2022-08-22 14:12:23 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
justin
3551ca2fcf
enhancement: replace sort.Slice with slices.SortFunc to reduce reflection
2022-04-18 10:35:43 +08:00
chrislu
21aaa4c1f1
ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount)
...
fix https://github.com/chrislusf/seaweedfs/issues/2642
2022-02-08 01:51:13 -08:00
chrislu
f18803424a
volume.balance: add delay during tight loop
...
fix https://github.com/chrislusf/seaweedfs/issues/2637
2022-02-08 00:53:55 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
e5fc35ed0c
change server address from string to a type
2021-09-12 22:47:52 -07:00
Chris Lu
1c233ad986
refactoring
2021-02-22 00:28:42 -08:00
Chris Lu
a0c6db361c
avoid nil
2021-02-16 05:33:38 -08:00
Chris Lu
36f95e50a9
avoid possible nil disk info
2021-02-16 05:13:48 -08:00
Chris Lu
f8446b42ab
this can compile now!!!
2021-02-16 02:47:02 -08:00
Chris Lu
a595916342
shell: add volumeServer.evacuate command
2020-09-14 23:47:11 -07:00
Chris Lu
d15682b4a1
shell: volume.balance plan by ratio of fullness
2020-09-12 04:06:26 -07:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
72a64a5cf8
use the same context object in order to retry
2020-01-26 14:42:11 -08:00
Chris Lu
37b64a50b4
ec: generate and copy .vif file
2019-12-28 12:44:59 -08:00
Chris Lu
3ebeae0c0b
ec encode distribute ec data and parity shards evenly
2019-12-24 16:52:21 -08:00
Chris Lu
09ca936c78
shell: add ec.decode command
2019-12-23 12:48:20 -08:00
Chris Lu
40514c5362
add sortEcNodesByFreeslotsDecending and sortEcNodesByFreeslotsAscending
...
addressing https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-12 13:47:36 -08:00
Chris Lu
717ec47fa2
avoid double counting ec slot
...
addressing issue found in https://github.com/chrislusf/seaweedfs/issues/1111
The ec slots are already counted on the server side.
2019-11-12 09:33:51 -08:00
Chris Lu
e6ab75972f
use constant
2019-11-10 12:05:58 -08:00
Chris Lu
f9d8bd51ad
ec shard balancing
2019-06-10 21:32:56 -07:00
Chris Lu
9d9162ca35
ec.balance: collect dc rack info
2019-06-07 00:25:05 -07:00
Chris Lu
d344e0a035
fix ec related bugs
2019-06-05 23:20:26 -07:00
Chris Lu
ede876cfdb
periodic scripts exeuction from leader master
2019-06-05 01:30:24 -07:00
Chris Lu
d85b41b904
fix ec.encode not finding the local ec shards
2019-06-03 10:38:21 -07:00
Chris Lu
7e80b2b882
fix multiple bugs
2019-06-03 02:26:31 -07:00