Commit Graph

10626 Commits

Author SHA1 Message Date
chrislu
edc87c16f9 balance subscribers
need to ensure the offsets are persisted
2024-05-27 17:30:16 -07:00
chrislu
6366898af2 add market 2024-05-26 14:35:22 -07:00
chrislu
96b326a304 use type ConsumerGroupInstanceId 2024-05-26 14:09:51 -07:00
chrislu
ba778539d9 Merge branch 'master' into mq 2024-05-24 17:15:27 -07:00
chrislu
364bb6c7b4 avoid ticker leak 2024-05-24 17:15:12 -07:00
chrislu
37d1ee562d refactor 2024-05-23 09:21:48 -07:00
chrislu
cdeaaf95b4 go fmt 2024-05-23 08:25:53 -07:00
chrislu
77a7c5c2a0 Merge branch 'master' into mq 2024-05-23 08:25:27 -07:00
chrislu
ba98f02d02 go fmt 2024-05-23 08:25:16 -07:00
chrislu
d40b350f54 coordinator receives unassignment ack 2024-05-23 08:23:35 -07:00
Tomer Eskenazi
fb9fe627f6
increase helm chart version to include the emptyDir storage feature (#5613) 2024-05-23 06:12:18 -07:00
chrislu
73d008fafa Merge branch 'master' into mq 2024-05-22 20:39:31 -07:00
Johnny Cederholm
9497db71e6
Helm: Fix warning when providing resource requests/limits for S3 (#5611) 2024-05-22 13:26:09 -07:00
Tomer Eskenazi
4e00ab4942
helm chart - add support for emptyDir storage type (#5610) 2024-05-22 09:56:48 -07:00
Chris Lu
cfcf383ee7
fix wrong prefix example 2024-05-21 21:45:29 -07:00
Vlad
a9b057389f
chore: add missing concurrency help in fs.meta.load command (#5609) 2024-05-21 21:42:48 -07:00
chrislu
554ae09f82 rename 2024-05-21 10:02:07 -07:00
chrislu
6ef2d010aa rename file 2024-05-21 09:57:59 -07:00
chrislu
67e6051585 rename Coordinator to SubCoordinator 2024-05-21 09:57:45 -07:00
chrislu
b5099263a4 rename file 2024-05-21 09:56:46 -07:00
chrislu
d5abffa42c rename Balancer to PubBalancer 2024-05-21 09:56:30 -07:00
chrislu
fa98ecf71e client side stop partition subscribing if unassigned 2024-05-21 08:42:04 -07:00
chrislu
6634b42981 fix compilation 2024-05-21 08:26:55 -07:00
chrislu
2190d59bc3 refactor
ToDo
1. handle unassignment on client side
2. get notified of completed unassignment
3. in consumer_group.go, balance and send unassignment
2024-05-21 08:05:56 -07:00
chrislu
3148dec1f6 avoid concurrent processing for the same key 2024-05-20 13:29:37 -07:00
chrislu
a7d30d0705 refactor 2024-05-20 12:32:12 -07:00
chrislu
33aecb7bb4 remove ProcessorConfiguration 2024-05-20 12:28:01 -07:00
chrislu
65dd5ac6fb fix 2024-05-20 12:27:45 -07:00
chrislu
3b7f24b1fc unused 2024-05-20 12:26:54 -07:00
chrislu
f13475ac2e sending keyed offset 2024-05-20 11:09:48 -07:00
chrislu
d45b1d058d minor 2024-05-20 11:07:54 -07:00
chrislu
3622196881 go fmt 2024-05-20 11:05:18 -07:00
chrislu
5038577f7e Merge branch 'master' into mq 2024-05-20 11:04:53 -07:00
chrislu
aba212dedd Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2024-05-20 11:03:58 -07:00
chrislu
d218fe54fa go fmt 2024-05-20 11:03:56 -07:00
chrislu
7a4ace5dfb adjust logs 2024-05-20 09:34:07 -07:00
chrislu
d8ab22012a track offset 2024-05-20 09:33:37 -07:00
chrislu
2b07a40da5 add InflightMessageTracker 2024-05-20 09:19:39 -07:00
dependabot[bot]
2f474c33d0
chore(deps): bump github.com/aws/aws-sdk-go from 1.51.30 to 1.53.5 (#5608)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.51.30 to 1.53.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.51.30...v1.53.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 09:05:57 -07:00
dependabot[bot]
2be58ea767
chore(deps): bump cloud.google.com/go/storage from 1.40.0 to 1.41.0 (#5607)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.40.0 to 1.41.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.40.0...spanner/v1.41.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 09:05:50 -07:00
dependabot[bot]
f483d8b8f4
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.26.1 to 1.27.0 (#5606)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.26.1 to 1.27.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.26.1...v1.27.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:56:21 -07:00
dependabot[bot]
50ec3d7385
chore(deps): bump google.golang.org/api from 0.177.0 to 0.181.0 (#5605)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.177.0 to 0.181.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.177.0...v0.181.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:56:12 -07:00
dependabot[bot]
07a289785a
chore(deps): bump google.golang.org/protobuf from 1.34.0 to 1.34.1 (#5604)
Bumps google.golang.org/protobuf from 1.34.0 to 1.34.1.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:55:56 -07:00
chrislu
a2885512e1 fix 2024-05-19 15:06:33 -07:00
chrislu
8d02712c63 ConcurrentPartitionLimit 2024-05-19 14:55:03 -07:00
chrislu
8ce2cced47 less logs 2024-05-19 14:54:46 -07:00
chrislu
c6db3f31a1 start consuming ASAP 2024-05-19 14:52:38 -07:00
chrislu
c21b32dd20 logs 2024-05-19 11:02:17 -07:00
chrislu
b1871427c3 persist consumer group offset
1. use one follower
2. read write consumer group offset
2024-05-19 00:46:12 -07:00
sb
a3a00d9499
Feature/mongodb security (#5602) 2024-05-18 12:19:10 -07:00