Commit Graph

10603 Commits

Author SHA1 Message Date
chrislu
2190d59bc3 refactor
ToDo
1. handle unassignment on client side
2. get notified of completed unassignment
3. in consumer_group.go, balance and send unassignment
2024-05-21 08:05:56 -07:00
chrislu
3148dec1f6 avoid concurrent processing for the same key 2024-05-20 13:29:37 -07:00
chrislu
a7d30d0705 refactor 2024-05-20 12:32:12 -07:00
chrislu
33aecb7bb4 remove ProcessorConfiguration 2024-05-20 12:28:01 -07:00
chrislu
65dd5ac6fb fix 2024-05-20 12:27:45 -07:00
chrislu
3b7f24b1fc unused 2024-05-20 12:26:54 -07:00
chrislu
f13475ac2e sending keyed offset 2024-05-20 11:09:48 -07:00
chrislu
d45b1d058d minor 2024-05-20 11:07:54 -07:00
chrislu
3622196881 go fmt 2024-05-20 11:05:18 -07:00
chrislu
5038577f7e Merge branch 'master' into mq 2024-05-20 11:04:53 -07:00
chrislu
aba212dedd Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2024-05-20 11:03:58 -07:00
chrislu
d218fe54fa go fmt 2024-05-20 11:03:56 -07:00
chrislu
7a4ace5dfb adjust logs 2024-05-20 09:34:07 -07:00
chrislu
d8ab22012a track offset 2024-05-20 09:33:37 -07:00
chrislu
2b07a40da5 add InflightMessageTracker 2024-05-20 09:19:39 -07:00
dependabot[bot]
2f474c33d0
chore(deps): bump github.com/aws/aws-sdk-go from 1.51.30 to 1.53.5 (#5608)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.51.30 to 1.53.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.51.30...v1.53.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 09:05:57 -07:00
dependabot[bot]
2be58ea767
chore(deps): bump cloud.google.com/go/storage from 1.40.0 to 1.41.0 (#5607)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.40.0 to 1.41.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.40.0...spanner/v1.41.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 09:05:50 -07:00
dependabot[bot]
f483d8b8f4
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.26.1 to 1.27.0 (#5606)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.26.1 to 1.27.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.26.1...v1.27.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:56:21 -07:00
dependabot[bot]
50ec3d7385
chore(deps): bump google.golang.org/api from 0.177.0 to 0.181.0 (#5605)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.177.0 to 0.181.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.177.0...v0.181.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:56:12 -07:00
dependabot[bot]
07a289785a
chore(deps): bump google.golang.org/protobuf from 1.34.0 to 1.34.1 (#5604)
Bumps google.golang.org/protobuf from 1.34.0 to 1.34.1.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 08:55:56 -07:00
chrislu
a2885512e1 fix 2024-05-19 15:06:33 -07:00
chrislu
8d02712c63 ConcurrentPartitionLimit 2024-05-19 14:55:03 -07:00
chrislu
8ce2cced47 less logs 2024-05-19 14:54:46 -07:00
chrislu
c6db3f31a1 start consuming ASAP 2024-05-19 14:52:38 -07:00
chrislu
c21b32dd20 logs 2024-05-19 11:02:17 -07:00
chrislu
b1871427c3 persist consumer group offset
1. use one follower
2. read write consumer group offset
2024-05-19 00:46:12 -07:00
sb
a3a00d9499
Feature/mongodb security (#5602) 2024-05-18 12:19:10 -07:00
sb
3fae87632f
add filer configuration options for username/password and tls (#5601) 2024-05-18 09:02:03 -07:00
chrislu
8d5bb7420d rename 2024-05-17 18:21:21 -07:00
chrislu
94742b8ace add subscribe follower 2024-05-17 18:17:23 -07:00
Johnny Cederholm
8c59348d28
Add ability to configure security context for the different components (#5600) 2024-05-17 13:06:00 -07:00
Riccardo Bertossa
f6e8a9bf9c
added s3 iam DeleteBucket permission management (#5599) 2024-05-17 04:54:09 -07:00
chrislu
279cb2b85e consumer acks received messages 2024-05-16 20:28:19 -07:00
chrislu
faffb2973c adjust errors 2024-05-16 11:02:48 -07:00
chrislu
759a8e50cf remove logs 2024-05-14 23:22:51 -07:00
chrislu
1f20178ded subscriber receives partitions and dispatch to processors 2024-05-14 23:22:43 -07:00
chrislu
972e9faaa2 move initial assignment to rebalance logic 2024-05-14 09:09:36 -07:00
chrislu
6e5075e14e move read write topic config into filer client accessor 2024-05-14 08:50:17 -07:00
Martin Stiborský
54f3913bed
[s3] Fixed s3 replication by sending content-md as base64 (#5596) 2024-05-14 06:48:24 -07:00
Gregor Tudan
31653e4b1e
Update the Grafana panels using the to the new time series (#5595) 2024-05-14 05:33:29 -07:00
Gregor Tudan
1db3bb9a9c
Helm: Fix warning when rendering with secretExtraEnvVars set (#5594) 2024-05-14 05:30:52 -07:00
Gregor Tudan
262fe2c723
Helm: Allow adding additional labels to the service monitors (#5593) 2024-05-14 05:29:31 -07:00
chrislu
ac63f2b5a1 refactor 2024-05-13 22:03:57 -07:00
chrislu
844a1a0192 rename 2024-05-13 19:24:47 -07:00
chrislu
372bd8d71d consumer instance passing MaxPartitionCount to coordinator 2024-05-13 18:30:10 -07:00
chrislu
2142842f82 refactor 2024-05-13 18:25:38 -07:00
Gregor Tudan
3f3f0ec17f
Allow selecting the namespace visualized by the grafana dashboard (#5591) 2024-05-13 10:16:58 -07:00
dependabot[bot]
c23e9f96a5
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.65.0 to 3.66.3 (#5588)
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.65.0 to 3.66.3.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.65.0...v3.66.3)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-13 10:05:50 -07:00
dependabot[bot]
66dfeb7ba2
chore(deps): bump github.com/shirou/gopsutil/v3 from 3.24.3 to 3.24.4 (#5587)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.24.3 to 3.24.4.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.24.3...v3.24.4)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-13 10:05:31 -07:00
dependabot[bot]
4bc7ff604a
chore(deps): bump cloud.google.com/go/pubsub from 1.37.0 to 1.38.0 (#5586)
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.37.0 to 1.38.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.37.0...pubsub/v1.38.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-13 10:05:21 -07:00