Commit Graph

10197 Commits

Author SHA1 Message Date
Chris Lu
c471265837 build with pub sub clients for testing 2024-02-05 16:47:11 -08:00
Chris Lu
4dc6681833 proxy requests to lock owner 2024-02-05 16:46:40 -08:00
Chris Lu
0d74ac9224 fix divide by zero 2024-02-04 13:38:00 -08:00
Chris Lu
f9131de43f fix lock TTL 2024-02-04 12:47:21 -08:00
Chris Lu
44b49bec99 less verbose 2024-02-04 12:47:07 -08:00
Chris Lu
53612b770c Merge branch 'master' into mq-subscribe 2024-02-04 10:44:01 -08:00
Chris Lu
56df44845f fix when two consecutive chunks with the same modified_ts_ns
fix https://github.com/seaweedfs/seaweedfs/issues/5276
2024-02-04 09:22:24 -08:00
Chris Lu
0a12301b3d avoid too large expiration time 2024-02-04 09:20:21 -08:00
sxlehua
0b49c16361
Adapt S3 POST ContentType (#5275) 2024-02-03 22:33:34 -08:00
Chris Lu
1b5ba4190c avoid timestamp overflow 2024-02-03 13:45:59 -08:00
chrislu
d41792461c lock returns host and owner 2024-02-02 15:54:57 -08:00
chrislu
d30150dde1 use a short-lived lock 2024-02-01 23:01:44 -08:00
chrislu
0aed16a9c4 add make brokers 2024-01-31 21:31:10 -08:00
cuisongliu
c5440055fd
helm using external master address (#5267)
using external master address
2024-01-30 20:55:36 -08:00
cuisongliu
9abd3a90c3
helm enable resource for template (#5268)
enable resource for template
2024-01-30 20:54:50 -08:00
cuisongliu
f8c4fb3ebd
fix helm install error for No such file or directory (#5250)
* weed/shell: Cluster check other disk types (#5245)

* week/shell: Cluster check other disk types

The `cluster.check` command only took the empty (`""`) and `hdd` disk types
into consideration, but a cluster with only `ssd` or `nvme` disk types would be
equally valid.

This commit simply checks that _any_ disk type is defined, and that some
volumes are available for it.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Replace loop that copies slice

Use the following construct instead of a `for` loop:

```golang
x = append(x, y...)
```

See https://staticcheck.dev/docs/checks#S1011.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Check disk types when filer is in use

Filer stores its metadata logs in generic (i.e. `""`) or HDD disk type volumes,
so make sure those disk types exist and have volumes associated with them when
Filer is deployed in the cluster.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

---------

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Cluster check other disk types (#5245)

* week/shell: Cluster check other disk types

The `cluster.check` command only took the empty (`""`) and `hdd` disk types
into consideration, but a cluster with only `ssd` or `nvme` disk types would be
equally valid.

This commit simply checks that _any_ disk type is defined, and that some
volumes are available for it.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Replace loop that copies slice

Use the following construct instead of a `for` loop:

```golang
x = append(x, y...)
```

See https://staticcheck.dev/docs/checks#S1011.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Check disk types when filer is in use

Filer stores its metadata logs in generic (i.e. `""`) or HDD disk type volumes,
so make sure those disk types exist and have volumes associated with them when
Filer is deployed in the cluster.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

---------

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* fix helm install error for No such file or directory

---------

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
Co-authored-by: Benoît Knecht <bknecht@protonmail.ch>
2024-01-30 20:46:22 -08:00
Sébastien
d6e0cae6e7
fix: publish helm chart at new release (#5265)
fix: publish helm chart at new release (tags)

instead of each push to master, to avoid silent breaking changes
2024-01-30 08:59:38 -08:00
chrislu
c1bb76d709 fix nil error 2024-01-30 08:21:42 -08:00
chrislu
ac50d8a822 re-lock if the lock owner is not found 2024-01-29 22:46:23 -08:00
Benoît Knecht
56287bd07d
weed/shell: Cluster check other disk types (#5245)
* week/shell: Cluster check other disk types

The `cluster.check` command only took the empty (`""`) and `hdd` disk types
into consideration, but a cluster with only `ssd` or `nvme` disk types would be
equally valid.

This commit simply checks that _any_ disk type is defined, and that some
volumes are available for it.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Replace loop that copies slice

Use the following construct instead of a `for` loop:

```golang
x = append(x, y...)
```

See https://staticcheck.dev/docs/checks#S1011.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

* weed/shell: Check disk types when filer is in use

Filer stores its metadata logs in generic (i.e. `""`) or HDD disk type volumes,
so make sure those disk types exist and have volumes associated with them when
Filer is deployed in the cluster.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

---------

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2024-01-29 10:36:37 -08:00
Sébastien
0775d05a23
fix: http range request return status 500 (#5251)
When volume server unavailable for at least one chunk; was returning status 206.

Split `StreamContent` in two parts,
- first prepare, to get chunk info and return stream function
- then write chunk, with that stream function

That allow to catch error in first step before setting response status code in `processRangeRequest`
2024-01-29 10:35:52 -08:00
dependabot[bot]
e5c0680dbc
Bump github.com/shirou/gopsutil/v3 from 3.23.11 to 3.23.12 (#5255)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.23.11 to 3.23.12.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.23.11...v3.23.12)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 10:29:39 -08:00
dependabot[bot]
e04264114b
Bump github.com/prometheus/client_golang from 1.17.0 to 1.18.0 (#5256)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 10:29:28 -08:00
dependabot[bot]
a0234afdd4
Bump github.com/klauspost/reedsolomon from 1.11.8 to 1.12.1 (#5257)
Bumps [github.com/klauspost/reedsolomon](https://github.com/klauspost/reedsolomon) from 1.11.8 to 1.12.1.
- [Release notes](https://github.com/klauspost/reedsolomon/releases)
- [Commits](https://github.com/klauspost/reedsolomon/compare/v1.11.8...v1.12.1)

---
updated-dependencies:
- dependency-name: github.com/klauspost/reedsolomon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 10:29:19 -08:00
dependabot[bot]
4ea41994bf
Bump github.com/rdleal/intervalst from 1.2.1 to 1.2.2 (#5258)
Bumps [github.com/rdleal/intervalst](https://github.com/rdleal/intervalst) from 1.2.1 to 1.2.2.
- [Commits](https://github.com/rdleal/intervalst/compare/v1.2.1...v1.2.2)

---
updated-dependencies:
- dependency-name: github.com/rdleal/intervalst
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 10:29:09 -08:00
dependabot[bot]
e107800a73
Bump wangyoucao577/go-release-action from 1.42 to 1.46 (#5254) 2024-01-29 07:27:17 -08:00
spastorclovr
5fa8f40fdf
Fix/bad mount path better default value (#5253) 2024-01-29 07:11:37 -08:00
chrislu
0b2e5ddc7c wait 3 seconds before shutting down publish client, to wait for all messages to be received 2024-01-28 22:04:42 -08:00
chrislu
545d5fbdf6 unused code 2024-01-28 22:03:13 -08:00
chrislu
a507069d19 lock for creating local partition 2024-01-28 15:55:26 -08:00
chrislu
9e6ea80dff read existing topic conf 2024-01-28 14:27:52 -08:00
chrislu
a297247137 wait for each publish clients 2024-01-28 14:27:32 -08:00
chrislu
b6c5e57c30 read will block if no items 2024-01-28 13:10:34 -08:00
chrislu
0bf5424a2e adjust visibility 2024-01-28 13:09:30 -08:00
chrislu
dedfd31dfb refactor 2024-01-28 12:30:08 -08:00
chrislu
cbf750a31f refactor 2024-01-28 12:23:20 -08:00
chrislu
b51dfe2bff wait for publishing clients 2024-01-28 12:06:58 -08:00
chrislu
f8c55f101e clean up 2024-01-28 12:06:30 -08:00
chrislu
d40225ecd2 return the lookup results 2024-01-28 12:05:44 -08:00
chrislu
841fafd0a8 publish to input buffer
currently, the input buffer may not exist when start to publish
2024-01-27 23:43:22 -08:00
chrislu
5dc1362bdc close the input 2024-01-27 16:12:49 -08:00
chrislu
fa835c9319 Update buffered_queue_test.go 2024-01-27 14:46:12 -08:00
Chris Lu
b6de35cdb2 moved. there are some deadlock. WIP 2024-01-27 13:51:19 -08:00
chrislu
91af1f3069 schedule jobs 2024-01-26 14:09:57 -08:00
Benoît Knecht
f6e39139d8
weed/storage: Fix volume info file permissions (#5246) 2024-01-26 08:31:27 -08:00
chrislu
08c5fba825 adds a buffered queue 2024-01-25 20:22:41 -08:00
chrislu
cb50b720fd Revert "Fix weed fix"
This reverts commit 6c1ce18541.
2024-01-25 08:09:36 -08:00
Benoît Knecht
1f08010ef0
weed/shell: Cleanup volume balance logic (#5241) 2024-01-25 06:19:58 -08:00
Konstantin Lebedev
0f8e76bbd6
fix: clean metric MasterReplicaPlacementMismatch for unregister volume (#5239) 2024-01-25 00:23:24 -08:00
Benoît Knecht
a6aee847b9
weed/shell: Fix volume.balance logic (#5238) 2024-01-25 00:17:32 -08:00