mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-23 18:49:17 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,140 Commits 32 Branches 296 Tags 147 MiB
1c143cfcea
Commit Graph

11140 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu
1c143cfcea display actual offset value 2024-11-21 00:38:03 -08:00
chrislu
c9f3448692 ReadAt may return io.EOF t end of file 
related to https://github.com/seaweedfs/seaweedfs/issues/6219
 2024-11-21 00:37:38 -08:00
chrislu
be09d08eb6 adjust error message 2024-11-20 23:49:01 -08:00
chrislu
6b2612690b consistent error message if auth is not setup 
fix https://github.com/seaweedfs/seaweedfs/issues/6262
 2024-11-20 23:11:18 -08:00
Konstantin Lebedev
254ed8897e
[shell] add noLock param for volume.move (#6261) 2024-11-20 08:35:24 -08:00
chrislu
96f1a2e01a adjust error message 2024-11-19 14:16:06 -08:00
dependabot[bot]
bb37886f6a
chore(deps): bump github.com/rclone/rclone from 1.68.1 to 1.68.2 (#6260) 
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.68.1 to 1.68.2.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.68.1...v1.68.2)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-19 13:44:32 -08:00
chrislu
e28f55eb08 typo 2024-11-19 08:32:31 -08:00
chrislu
98f03862aa rename 2024-11-19 08:31:54 -08:00
chrislu
07cf8cf22d minor 2024-11-19 08:31:33 -08:00
Lisandro Pin
0d5393641e
Unify usage of shell.EcNode.dc as DataCenterId. (#6258) 2024-11-19 06:33:18 -08:00
chrislu
2caa0e3741 java 3.80 2024-11-18 22:41:59 -08:00
chrislu
bc7650bd61 adds more info on skipped volumes 2024-11-18 18:25:44 -08:00
Lisandro Pin
f2db746690
Introduce logic to resolve volume replica placement within EC rebalancing. (#6254) 
* Rename `command_ec_encode_test.go` to `command_ec_common_test.go`.

All tests defined in this file are now for `command_ec_common.go`.

* Minor code cleanups.

- Fix broken `ec.balance` test.
- Rework integer ceiling division to not use floats, which can introduce precision errors.

* Introduce logic to resolve volume replica placement within EC rebalancing.

This will be used to make rebalancing logic topology-aware.

* Give shell.EcNode.dc a dedicated DataCenterId type.
 2024-11-18 18:05:06 -08:00
chrislu
7b3c0e937f 3.80 2024-11-18 09:48:33 -08:00
dependabot[bot]
180f267867
chore(deps): bump github.com/linxGnu/grocksdb from 1.9.5 to 1.9.7 (#6249) 
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb) from 1.9.5 to 1.9.7.
- [Release notes](https://github.com/linxGnu/grocksdb/releases)
- [Commits](https://github.com/linxGnu/grocksdb/compare/v1.9.5...v1.9.7)

---
updated-dependencies:
- dependency-name: github.com/linxGnu/grocksdb
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-18 09:37:06 -08:00
dependabot[bot]
b21f599cde
chore(deps): bump modernc.org/sqlite from 1.33.1 to 1.34.1 (#6251) 
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.33.1 to 1.34.1.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.1...v1.34.1)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-18 09:36:52 -08:00
dependabot[bot]
3fba0adc9d
chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.0 to 1.28.4 (#6252) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/config

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.28.0 to 1.28.4.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.28.0...config/v1.28.4)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-18 09:36:43 -08:00
dependabot[bot]
f8f162aa72
chore(deps): bump go.etcd.io/etcd/client/v3 from 3.5.16 to 3.5.17 (#6253) 
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.16 to 3.5.17.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.16...v3.5.17)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-18 09:36:31 -08:00
zemul
6d3e38f439
Adjust threshold config (#6246) 
* fix:mount deadlock

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2024-11-17 20:51:26 -08:00
Kevin Bulteel
4a0dc883cf
Fix/parse upload filename (#6241) 
* fix: parse filename in PUT + refactor

* fix: master iu public url with http

* fix: better parsing and handle disposition header

* fix: take mime type from file extension if not set
 2024-11-15 11:25:18 -08:00
Omar Hamadeh
8ebbf37e8e
Add a loop to wait for master and filer services in post-install-bucket-hook (#6240) 
Add a loop to wait for master and filer services before creating buckets
 2024-11-15 11:15:33 -08:00
fo40225
008ac38ebc
fix java.lang.IllegalArgumentException: Comparison method violates its general contract! (#6239) 2024-11-14 17:26:50 -08:00
Max Denushev
a5fe6e21bc
feat(filer.backup): add ignore errors option (#6235) 
* feat(filer.backup): add ignore errors option

* feat(filer.backup): fix 404 error wrap

* feat(filer.backup): fix wrapping function

* feat(filer.backup): fix wrapping errors in genProcessFunction

* Update weed/command/filer_backup.go

* Update weed/command/filer_backup.go

* Update weed/command/filer_backup.go

---------

Co-authored-by: Max Denushev <denushev@tochka.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-11-14 08:40:55 -08:00
Eugeniy E. Mikhailov
3003c9e17e
added backward compatible CRC check and extra explanatory comments (#6233) 2024-11-12 13:56:14 -08:00
dependabot[bot]
31d8907e77
chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (#6231) 
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.0...v4.5.1)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-11 11:54:07 -08:00
dependabot[bot]
7b0a37312d
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.2 to 1.66.3 (#6225) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.66.2 to 1.66.3.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.66.2...service/s3/v1.66.3)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-11 11:15:24 -08:00
dependabot[bot]
623fc3a4fc
chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.3 to 1.32.4 (#6227) 
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.32.3 to 1.32.4.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.3...v1.32.4)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-11 11:15:09 -08:00
dependabot[bot]
aa31c5422b
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.89.2 to 3.91.0 (#6228) 
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.89.2 to 3.91.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.89.2...v3.91.0)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-11 11:15:00 -08:00
dependabot[bot]
6232041c64
chore(deps): bump golang.org/x/tools from 0.26.0 to 0.27.0 (#6229) 
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-11 11:14:48 -08:00
Chris Lu
72b14a451e
delete aborted ec shards from both source and target servers (#6221) 
fix https://github.com/seaweedfs/seaweedfs/issues/6205#issuecomment-2465004586
 2024-11-09 11:32:08 -08:00
Nathan Dorsey
3400962877
[helm] fix s3 servicemonitor label matching (#6218)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
helm: lint and test charts / lint-test (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
2024-11-08 12:04:40 -08:00
chrislu
113c9ce6a8 remove the direct_io flag, as it is not well-supported on macOS 2024-11-07 14:18:59 -08:00
chrislu
88a1eee31c adjust logs 2024-11-07 13:00:12 -08:00
chrislu
62c4980754 add debug info 2024-11-06 11:59:03 -08:00
Konstantin Lebedev
9a5d3e7b31
[shell] add admin noLock for balance (#6209) 
add admin noLock for balance
 2024-11-05 19:09:22 -08:00
Bruce
15fc58bfd9
remove dup code for section.reader access (#6169) 2024-11-05 19:08:59 -08:00
chrislu
706d1a8e22 fix tests 2024-11-05 08:52:07 -08:00
chrislu
90acfd9394 remove case when read request is out side of the file size 2024-11-05 08:42:44 -08:00
chrislu
6161d0b05d a bit optimization 2024-11-05 08:41:14 -08:00
chrislu
98b519b113 fix FUSE mount on mac 2024-11-05 08:28:54 -08:00
Lisandro Pin
efdebf712e
Refactor ec.balance logic into a weeed/shell/command_ec_common.go… (#6195) 
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function.

This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'.

* s/Balance()/EcBalance()/g
 2024-11-04 17:56:20 -08:00
Chris Lu
dc784bf217
merge current message queue code changes (#6201) 
* listing files to convert to parquet

* write parquet files

* save logs into parquet files

* pass by value

* compact logs into parquet format

* can skip existing files

* refactor

* refactor

* fix compilation

* when no partition found

* refactor

* add untested parquet file read

* rename package

* refactor

* rename files

* remove unused

* add merged log read func

* parquet wants to know the file size

* rewind by time

* pass in stop ts

* add stop ts

* adjust log

* minor

* adjust log

* skip .parquet files when reading message logs

* skip non message files

* Update subscriber_record.go

* send messages

* skip message data with only ts

* skip non log files

* update parquet-go package

* ensure a valid record type

* add new field to a record type

* Update read_parquet_to_log.go

* fix parquet file name generation

* separating reading parquet and logs

* add key field

* add skipped logs

* use in memory cache

* refactor

* refactor

* refactor

* refactor, and change compact log

* refactor

* rename

* refactor

* fix format

* prefix v to version directory
 2024-11-04 12:08:25 -08:00
dependabot[bot]
ffe908371d
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.61.2 to 1.66.2 (#6197) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.61.2 to 1.66.2.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.2...service/s3/v1.66.2)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-11-04 09:34:36 -08:00
dependabot[bot]
92366134e3
chore(deps): bump github.com/gocql/gocql from 1.6.0 to 1.7.0 (#6196) 
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/gocql/gocql/releases)
- [Changelog](https://github.com/apache/cassandra-gocql-driver/blob/trunk/CHANGELOG.md)
- [Commits](https://github.com/gocql/gocql/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/gocql/gocql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-04 09:26:52 -08:00
dependabot[bot]
e2d30cc3bc
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.39.0 to 0.40.0 (#6198) 
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud) from 0.39.0 to 0.40.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.39.0...v0.40.0)

---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-04 09:26:27 -08:00
dependabot[bot]
d185e6beea
chore(deps): bump golang.org/x/tools from 0.25.0 to 0.26.0 (#6199) 
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.25.0 to 0.26.0.
- [Release notes](https://github.com/golang/tools/releases)
- [Commits](https://github.com/golang/tools/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: golang.org/x/tools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-04 09:26:17 -08:00
dependabot[bot]
f67b5b20ca
chore(deps): bump cloud.google.com/go/pubsub from 1.43.0 to 1.45.1 (#6200) 
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.43.0 to 1.45.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.1)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-11-04 09:25:59 -08:00
wyang
a7973ed7d1
fix deadlock hang when broadcast to clients (#6184) 
fix deadlock when broadcast to clients

when master thransfer leader, the old master will disconnect with all
filers and volumeServers, if the cluster is a big , the broadcast
messages may be more big than the max of the channel len 100, then if the
KeepConnect was not listen on the channel in disconnect, it will
deadlock. and the whole cluster will not serve!
 2024-11-03 23:20:48 -08:00
chrislu
0f2c3648dc fix mount: cannot locate OSXFUSE 
addressing https://github.com/seaweedfs/seaweedfs/discussions/6193
 2024-11-03 13:08:19 -08:00