chrislu
4fee496b49
conditional delete
2024-08-01 13:00:54 -07:00
Andrei Kvapil
a7e00d9074
Update COSI driver v0.1.1 ( #5845 )
...
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-08-01 09:47:41 -07:00
Andrei Kvapil
3b0130b070
Add seaweedfs-cosi-driver ( #5843 )
...
add: seaweedfs-cosi-driver
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-07-31 13:53:30 -07:00
wyang
31b89c1062
fsck: only check the appendNs of deleted needle ( #5841 )
...
increase fsck speed
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-31 01:12:57 -07:00
Konstantin Lebedev
b2ffcdaab2
[master] do sync grow request only if absolutely necessary ( #5821 )
...
* do sync grow request only if absolutely necessary
https://github.com/seaweedfs/seaweedfs/pull/5819
* remove check VolumeGrowStrategy Threshold on PickForWrite
* fix fmt.Errorf
2024-07-30 13:21:35 -07:00
dependabot[bot]
69bcdf470b
chore(deps): bump gocloud.dev/pubsub/rabbitpubsub from 0.37.0 to 0.38.0 ( #5829 )
...
Bumps [gocloud.dev/pubsub/rabbitpubsub](https://github.com/google/go-cloud ) from 0.37.0 to 0.38.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.37.0...v0.38.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/rabbitpubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-07-30 10:49:01 -07:00
dependabot[bot]
41ddbe329e
chore(deps): bump github.com/rclone/rclone from 1.66.0 to 1.67.0 ( #5831 )
...
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.66.0 to 1.67.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.66.0...v1.67.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-07-30 10:22:11 -07:00
Jiffs Maverick
c8506609d8
[filer] Added a version to the title of Filer UI ( #5836 )
...
Added a version to the title of Filer UI
2024-07-29 14:07:12 -07:00
Andrei Kvapil
c9e526665b
fix case for keyAlgorithm in Helm chart ( #5835 )
...
fix keyAlgorithm in Helm chart
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-07-29 12:17:43 -07:00
dependabot[bot]
ddd2ec21d0
chore(deps): bump docker/setup-buildx-action from 3.5.0 to 3.6.0 ( #5825 )
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 3.5.0 to 3.6.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](aa33708b10...3d68780484
2024-07-29 09:14:15 -07:00
Andrei Kvapil
545c3f83bf
Correct gopher on SVG logo ( #5833 )
2024-07-29 09:13:41 -07:00
dependabot[bot]
b9063404ba
chore(deps): bump github.com/redis/go-redis/v9 from 9.5.1 to 9.6.1 ( #5832 )
...
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis ) from 9.5.1 to 9.6.1.
- [Release notes](https://github.com/redis/go-redis/releases )
- [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md )
- [Commits](https://github.com/redis/go-redis/compare/v9.5.1...v9.6.1 )
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-29 09:12:30 -07:00
dependabot[bot]
cb03496c15
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.68.1 to 3.76.0 ( #5830 )
...
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.68.1 to 3.76.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.68.1...v3.76.0 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-29 09:12:17 -07:00
dependabot[bot]
f833c2b2fa
chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.37.0 to 0.38.0 ( #5828 )
...
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud ) from 0.37.0 to 0.38.0.
- [Release notes](https://github.com/google/go-cloud/releases )
- [Commits](https://github.com/google/go-cloud/compare/v0.37.0...v0.38.0 )
---
updated-dependencies:
- dependency-name: gocloud.dev/pubsub/natspubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-29 09:12:03 -07:00
Andrei Kvapil
acccae831f
Add SVG logo ( #5827 )
...
add SVG logo
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2024-07-29 08:55:59 -07:00
Konstantin Lebedev
e1f9336688
[filer] fix panic filer on webdav.maxMB ( #5822 )
...
fix panic filer on webdav.maxMB
2024-07-27 13:40:22 -07:00
wyang
4b1f539ab8
fix allocate reduplicated volumeId to different volume ( #5811 )
...
* fix allocate reduplicated volumeId to different volume
* only check barrier when read
---------
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-26 21:48:36 -07:00
wyang
c1bffca246
fix evacuate volume to different disk types ( #5530 )
...
Co-authored-by: wyang <wyang@wyangs-Air.lan>
2024-07-26 21:36:56 -07:00
chrislu
e2a07d11d5
Revert "Check ShouldGrowVolumes before returning error in assign. ( #5819 )"
...
This reverts commit 98d66338d0333cd955f7840c64ef95e3c4807a17.
2024-07-26 11:21:50 -07:00
wyang
0581ce6096
fix delete chunk failed if volumeSever specified grpc.port ( #5820 )
...
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-26 11:14:19 -07:00
chrislu
ec9455dab7
same change as #5819
...
follow up with https://github.com/seaweedfs/seaweedfs/pull/5819
2024-07-26 11:06:47 -07:00
Ruoxi
d15966ae8e
Check ShouldGrowVolumes before returning error in assign. ( #5819 )
2024-07-26 11:04:38 -07:00
Chris Lu
952afd810c
Fix dead lock ( #5815 )
...
* reduce locks to avoid dead lock
Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0
goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)
SaveContent -> MemChunk.RLock ->
ChunkedDirtyPages.saveChunkedFileIntervalToStorage
pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
fh.entryLock.Lock()
sync.(*RWMutex).Lock(0x0?)
/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
/github/workspace/weed/util/limited_executor.go:33 +0x97
On metadata update
fh.entryLock.Lock()
fh.dirtyPages.Destroy()
up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock
goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570
* use locked entry everywhere
* modifiable remote entry
* skip locking after getting lock from fhLockTable
2024-07-24 23:46:40 -07:00
dependabot[bot]
8b402fb4b9
chore(deps): bump google.golang.org/api from 0.187.0 to 0.189.0 ( #5808 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.187.0 to 0.189.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.187.0...v0.189.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 12:03:56 -07:00
dependabot[bot]
a7b48b971e
chore(deps): bump github.com/Jille/raft-grpc-transport from 1.5.0 to 1.6.1 ( #5807 )
...
chore(deps): bump github.com/Jille/raft-grpc-transport
Bumps [github.com/Jille/raft-grpc-transport](https://github.com/Jille/raft-grpc-transport ) from 1.5.0 to 1.6.1.
- [Release notes](https://github.com/Jille/raft-grpc-transport/releases )
- [Commits](https://github.com/Jille/raft-grpc-transport/compare/v1.5.0...v1.6.1 )
---
updated-dependencies:
- dependency-name: github.com/Jille/raft-grpc-transport
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 12:03:47 -07:00
dependabot[bot]
5e554582f5
chore(deps): bump go.etcd.io/etcd/client/pkg/v3 from 3.5.14 to 3.5.15 ( #5806 )
...
Bumps [go.etcd.io/etcd/client/pkg/v3](https://github.com/etcd-io/etcd ) from 3.5.14 to 3.5.15.
- [Release notes](https://github.com/etcd-io/etcd/releases )
- [Commits](https://github.com/etcd-io/etcd/compare/v3.5.14...v3.5.15 )
---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/pkg/v3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 12:03:37 -07:00
dependabot[bot]
e7cbed63a6
chore(deps): bump go.mongodb.org/mongo-driver from 1.15.0 to 1.16.0 ( #5805 )
...
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver ) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases )
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.15.0...v1.16.0 )
---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 12:03:27 -07:00
dependabot[bot]
c2a6055c04
chore(deps): bump github.com/klauspost/reedsolomon from 1.12.1 to 1.12.3 ( #5804 )
...
Bumps [github.com/klauspost/reedsolomon](https://github.com/klauspost/reedsolomon ) from 1.12.1 to 1.12.3.
- [Release notes](https://github.com/klauspost/reedsolomon/releases )
- [Commits](https://github.com/klauspost/reedsolomon/compare/v1.12.1...v1.12.3 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/reedsolomon
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 12:03:20 -07:00
dependabot[bot]
1c32ac133b
chore(deps): bump docker/setup-buildx-action from 3.4.0 to 3.5.0 ( #5803 )
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](4fd812986e...aa33708b10
2024-07-22 12:03:10 -07:00
dependabot[bot]
5e2471d790
chore(deps): bump docker/login-action from 3.2.0 to 3.3.0 ( #5802 )
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](0d4c9c5ea7...9780b0c442
2024-07-22 12:03:03 -07:00
dependabot[bot]
171e824444
chore(deps): bump docker/setup-qemu-action from 3.1.0 to 3.2.0 ( #5801 )
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](5927c834f5...49b3bc8e6b
2024-07-22 12:02:56 -07:00
dependabot[bot]
509ac40936
chore(deps): bump docker/build-push-action from 6.4.0 to 6.5.0 ( #5800 )
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 6.4.0 to 6.5.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](a254f8ca60...5176d81f87
2024-07-22 12:02:48 -07:00
chrislu
ed7e721efe
3.71
2024-07-21 21:03:27 -07:00
chrislu
9265be43c0
avoid nil
2024-07-21 21:01:29 -07:00
chrislu
4020149690
3.70
2024-07-21 20:27:39 -07:00
chrislu
9dd008f8f1
add version to filer
2024-07-21 20:13:51 -07:00
chrislu
9dcc576499
Revert "add collection for buckets"
...
This reverts commit 96af571219
2024-07-20 12:00:50 -07:00
chrislu
aed2a18223
get, set and add path conf
2024-07-20 11:27:11 -07:00
chrislu
96af571219
add collection for buckets
2024-07-17 23:14:36 -07:00
chrislu
a8fa78b892
refactoring
2024-07-17 23:14:36 -07:00
chrislu
7a75928e7d
Azure sink: avoid overwriting existing files
2024-07-17 23:14:36 -07:00
vadimartynov
86d92a42b4
Added tls for http clients ( #5766 )
...
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
chrislu
c6dec11ea5
[filer.sync] skip overwriting existing fresh entry
2024-07-16 09:38:10 -07:00
chrislu
19d14b4c71
minor
2024-07-16 09:21:01 -07:00
vadimartynov
b796c21fa9
Added loadSecurityConfigOnce ( #5792 )
2024-07-16 09:15:55 -07:00
Konstantin Lebedev
67edf1d014
[master] Do Automatic Volume Grow in background ( #5781 )
...
* Do Automatic Volume Grow in backgound
* pass lastGrowCount to master
* fix build
* fix count to uint64
2024-07-16 08:03:40 -07:00
wusong
ce61a66b65
Fix mmap write fail ( #5791 )
...
* [mount] fix GetAttr blocks count
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
* [mount] fix mmap read
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
---------
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2024-07-16 07:59:55 -07:00
Dan
794a5aa67d
Add warning with hints to why a EC shard cant be moved ( #5790 )
2024-07-15 11:17:22 -07:00
Konstantin Lebedev
a53e406c99
[master] refactor HasGrowRequest to atomic bool ( #5782 )
...
refactor HasGrowRequest to atomit bool
2024-07-15 10:51:21 -07:00
dependabot[bot]
feb911a0b4
chore(deps): bump docker/build-push-action from 6.3.0 to 6.4.0 ( #5784 )
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 6.3.0 to 6.4.0.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](1a162644f9...a254f8ca60
2024-07-15 09:55:52 -07:00
