coffeecloudgit
6cbce110d5
Update volume_grpc_erasure_coding.go ( #6073 )
2024-09-27 07:02:30 -07:00
Neo
aa61824442
master:fix empty target in Build() ( #6069 )
2024-09-27 00:45:18 -07:00
Max Denushev
f1e700ce2f
Fix/copy before delete replication ( #6064 )
...
* fix(shell): volume.fix.replication misplaced volumes unsatisfying replication factor
* fix(shell): simplify replication check
* fix(shell): add test for satisfyReplicaCurrentLocation
2024-09-26 08:34:13 -07:00
steve.wei
ae8c6d95a6
fix: Prevent potential metadata change events from being lost. ( #6066 )
2024-09-26 08:31:57 -07:00
LHHDZ
4dc33cc143
fix unclaimed spaces calculation when volumePreallocate is enabled ( #6063 )
...
the calculation of `unclaimedSpaces` only needs to subtract `unusedSpace` when `preallocate` is not enabled.
Signed-off-by: LHHDZ <shichanglin5@qq.com>
2024-09-24 23:04:18 -07:00
Max Denushev
d056c0ddf2
fix(volume): don't persist RO state in specific cases ( #6058 )
...
* fix(volume): don't persist RO state in specific cases
* fix(volume): writable always persist
2024-09-24 16:15:54 -07:00
zemul
028ebb1d0d
Feat:merge small chunk ( #6049 )
...
* fix:mount deadlock
* feat: merge small chunk
* adjust MergeChunkMinCount
* fix
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2024-09-23 08:51:03 -07:00
chrislu
4ec6eab004
fix max volume count auto setting
...
related to https://github.com/seaweedfs/seaweedfs/issues/6041#issuecomment-2360448182
2024-09-19 09:13:08 -07:00
dsd
a3572999bb
Vol check disk bug ( #6044 )
...
* fix volume.check.disk
* ensure multiple replica sync
* add comment
---------
Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com>
2024-09-19 08:32:22 -07:00
Konstantin Lebedev
6d4f5c94ac
[volume] remove truncate idx file if size not healthy ( #6043 )
2024-09-19 01:04:08 -07:00
chrislu
50e22864cf
refactor
2024-09-17 23:10:41 -07:00
chrislu
99ae99e3b9
fix nil filer conf
...
fix https://github.com/seaweedfs/seaweedfs/issues/6034
2024-09-17 11:07:29 -07:00
Konstantin Lebedev
2b13d2c083
[filer] fix return 204 for DELETE on entry not found ( #6023 )
...
fix return 204 for DELETE on entry not found
2024-09-16 23:11:03 -07:00
Guang Jiong Lou
6c986e9d70
improve worm support ( #5983 )
...
* improve worm support
Signed-off-by: lou <alex1988@outlook.com>
* worm mode in filer
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* move to fs configure
Signed-off-by: lou <alex1988@outlook.com>
* remove flag
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* support worm hardlink
Signed-off-by: lou <alex1988@outlook.com>
* update after review
Signed-off-by: lou <alex1988@outlook.com>
* typo
Signed-off-by: lou <alex1988@outlook.com>
* sync filer conf
Signed-off-by: lou <alex1988@outlook.com>
---------
Signed-off-by: lou <alex1988@outlook.com>
2024-09-16 21:02:21 -07:00
Guang Jiong Lou
367e7638d7
fix invalid file read ( #6024 )
2024-09-16 08:45:50 -07:00
Eugeniy E. Mikhailov
0da9e40abb
Feature: compress before encryption ( #6013 )
2024-09-15 08:47:53 -07:00
Bruce
5428229347
fix file read crash ( #6021 )
2024-09-14 08:33:35 -07:00
zouyixiong
881c9a009e
[master] master missing start LoopPushingMetric routine fixed. ( #6018 )
2024-09-13 20:01:34 -07:00
Aleksey Kosov
7340c62c47
changing FindEntry error handling in cassandra store ( #6015 )
2024-09-13 06:53:04 -07:00
chrislu
a9c9e1bcb3
refactor
2024-09-12 22:45:30 -07:00
chrislu
6063a889ed
3.73
2024-09-12 13:11:55 -07:00
Eugeniy E. Mikhailov
dab0bb8097
Feature limit caching to prescribed number of bytes per file ( #6009 )
...
* feature: we can check if a fileId is already in the cache
We using this to protect cache from adding the same needle to
the cache over and over.
* fuse mount: Do not start dowloader if needle is already in the cache
* added maxFilePartSizeInCache property to ChunkCache
If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).
* feature: for large files put in cache no more than prescribed number of bytes
Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.
* added dummy mock methods to satisfy interfaces of ChunkCache
2024-09-11 21:09:20 -07:00
dsd
151f2ff7a9
prevent dead loop for followers of master node ( #6007 )
...
* prevent dead loop for followers of master node
* fix comments
---------
Co-authored-by: levi <344887649@foxmail.com>
2024-09-11 10:24:52 -07:00
dsd
3b840c20e3
change math/rand => math/rand/v2 in volume_layout.go where is a perfo… ( #6006 )
2024-09-11 07:39:40 -07:00
Eugeniy E. Mikhailov
c04edeed68
bug fix in the data received from cache processing ( #6002 )
...
The patch addresses #3745 .
The cache should return the exact amount of data requested by the buffer.
By construction of the cache it is always all requested data range
or we have error happening.
The old use of minsize miscalculate the requested data size,
if non zero offset is requested.
2024-09-10 13:30:18 -07:00
chrislu
d660d5c7d4
increasing default cache size
2024-09-10 10:30:19 -07:00
chrislu
f23015ac5b
weed mount caching
2024-09-10 10:30:05 -07:00
chrislu
93decd1114
remove unused
2024-09-10 10:29:25 -07:00
chrislu
69fcdd0840
adjust logging
2024-09-10 10:28:49 -07:00
Konstantin Lebedev
34bbaa2cdd
[master] process grow request with must grow ( #5999 )
...
process grow request with must grow
2024-09-09 23:45:02 -07:00
Konstantin Lebedev
15965f7c54
[shell] fix volume grow in shell ( #5992 )
...
* fix volume grow in shell
* revert add Async
* check available volume space
* create a VolumeGrowRequest and remove unnecessary fields
2024-09-09 11:42:56 -07:00
chrislu
43d9f68219
more consistent server header for s3 apis
2024-09-09 09:07:58 -07:00
zouyixiong
d8e18a8eb7
bug fixed to filer store elastic deleteEntry ( #5988 )
2024-09-09 02:15:25 -07:00
chrislu
ff3d46637d
better logging for volume growth
2024-09-07 12:38:34 -07:00
chrislu
accba3070a
refactor
2024-09-07 11:54:22 -07:00
Eugeniy E. Mikhailov
d6b0e0ff1c
Remove "Content-Length" header if http.Error is issued ( #5981 )
2024-09-05 17:31:28 -07:00
Eugeniy E. Mikhailov
bc01f09e37
Do CRC check if the buffer contains the full needle data before it is sent ( #5980 )
2024-09-05 17:14:55 -07:00
Bruce
f9e141a412
persist readonly state to volume info ( #5977 )
2024-09-05 07:58:24 -07:00
Bruce
310d41998d
Skip volume data integrity check for volume which has remote file ( #5976 )
2024-09-04 21:25:04 -07:00
Konstantin Lebedev
16984ef07a
[s3] reducing the number of metrics for 403 ( #5961 )
...
reducing the number of metrics
2024-09-04 11:06:54 -07:00
Konstantin Lebedev
67a252ee8a
[master] refactor func ShouldGrowVolumes ( #5884 )
2024-09-04 08:16:44 -07:00
chrislu
eb02946c97
support write once read many
...
fix https://github.com/seaweedfs/seaweedfs/issues/5954
2024-09-04 02:25:07 -07:00
chrislu
18afdb15b6
Revert "weed mount, weed dav add option to force cache"
...
This reverts commit 7367b976b0
2024-09-04 01:38:29 -07:00
chrislu
7367b976b0
weed mount, weed dav add option to force cache
2024-09-04 01:19:14 -07:00
chrislu
66ac82bb8f
default cacheDirWrite to cacheDir
2024-09-04 00:05:58 -07:00
chrislu
8d5a6d7e74
fix for spreading ec shards
2024-09-03 22:30:11 -07:00
chrislu
8e4bffc66b
copy ec shards to disks already having ec volumes
...
fix https://github.com/seaweedfs/seaweedfs/issues/5615
2024-09-03 21:19:10 -07:00
wyang
361e7ebc74
fix volume.move io.EOF ( #5960 )
2024-09-02 06:17:41 -07:00
Eugeniy E. Mikhailov
282867a71e
Show volume server which failed to verify a chunk ( #5958 )
2024-09-01 21:13:22 -07:00
Eugeniy E. Mikhailov
220061573d
update metric for ErrorCRC if it happens during volume readNeedleDataInto ( #5957 )
2024-09-01 20:30:09 -07:00
chrislu
3c0854e986
unnecessary skipping
2024-08-30 16:38:01 -07:00
chrislu
654b8210f7
parameter name
2024-08-30 14:52:07 -07:00
chrislu
8679870008
fix typo
2024-08-30 14:51:55 -07:00
chrislu
22d5d2ea96
adjust warning log
2024-08-30 14:51:42 -07:00
chrislu
a4b25a642d
math/rand => math/rand/v2
2024-08-29 09:52:21 -07:00
chrislu
ded5e084ea
ensure none zero lastGrowCount
2024-08-27 09:03:11 -07:00
chrislu
0cf2c15828
rename
2024-08-27 09:02:48 -07:00
chrislu
bebc385607
randomly pick one replica to write
2024-08-27 09:02:39 -07:00
chrislu
3b7bb62e38
logs on error
2024-08-26 09:09:11 -07:00
Shaun Alexander
0d435f9da4
filer.remote.gateway not filer.remote.sync for filer.remote.gateway help string. ( #5935 )
2024-08-23 20:42:59 -07:00
Shaun Alexander
766290411b
filer.remote.gateway not filer.remote.sync for remote.mount.buckets help string. ( #5934 )
2024-08-23 20:41:33 -07:00
chrislu
4463296811
add parallel vacuuming
2024-08-21 22:53:54 -07:00
chrislu
5535b60e8c
fix replication range check
...
fix for https://github.com/seaweedfs/seaweedfs/wiki/Replication#replication-string
2024-08-21 16:58:43 -07:00
zemul
95bae91ca7
Fix get dir metadata ( #5923 )
...
* fix:mount deadlock
* feat: query dir metadata
* feat: query dir metadata
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2024-08-21 11:01:18 -07:00
chrislu
6ce27affa4
also use /healthz for most consistent health check
2024-08-21 10:55:39 -07:00
chrislu
b3696024d1
add warning for not enough copies when skipping vacuuming volumes
...
fix https://github.com/seaweedfs/seaweedfs/issues/5906
2024-08-20 09:39:35 -07:00
Riccardo Bertossa
6fe8639504
add http endpoint to get the size of a collection ( #5910 )
2024-08-19 07:44:45 -07:00
chrislu
e50d85c0f3
3.72
2024-08-18 21:51:53 -07:00
Oleg Salionov
4cfc7d3b10
Dont try lazy decode content in proxyToFiler if no accept-encoding provided ( #5907 )
2024-08-18 21:51:17 -07:00
chrislu
e8462ba3ad
prevent compacting on the same volume
2024-08-18 12:08:43 -07:00
augustazz
db833abfa2
fix ec volume lookup data sync ( #5900 )
2024-08-16 06:08:33 -07:00
augustazz
0b00706454
EC volume supports expiration and displays expiration message when executing volume.list ( #5895 )
...
* ec volume expire
* volume.list show DestroyTime
* comments
* code optimization
---------
Co-authored-by: xuwenfeng <xuwenfeng1@zto.com>
2024-08-16 00:20:00 -07:00
chrislu
8f1f1730e9
minor
2024-08-15 19:12:11 -07:00
wusong
6f58ab7e8b
[master] fix master panic ( #5893 )
2024-08-15 06:20:51 -07:00
chrislu
fdf7193ae7
rename
2024-08-13 13:59:24 -07:00
chrislu
07f4998188
add dat file size into vif for EC
2024-08-13 13:56:00 -07:00
chrislu
3a2e21fee7
fix EC reading on nLargeBlockRows
...
fix https://github.com/seaweedfs/seaweedfs/issues/5465
2024-08-13 12:01:46 -07:00
chrislu
e14b6e02df
load ec volume id locations
...
fix https://github.com/seaweedfs/seaweedfs/issues/5713
2024-08-12 11:54:25 -07:00
chrislu
480fcb1d03
fix build
2024-08-12 09:45:51 -07:00
chrislu
ff66155302
fix tests
2024-08-12 09:30:42 -07:00
chrislu
270e91b0be
adds locking
2024-08-11 13:06:01 -07:00
chrislu
b6fd1ff4ce
refactor
2024-08-10 12:39:16 -07:00
chrislu
7438648d1c
Merge branch 'master' into mq
2024-08-10 10:01:57 -07:00
Kamran Sarwar
63c707f9c1
added access-control-expose-headers to put request ( #5876 )
2024-08-09 15:52:05 -07:00
chrislu
e825dd7496
fix opposite logic
...
fix https://github.com/seaweedfs/seaweedfs/issues/5871
2024-08-08 23:16:38 -07:00
chrislu
e069009f09
[remote.cache, remote.uncache] fix min/max age
...
fix https://github.com/seaweedfs/seaweedfs/issues/5871
2024-08-08 21:25:52 -07:00
chrislu
9ca30e52d5
fuse mount handles ttl entries
...
fix https://github.com/seaweedfs/seaweedfs/issues/5527
2024-08-08 08:18:54 -07:00
chrislu
2074e7db67
fix invalid toml
2024-08-08 00:03:42 -07:00
chrislu
e86b0bcaaa
simplify
2024-08-07 10:46:29 -07:00
chrislu
57dc39c451
randomizing next file handle id
2024-08-07 10:42:00 -07:00
Konstantin Lebedev
3e6ca6e706
[webdav] status code 500 if internal error from filer ( #5865 )
2024-08-06 07:39:52 -07:00
Konstantin Lebedev
ef4c555e90
[shell] volume.grow ( #5855 )
2024-08-05 09:22:17 -07:00
qinguoyi
bf2366ba3a
Fix KeyMaterial nil bug ( #5849 )
2024-08-02 11:06:48 -07:00
qinguoyi
e88da0be1b
Fix KeyMaterial nil bug ( #5848 )
2024-08-02 09:47:11 -07:00
qinguoyi
606d0cba64
Fix extractToFile gz file nil bug ( #5847 )
2024-08-02 09:46:37 -07:00
qinguoyi
b74b506e52
add command backup destory volume error log ( #5846 )
2024-08-02 00:04:57 -07:00
chrislu
8378a5b70b
rename
2024-08-01 23:54:42 -07:00
chrislu
d49ecde535
rename functions
2024-08-01 23:27:23 -07:00
zuzuviewer
5ee50d26d8
fix CVE-2016-2183 ( #5844 )
...
* fix CVE-2016-2183
* allow tls.min_version and tls.max_version not set
2024-08-01 20:00:22 -07:00
chrislu
4fee496b49
conditional delete
2024-08-01 13:00:54 -07:00
wyang
31b89c1062
fsck: only check the appendNs of deleted needle ( #5841 )
...
increase fsck speed
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-31 01:12:57 -07:00
Konstantin Lebedev
b2ffcdaab2
[master] do sync grow request only if absolutely necessary ( #5821 )
...
* do sync grow request only if absolutely necessary
https://github.com/seaweedfs/seaweedfs/pull/5819
* remove check VolumeGrowStrategy Threshold on PickForWrite
* fix fmt.Errorf
2024-07-30 13:21:35 -07:00
Jiffs Maverick
c8506609d8
[filer] Added a version to the title of Filer UI ( #5836 )
...
Added a version to the title of Filer UI
2024-07-29 14:07:12 -07:00
Konstantin Lebedev
e1f9336688
[filer] fix panic filer on webdav.maxMB ( #5822 )
...
fix panic filer on webdav.maxMB
2024-07-27 13:40:22 -07:00
wyang
4b1f539ab8
fix allocate reduplicated volumeId to different volume ( #5811 )
...
* fix allocate reduplicated volumeId to different volume
* only check barrier when read
---------
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-26 21:48:36 -07:00
wyang
c1bffca246
fix evacuate volume to different disk types ( #5530 )
...
Co-authored-by: wyang <wyang@wyangs-Air.lan>
2024-07-26 21:36:56 -07:00
chrislu
e2a07d11d5
Revert "Check ShouldGrowVolumes before returning error in assign. ( #5819 )"
...
This reverts commit 98d66338d0333cd955f7840c64ef95e3c4807a17.
2024-07-26 11:21:50 -07:00
wyang
0581ce6096
fix delete chunk failed if volumeSever specified grpc.port ( #5820 )
...
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-26 11:14:19 -07:00
chrislu
ec9455dab7
same change as #5819
...
follow up with https://github.com/seaweedfs/seaweedfs/pull/5819
2024-07-26 11:06:47 -07:00
Ruoxi
d15966ae8e
Check ShouldGrowVolumes before returning error in assign. ( #5819 )
2024-07-26 11:04:38 -07:00
Chris Lu
952afd810c
Fix dead lock ( #5815 )
...
* reduce locks to avoid dead lock
Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0
goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)
SaveContent -> MemChunk.RLock ->
ChunkedDirtyPages.saveChunkedFileIntervalToStorage
pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
fh.entryLock.Lock()
sync.(*RWMutex).Lock(0x0?)
/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
/github/workspace/weed/util/limited_executor.go:33 +0x97
On metadata update
fh.entryLock.Lock()
fh.dirtyPages.Destroy()
up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock
goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570
* use locked entry everywhere
* modifiable remote entry
* skip locking after getting lock from fhLockTable
2024-07-24 23:46:40 -07:00
chrislu
ed7e721efe
3.71
2024-07-21 21:03:27 -07:00
chrislu
9265be43c0
avoid nil
2024-07-21 21:01:29 -07:00
chrislu
4020149690
3.70
2024-07-21 20:27:39 -07:00
chrislu
9dd008f8f1
add version to filer
2024-07-21 20:13:51 -07:00
chrislu
9dcc576499
Revert "add collection for buckets"
...
This reverts commit 96af571219
2024-07-20 12:00:50 -07:00
chrislu
aed2a18223
get, set and add path conf
2024-07-20 11:27:11 -07:00
chrislu
96af571219
add collection for buckets
2024-07-17 23:14:36 -07:00
chrislu
a8fa78b892
refactoring
2024-07-17 23:14:36 -07:00
chrislu
7a75928e7d
Azure sink: avoid overwriting existing files
2024-07-17 23:14:36 -07:00
vadimartynov
86d92a42b4
Added tls for http clients ( #5766 )
...
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
chrislu
c6dec11ea5
[filer.sync] skip overwriting existing fresh entry
2024-07-16 09:38:10 -07:00
chrislu
19d14b4c71
minor
2024-07-16 09:21:01 -07:00
vadimartynov
b796c21fa9
Added loadSecurityConfigOnce ( #5792 )
2024-07-16 09:15:55 -07:00
Konstantin Lebedev
67edf1d014
[master] Do Automatic Volume Grow in background ( #5781 )
...
* Do Automatic Volume Grow in backgound
* pass lastGrowCount to master
* fix build
* fix count to uint64
2024-07-16 08:03:40 -07:00
wusong
ce61a66b65
Fix mmap write fail ( #5791 )
...
* [mount] fix GetAttr blocks count
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
* [mount] fix mmap read
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
---------
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2024-07-16 07:59:55 -07:00
Dan
794a5aa67d
Add warning with hints to why a EC shard cant be moved ( #5790 )
2024-07-15 11:17:22 -07:00
Konstantin Lebedev
a53e406c99
[master] refactor HasGrowRequest to atomic bool ( #5782 )
...
refactor HasGrowRequest to atomit bool
2024-07-15 10:51:21 -07:00
Konstantin Lebedev
33964fa292
metrics stats of volume layout depends on the data center ( #5775 )
...
stats volume layout depends on the data center
2024-07-12 12:32:25 -07:00
Konstantin Lebedev
04f4b10884
fix: avoid timeout if datacenter does not exist in topology ( #5772 )
...
* fix: avoid timeout if datacenter does not exist in topology
* fix: error msg
* fix: rm dublicate check
* fix: compare
* revert minor change
2024-07-12 11:19:08 -07:00
chrislu
3a82f5ffad
ensure metadata follow a specific folder
...
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12 11:17:30 -07:00
vadimartynov
ec9e7493b3
-metricsIp cmd flag ( #5773 )
...
* Added/Updated:
- Added metrics ip options for all servers;
- Fixed a bug with the selection of the binIp or ip parameter for the metrics handler;
* Fixed cmd flags
2024-07-12 10:56:26 -07:00
小羽
cb5dae0c9c
Fix "no more writable volumes" error when volume grows and master leader changed at the same time ( #5771 )
2024-07-12 10:49:15 -07:00
小羽
e8537d7172
Different disk labels should not use the same DiskUsages instance while master received volume heatbeat ( #5770 )
2024-07-12 08:09:51 -07:00
chrislu
4e7d8eb3f1
small refactoring
2024-07-11 14:50:11 -07:00
vadimartynov
de60f383de
Added tls for http handlers ( #5764 )
...
* Added https handler for filer
* Added example for security.toml
2024-07-11 07:53:18 -07:00
chrislu
2addcd1623
fix test
2024-07-11 07:53:02 -07:00
chrislu
590397229e
fix heap init
2024-07-11 00:26:18 -07:00
chrislu
06e3dd306f
clean up
2024-07-11 00:19:39 -07:00
chrislu
c59517e284
upgrade to github.com/redis/go-redis/v9
...
fix https://github.com/seaweedfs/seaweedfs/issues/4532
2024-07-11 00:15:19 -07:00
chrislu
33b11f92bc
Squashed commit of the following:
...
commit 465b47b8f72668dfc12eaa72befb232513c9e1db
Merge: f39272b54 9c440d472
2024-07-11 00:04:24 -07:00
Konstantin Lebedev
9c440d472c
[s3] revert skip deletion error, since the error file was not found is already skipped on the side of the grpc function ( #5760 )
...
* revert skip deletion error, since the error file was not found is already skipped on the side of the grpc function
* fix response error
* fix test_lifecycle_get
* Revert "fix test_lifecycle_get"
This reverts commit 8f991bdcf9
2024-07-10 08:47:43 -07:00
chrislu
182f6d00a9
always copy ecj files
2024-07-10 01:25:57 -07:00
Dan
bd54669d58
Detect underflow when calculating unused space ( #5758 )
...
* Detect underflow when calculating unused space
* Detect underflow when calculating unused space
2024-07-10 00:30:28 -07:00
chrislu
b8121e4b24
simplify
2024-07-08 11:10:41 -07:00
chrislu
f618121e63
Fix S3 deletion in deep folders, and names with empty spaces
...
fix https://github.com/seaweedfs/seaweedfs/issues/5748
2024-07-08 10:20:09 -07:00
chrislu
f3d708d3b6
add a test case
2024-07-07 23:44:36 -07:00
chrislu
d29fccc521
skip checking if it is the root directory
2024-07-07 23:44:36 -07:00
chrislu
114ac39917
add a simple test
2024-07-07 23:44:36 -07:00
Konstantin Lebedev
f77eee667d
add s3test for sql ( #5718 )
...
* add s3test for sql
* fix test test_bucket_listv2_delimiter_basic for s3
* fix action s3tests
* regen s3 api xsd
* rm minor s3 test test_bucket_listv2_fetchowner_defaultempty
* add docs
* without xmlns
2024-07-04 11:00:41 -07:00
