mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 13:37:49 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,028 Commits 32 Branches 296 Tags 152 MiB
c0e36231ad
Commit Graph

56 Commits

Author SHA1 Message Date
chrislu
c0e36231ad use only one metadata follow process 2024-10-06 12:55:19 -07:00
Guang Jiong Lou
6c986e9d70
improve worm support (#5983) 
* improve worm support

Signed-off-by: lou <alex1988@outlook.com>

* worm mode in filer

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* move to fs configure

Signed-off-by: lou <alex1988@outlook.com>

* remove flag

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* support worm hardlink

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* typo

Signed-off-by: lou <alex1988@outlook.com>

* sync filer conf

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
 2024-09-16 21:02:21 -07:00
chrislu
a9c9e1bcb3 refactor 2024-09-12 22:45:30 -07:00
chrislu
eb02946c97 support write once read many 
fix https://github.com/seaweedfs/seaweedfs/issues/5954
 2024-09-04 02:25:07 -07:00
chrislu
18afdb15b6 Revert "weed mount, weed dav add option to force cache" 
This reverts commit 7367b976b0.
 2024-09-04 01:38:29 -07:00
chrislu
7367b976b0 weed mount, weed dav add option to force cache 2024-09-04 01:19:14 -07:00
Chris Lu
952afd810c
Fix dead lock (#5815) 
* reduce locks to avoid dead lock

Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0

goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
	/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
	/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
	/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
	/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
	/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
	/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)

SaveContent -> MemChunk.RLock ->
	ChunkedDirtyPages.saveChunkedFileIntervalToStorage
	pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
		fh.entryLock.Lock()

sync.(*RWMutex).Lock(0x0?)
	/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
	/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
	/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
	/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
	/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
	/github/workspace/weed/util/limited_executor.go:33 +0x97

On metadata update
fh.entryLock.Lock()
	fh.dirtyPages.Destroy()
		up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock

goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
	/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
	/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
	/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
	/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
	/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
	/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
	/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
	/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570

* use locked entry everywhere

* modifiable remote entry

* skip locking after getting lock from fhLockTable
 2024-07-24 23:46:40 -07:00
Gaspare Iengo
fce8fc1e16
Fix Issue #5649 (#5652) 2024-06-06 12:49:33 -07:00
chrislu
31fc165715 mount switch to ordered lock requests 2023-09-21 11:08:26 -07:00
chrislu
f5c4b642bb re-ordering 2023-08-16 23:54:23 -07:00
chrislu
3852307e94 renaming 2023-08-16 23:47:43 -07:00
chrislu
6c7fa567d4 add separate cache directory for write buffers 2023-08-16 23:39:21 -07:00
chrislu
5c2eb602c9 Revert "mount: remove mount directory from generating cache directory id" 
This reverts commit 109dc7fdf6.
 2022-10-14 22:24:14 -07:00
chrislu
109dc7fdf6 mount: remove mount directory from generating cache directory id 
fix https://github.com/seaweedfs/seaweedfs/discussions/3811
 2022-10-10 00:12:10 -07:00
Patrick Schmidt
5df105b1f9
Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
chrislu
8b3429858d align memory for atomic read/write 2022-08-21 12:20:27 -07:00
Patrick Schmidt
2ef6ab998c
Avoid race conditions with current filer address (#3474) 
When multiple filer requests are in-flight and the current filer
disappears and a new one is selected by the first goroutine, then
there can be a lot of race conditions while retrieving the current
filer.
Therefore, load/save the current filer index atomically.
 2022-08-21 12:18:13 -07:00
chrislu
bd13a7968f remove unused symlink resolving 2022-08-04 01:35:18 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
e0eda52c54 mount: ensure symlink parent directory is tracked 
fix https://github.com/chrislusf/seaweedfs/issues/3373
 2022-07-28 16:32:00 -07:00
chrislu
2dc5d4adac disable symlink translation for now 2022-07-24 18:24:34 -07:00
chrislu
2b64916d3e set status to fuse.OK 2022-07-24 15:30:55 -07:00
chrislu
7386cf6a06 mount: resolve symlink 2022-07-24 15:11:24 -07:00
ningfd
f32142f6f5 add disableXAttr in mount option 2022-06-06 14:09:01 +08:00
chrislu
d65bb2c6df mount: file handle locks entry better 
related to https://github.com/chrislusf/seaweedfs/issues/2952
 2022-06-05 18:15:06 -07:00
chrislu
5b8b022985 remove unused parameter 2022-05-24 18:52:04 -07:00
chrislu
3ab2c0e5c0 avoid possible nil attributes 2022-04-06 10:11:11 -07:00
chrislu
958f880b70 mount: add grpc method to adjust quota 2022-04-02 15:14:37 -07:00
chrislu
f70c1e449b add useful doc link 2022-03-13 00:14:50 -08:00
chrislu
011a41b561 add back writes to swap file when too many in memory chunks are used. 2022-03-09 22:26:51 -08:00
chrislu
c7e8ac18f0 mount: quota for one mounted collection 
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
 2022-03-06 02:44:40 -08:00
chrislu
b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu
fcf3714443 mount: add back support for filer.path 2022-02-28 12:16:53 -08:00
chrislu
9ef5bb20f6 mount2: invalidate fuse cache for replaced inode 2022-02-27 00:00:23 -08:00
chrislu
93e12d5f30 fix build 2022-02-18 00:48:00 -08:00
chrislu
f9d33f70b0 return fuse.Status when looking up by inode 2022-02-18 00:45:43 -08:00
chrislu
49b84b6e2a list entries while reading from remote 2022-02-16 21:32:15 -08:00
chrislu
22739c653a clean up 2022-02-16 07:57:08 -08:00
chrislu
65bfeafb5a add back mkdir 2022-02-16 01:09:21 -08:00
chrislu
37e8fce841 clean up cache on exit 2022-02-16 00:39:21 -08:00
chrislu
1560ec7e26 remove unused code 2022-02-16 00:37:24 -08:00
chrislu
17ac5244c3 mount2: avoid double listing directories 2022-02-15 01:44:17 -08:00
chrislu
fe57a2e770 file set attribute 2022-02-14 01:36:10 -08:00
chrislu
dbeeda8123 listen for metadata updates 2022-02-14 01:09:31 -08:00
chrislu
2b955c1713 support read 2022-02-13 22:50:44 -08:00
chrislu
bb9919b07a add open release, refactor 2022-02-13 19:14:34 -08:00
chrislu
3d93570979 supports forget 2022-02-13 03:31:47 -08:00
chrislu
7cfbf1e85f fix ok status 2022-02-12 22:41:45 -08:00
chrislu
72faae91e1 implement read directory and read directory plus 2022-02-12 02:48:44 -08:00
chrislu
f4d88862c4 can attr root directory 2022-02-12 01:54:16 -08:00