chrislu
eb02946c97
support write once read many
...
fix https://github.com/seaweedfs/seaweedfs/issues/5954
2024-09-04 02:25:07 -07:00
chrislu
18afdb15b6
Revert "weed mount, weed dav add option to force cache"
...
This reverts commit 7367b976b0
2024-09-04 01:38:29 -07:00
chrislu
7367b976b0
weed mount, weed dav add option to force cache
2024-09-04 01:19:14 -07:00
Chris Lu
952afd810c
Fix dead lock ( #5815 )
...
* reduce locks to avoid dead lock
Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0
goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)
SaveContent -> MemChunk.RLock ->
ChunkedDirtyPages.saveChunkedFileIntervalToStorage
pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
fh.entryLock.Lock()
sync.(*RWMutex).Lock(0x0?)
/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
/github/workspace/weed/util/limited_executor.go:33 +0x97
On metadata update
fh.entryLock.Lock()
fh.dirtyPages.Destroy()
up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock
goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570
* use locked entry everywhere
* modifiable remote entry
* skip locking after getting lock from fhLockTable
2024-07-24 23:46:40 -07:00
Gaspare Iengo
fce8fc1e16
Fix Issue #5649 ( #5652 )
2024-06-06 12:49:33 -07:00
chrislu
31fc165715
mount switch to ordered lock requests
2023-09-21 11:08:26 -07:00
chrislu
f5c4b642bb
re-ordering
2023-08-16 23:54:23 -07:00
chrislu
3852307e94
renaming
2023-08-16 23:47:43 -07:00
chrislu
6c7fa567d4
add separate cache directory for write buffers
2023-08-16 23:39:21 -07:00
chrislu
5c2eb602c9
Revert "mount: remove mount directory from generating cache directory id"
...
This reverts commit 109dc7fdf6
2022-10-14 22:24:14 -07:00
chrislu
109dc7fdf6
mount: remove mount directory from generating cache directory id
...
fix https://github.com/seaweedfs/seaweedfs/discussions/3811
2022-10-10 00:12:10 -07:00
Patrick Schmidt
5df105b1f9
Fix a few data races when reading files in mount ( #3527 )
2022-08-26 16:41:37 -07:00
chrislu
8b3429858d
align memory for atomic read/write
2022-08-21 12:20:27 -07:00
Patrick Schmidt
2ef6ab998c
Avoid race conditions with current filer address ( #3474 )
...
When multiple filer requests are in-flight and the current filer
disappears and a new one is selected by the first goroutine, then
there can be a lot of race conditions while retrieving the current
filer.
Therefore, load/save the current filer index atomically.
2022-08-21 12:18:13 -07:00
chrislu
bd13a7968f
remove unused symlink resolving
2022-08-04 01:35:18 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
chrislu
e0eda52c54
mount: ensure symlink parent directory is tracked
...
fix https://github.com/chrislusf/seaweedfs/issues/3373
2022-07-28 16:32:00 -07:00
chrislu
2dc5d4adac
disable symlink translation for now
2022-07-24 18:24:34 -07:00
chrislu
2b64916d3e
set status to fuse.OK
2022-07-24 15:30:55 -07:00
chrislu
7386cf6a06
mount: resolve symlink
2022-07-24 15:11:24 -07:00
ningfd
f32142f6f5
add disableXAttr in mount option
2022-06-06 14:09:01 +08:00
chrislu
d65bb2c6df
mount: file handle locks entry better
...
related to https://github.com/chrislusf/seaweedfs/issues/2952
2022-06-05 18:15:06 -07:00
chrislu
5b8b022985
remove unused parameter
2022-05-24 18:52:04 -07:00
chrislu
3ab2c0e5c0
avoid possible nil attributes
2022-04-06 10:11:11 -07:00
chrislu
958f880b70
mount: add grpc method to adjust quota
2022-04-02 15:14:37 -07:00
chrislu
f70c1e449b
add useful doc link
2022-03-13 00:14:50 -08:00
chrislu
011a41b561
add back writes to swap file when too many in memory chunks are used.
2022-03-09 22:26:51 -08:00
chrislu
c7e8ac18f0
mount: quota for one mounted collection
...
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
2022-03-06 02:44:40 -08:00
chrislu
b7c992f410
add flag to enforce quota
2022-03-05 22:10:43 -08:00
chrislu
fcf3714443
mount: add back support for filer.path
2022-02-28 12:16:53 -08:00
chrislu
9ef5bb20f6
mount2: invalidate fuse cache for replaced inode
2022-02-27 00:00:23 -08:00
chrislu
93e12d5f30
fix build
2022-02-18 00:48:00 -08:00
chrislu
f9d33f70b0
return fuse.Status when looking up by inode
2022-02-18 00:45:43 -08:00
chrislu
49b84b6e2a
list entries while reading from remote
2022-02-16 21:32:15 -08:00
chrislu
22739c653a
clean up
2022-02-16 07:57:08 -08:00
chrislu
65bfeafb5a
add back mkdir
2022-02-16 01:09:21 -08:00
chrislu
37e8fce841
clean up cache on exit
2022-02-16 00:39:21 -08:00
chrislu
1560ec7e26
remove unused code
2022-02-16 00:37:24 -08:00
chrislu
17ac5244c3
mount2: avoid double listing directories
2022-02-15 01:44:17 -08:00
chrislu
fe57a2e770
file set attribute
2022-02-14 01:36:10 -08:00
chrislu
dbeeda8123
listen for metadata updates
2022-02-14 01:09:31 -08:00
chrislu
2b955c1713
support read
2022-02-13 22:50:44 -08:00
chrislu
bb9919b07a
add open release, refactor
2022-02-13 19:14:34 -08:00
chrislu
3d93570979
supports forget
2022-02-13 03:31:47 -08:00
chrislu
7cfbf1e85f
fix ok status
2022-02-12 22:41:45 -08:00
chrislu
72faae91e1
implement read directory and read directory plus
2022-02-12 02:48:44 -08:00
chrislu
f4d88862c4
can attr root directory
2022-02-12 01:54:16 -08:00
chrislu
180445f5a8
change to use fuse file system
2022-02-11 21:35:09 -08:00
chrislu
45a0fda9bd
need to follow https://github.com/hanwen/go-fuse/blob/master/fuse/api.go
2022-02-11 03:12:52 -08:00
chrislu
f87da798a4
to be re-written following fuse virtual file system
2022-02-11 03:09:30 -08:00
