Commit Graph

230 Commits

Author SHA1 Message Date
Chris Lu
952afd810c
Fix dead lock (#5815)
* reduce locks to avoid dead lock

Flush->FlushData->uplloadPipeline.FluahAll
uploaderCount>0

goroutine 1 [sync.Cond.Wait, 71 minutes]:
sync.runtime_notifyListWait(0xc0007ae4d0, 0x0)
	/usr/local/go/src/runtime/sema.go:569 +0x159
sync.(*Cond).Wait(0xc001a59290?)
	/usr/local/go/src/sync/cond.go:70 +0x85
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840)
	/github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...)
	/github/workspace/weed/mount/page_writer.go:50
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0)
	/github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0)
	/github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48
github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08)

SaveContent -> MemChunk.RLock ->
	ChunkedDirtyPages.saveChunkedFileIntervalToStorage
	pages.fh.AddChunks([]*filer_pb.FileChunk{chunk})
		fh.entryLock.Lock()

sync.(*RWMutex).Lock(0x0?)
	/usr/local/go/src/sync/rwmutex.go:146 +0x31
github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1})
	/github/workspace/weed/mount/filehandle.go:93 +0x45
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?)
	/github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0)
	/github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1()
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55
github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1()
	/github/workspace/weed/util/limited_executor.go:38 +0x62
created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1
	/github/workspace/weed/util/limited_executor.go:33 +0x97

On metadata update
fh.entryLock.Lock()
	fh.dirtyPages.Destroy()
		up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock

goroutine 134 [sync.RWMutex.Lock, 71 minutes]:
sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?)
	/usr/local/go/src/runtime/sema.go:87 +0x25
sync.(*RWMutex).Lock(0xc0007c35a8?)
	/usr/local/go/src/sync/rwmutex.go:151 +0x6a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130)
	/github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20})
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7
github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0)
	/github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185
github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?)
	/github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17
github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...)
	/github/workspace/weed/mount/page_writer.go:78
github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?)
	/github/workspace/weed/mount/weedfs.go:119 +0x17a
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?)
	/github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810)
	/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570

* use locked entry everywhere

* modifiable remote entry

* skip locking after getting lock from fhLockTable
2024-07-24 23:46:40 -07:00
vadimartynov
86d92a42b4
Added tls for http clients (#5766)
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
chrislu
3a82f5ffad ensure metadata follow a specific folder
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12 11:17:30 -07:00
Gaspare Iengo
fce8fc1e16
Fix Issue #5649 (#5652) 2024-06-06 12:49:33 -07:00
chrislu
d218fe54fa go fmt 2024-05-20 11:03:56 -07:00
chrislu
2a4a9c6343 log possible racing condition
fix https://github.com/seaweedfs/seaweedfs/issues/5412
2024-04-11 23:44:18 -07:00
chrislu
6aa804b368 lock instead of RLock, to prevent racing condition
https://github.com/seaweedfs/seaweedfs/discussions/5432#discussioncomment-8933608
2024-03-27 16:05:11 -07:00
wusong
a5582cc921
[mount] fix GetAttr blocks count (#5409) 2024-03-22 06:11:16 -07:00
Viktor Kuzmin
06e723f629 Waiting for writes to complete with acquired locks may lead to deadlock, fixes #4952 2023-11-17 07:38:14 -08:00
chrislu
310eec9a9c skip if path changed
mitigate https://github.com/seaweedfs/seaweedfs/issues/4968
2023-11-01 16:27:59 -07:00
chrislu
cbc24c7b24 disconnect from old subscribers 2023-10-13 16:48:14 -07:00
chrislu
b02fdeabff RetryForever => RetryUntil 2023-10-01 11:33:56 -07:00
chrislu
186b1f029b read source uses shared lock 2023-09-23 10:33:44 -07:00
chrislu
31fc165715 mount switch to ordered lock requests 2023-09-21 11:08:26 -07:00
chrislu
27af11f1e8 Revert "Revert "Merge branch 'master' into sub""
This reverts commit 0bb97709d4.
2023-09-18 18:47:34 -07:00
chrislu
0bb97709d4 Revert "Merge branch 'master' into sub"
This reverts commit 4d414f54a2, reversing
changes made to 4827425146.
2023-09-18 16:13:20 -07:00
wang wusong
9e5b1eef9b avoid passe lock by value 2023-09-14 06:51:12 -07:00
chrislu
f5c4b642bb re-ordering 2023-08-16 23:54:23 -07:00
chrislu
3852307e94 renaming 2023-08-16 23:47:43 -07:00
chrislu
6c7fa567d4 add separate cache directory for write buffers 2023-08-16 23:39:21 -07:00
wusong
33b9192536
Fix large file chunks deleted by mistake (#4678) 2023-07-17 07:52:14 -07:00
wusong
0cb9ddd8ec
Fix data loss: add lock for metacache (#4664)
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-07-11 22:23:32 -07:00
wusong
c44a6552d2
Fix buff use (#4628) 2023-07-03 08:33:34 -07:00
zemul
68f1cc34a5
[mount]Fix copy file range (#4407)
* fix:mount deadlock

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-23 22:34:38 -07:00
Chengyu Liu
0f653d2153
update fuse client log which should output the fh of FileHandle. (#4404) 2023-04-15 22:15:16 -07:00
zemul
0122e022ea
Mount concurrent read (#4400)
* fix:mount deadlock

* feat: concurrent read

* fix

* Remove useless code

* fix

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-13 22:32:45 -07:00
chrislu
5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
Patrick Schmidt
71b33faef0
Skip parent directory creation in mount (#4310)
A POSIX filesystem does not implicitly create parent directories when
they do not exist. Directories must be explicitly created and
permissions be set.

This also fixes a bug where asynchronous operations would create a
file in the filer before the parent directory was created. If the
file was a symlink or another special type of file the directory
would inherit that type and become unusable in the mounted FS.
2023-03-15 12:37:25 -07:00
wusong
79af6b41d4
[mount] Fix read file-data in entry.content (#4207) 2023-02-15 07:18:56 -08:00
wusong
fe573f26a8
memchunk save content twice (#4174) 2023-01-31 19:36:59 -08:00
wusong
665917fdf1
[mount]: delete redundant code (#4173) 2023-01-31 18:54:46 -08:00
wusong
de081c0d64
[mount] fix metacache update (#4161) 2023-01-29 07:55:27 -08:00
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
chrislu
bfe5d910c6 use one readerCache for the whole file 2023-01-16 22:43:02 -08:00
chrislu
c24678153c use memory chunks only for sequential writes 2023-01-15 21:10:08 -08:00
chrislu
313fd17ae4 allow random writes for mem chunk 2023-01-13 20:32:18 -08:00
Chris Lu
1cd2e64aac
merge chunks during upload (#4130)
* merge chunks during upload

* fix test
2023-01-12 23:07:36 -08:00
chrislu
efcffe4f77 no need to set chunks during flush 2023-01-10 10:01:52 -08:00
chrislu
86657ed3a8 avoid repeatedly adding manifest chunks 2023-01-10 01:35:10 -08:00
chrislu
28fe578944 avoid possible nil entry 2023-01-10 01:07:34 -08:00
chrislu
5423790b2c add back previous chunk upload selection algo 2023-01-03 22:50:39 -08:00
Chris Lu
d4566d4aaa
more solid weed mount (#4089)
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
2023-01-02 23:20:45 -08:00
chrislu
77b7c9f7cc lock and unlock chunks for read 2022-12-21 14:15:35 -08:00
chrislu
d1797deccb remove duplicated logic of removing from writableChunks 2022-12-21 14:15:11 -08:00
chrislu
f7beba8515 unlock before submitting the uploading jobs 2022-12-20 12:52:30 -08:00
chrislu
28d479e5c0 mount: adjust locking for upload pipeline 2022-12-19 15:07:22 -08:00
wusong
549354e324
Fix hardlink counting (#4042)
Signed-off-by: wusong <wangwusong@virtaitech.com>

Signed-off-by: wusong <wangwusong@virtaitech.com>
Co-authored-by: wusong <wangwusong@virtaitech.com>
2022-12-08 10:50:57 -08:00
chrislu
659f28b855 simple refactoring 2022-12-05 23:19:38 -08:00
chrislu
ca49ef8ed8 duplicated logic 2022-12-05 23:19:22 -08:00
Chris Lu
dac9c28d05
Revert "refactor: moved to locked entry" (#4035)
* Revert "refactor: moved to locked entry"

This reverts commit 94bc9afd9d.

* only add LockedEntry, no changes to entryLock

* fix compilation
2022-12-05 12:32:27 -08:00