Commit Graph

18 Commits

Author SHA1 Message Date
chrislu
efcffe4f77 no need to set chunks during flush 2023-01-10 10:01:52 -08:00
chrislu
86657ed3a8 avoid repeatedly adding manifest chunks 2023-01-10 01:35:10 -08:00
Chris Lu
d4566d4aaa
more solid weed mount (#4089)
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
2023-01-02 23:20:45 -08:00
chrislu
659f28b855 simple refactoring 2022-12-05 23:19:38 -08:00
Chris Lu
dac9c28d05
Revert "refactor: moved to locked entry" (#4035)
* Revert "refactor: moved to locked entry"

This reverts commit 94bc9afd9d.

* only add LockedEntry, no changes to entryLock

* fix compilation
2022-12-05 12:32:27 -08:00
chrislu
94bc9afd9d refactor: moved to locked entry 2022-12-04 23:33:05 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
22064c3425 mount: ensure ordered file handle lock and unlock 2022-09-11 19:44:34 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
2022-06-06 00:39:35 -07:00
chrislu
d65bb2c6df mount: file handle locks entry better
related to https://github.com/chrislusf/seaweedfs/issues/2952
2022-06-05 18:15:06 -07:00
chrislu
7542fd7f4d mount: optimize a bit when writing files 2022-03-17 00:02:38 -07:00
chrislu
f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
chrislu
b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu
3639fedd01 mount: fix fsync opened and renamed files 2022-02-28 00:34:17 -08:00
chrislu
e8ce30fdc5 mount2: adjust file mode 2022-02-18 00:47:02 -08:00
chrislu
7286e525ad support write 2022-02-13 23:27:11 -08:00
chrislu
bb9919b07a add open release, refactor 2022-02-13 19:14:34 -08:00