seaweedfs/weed/util
Eugeniy E. Mikhailov dab0bb8097
Feature limit caching to prescribed number of bytes per file (#6009)
* feature: we can check if a fileId is already in the cache

We using this to protect cache from adding the same needle to
the cache over and over.

* fuse mount: Do not start dowloader if needle is already in the cache

* added maxFilePartSizeInCache property to ChunkCache

If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).

* feature: for large files put in cache no more than prescribed number of bytes

Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.

* added dummy mock methods to satisfy interfaces of ChunkCache
2024-09-11 21:09:20 -07:00
..
buffer_pool [volume] Reduce the number of buffers for uploading one chunk (#5458) 2024-04-11 04:47:21 -07:00
buffered_queue ensure head index is within range 2024-05-05 12:20:08 -07:00
buffered_writer webdav: improve webdav upload speed 2020-12-01 15:32:27 -08:00
chunk_cache Feature limit caching to prescribed number of bytes per file (#6009) 2024-09-11 21:09:20 -07:00
fla9 go fmt 2023-11-26 11:47:20 -08:00
grace master fix interruption through ctrl+c (#3834) 2022-10-12 07:18:40 -07:00
http Added tls for http clients (#5766) 2024-07-16 23:14:09 -07:00
httpdown test passed 2019-11-17 11:40:36 +08:00
log_buffer go fmt 2024-05-20 11:03:56 -07:00
mem remove logs 2022-08-15 01:05:35 -07:00
skiplist chore: fix function names in comment (#5478) 2024-04-08 07:19:02 -07:00
bytes_test.go minFreeSpace refactored 2021-04-27 10:37:24 +08:00
bytes.go randomizing next file handle id 2024-08-07 10:42:00 -07:00
cipher_test.go refactor(cipher_test): plantext -> plaintext (#3669) 2022-09-14 11:24:09 -07:00
cipher.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
compression_stream.go reduce gzip allocation 2021-08-20 18:38:18 +08:00
compression.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
concurrent_read_map.go directory structure change to work with glide 2016-06-02 18:09:14 -07:00
config.go Added loadSecurityConfigOnce (#5792) 2024-07-16 09:15:55 -07:00
constants_4bytes.go go fmt 2021-09-01 02:45:42 -07:00
constants_5bytes.go go fmt 2021-09-01 02:45:42 -07:00
constants.go 3.72 2024-08-18 21:51:53 -07:00
file_util_non_posix.go go fmt 2021-09-01 02:45:42 -07:00
file_util_posix.go go fmt 2021-09-01 02:45:42 -07:00
file_util_test.go fix: avoid error file name too long when writing a file (#4876) 2023-09-27 05:40:51 -07:00
file_util.go fix: avoid error file name too long when writing a file (#4876) 2023-09-27 05:40:51 -07:00
fullpath.go fix: disallow file name too long when writing a file (#4881) 2023-10-12 14:29:55 -07:00
inits_test.go try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
inits.go try showing the first 100 volume ids and an extra ... 2020-05-29 16:15:33 +08:00
limited_async_pool_test.go add future list 2022-09-25 17:43:42 -07:00
limited_async_pool.go add future list 2022-09-25 17:43:42 -07:00
limited_executor.go rename 2022-09-25 13:34:55 -07:00
lock_table_test.go adjust logs 2023-09-21 23:51:32 -07:00
lock_table.go Fix deadlock in lock table locks (#5566) 2024-05-07 05:44:53 -07:00
minfreespace_test.go minFreeSpace refactored 2021-04-27 10:37:24 +08:00
minfreespace.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
net_timeout.go refactor(various): Listner -> Listener readability improvements (#3672) 2022-09-14 11:59:55 -07:00
network.go Revert "Update network.go since To16 converts the IP address ip up to a 16-byte representation. If ip is not an IP address (it is the wrong length), To16 returns nil. (#5134)" 2023-12-24 12:31:08 -08:00
parse.go fix preconditions according to https://tools.ietf.org/id/draft-ietf-httpbis-p4-conditional-26.html#preconditions 2022-02-08 10:13:19 +08:00
queue_test.go add a simple test 2024-07-07 23:44:36 -07:00
queue_unbounded_test.go filer: avoid possible timeouts for updates and deletions 2020-02-20 15:44:17 -08:00
queue_unbounded.go refactor(queue_unbounded): inbountLen -> inboundLen (#3666) 2022-09-14 10:12:45 -07:00
queue.go Squashed commit of the following: 2024-07-11 00:04:24 -07:00
reflect.go show RemoteVolumes/EcVolumes only if it is not empty 2021-07-06 15:20:18 +08:00
retry.go comment 2023-10-01 11:36:27 -07:00
throttler.go refactor: extract out the write throttler 2019-05-06 13:56:08 -07:00