mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 21:47:50 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,065 Commits 32 Branches 296 Tags 152 MiB
970059d4ce
Commit Graph

796 Commits

Author SHA1 Message Date
Konstantin Lebedev
bae93e611d
[filer] avoid 500 if table doesn't exist (#6075) 2024-09-27 07:15:54 -07:00
Guang Jiong Lou
6c986e9d70
improve worm support (#5983) 
* improve worm support

Signed-off-by: lou <alex1988@outlook.com>

* worm mode in filer

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* move to fs configure

Signed-off-by: lou <alex1988@outlook.com>

* remove flag

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* support worm hardlink

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* typo

Signed-off-by: lou <alex1988@outlook.com>

* sync filer conf

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
 2024-09-16 21:02:21 -07:00
Guang Jiong Lou
367e7638d7
fix invalid file read (#6024) 2024-09-16 08:45:50 -07:00
Bruce
5428229347
fix file read crash (#6021) 2024-09-14 08:33:35 -07:00
Aleksey Kosov
7340c62c47
changing FindEntry error handling in cassandra store (#6015) 2024-09-13 06:53:04 -07:00
Eugeniy E. Mikhailov
dab0bb8097
Feature limit caching to prescribed number of bytes per file (#6009) 
* feature: we can check if a fileId is already in the cache

We using this to protect cache from adding the same needle to
the cache over and over.

* fuse mount: Do not start dowloader if needle is already in the cache

* added maxFilePartSizeInCache property to ChunkCache

If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).

* feature: for large files put in cache no more than prescribed number of bytes

Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.

* added dummy mock methods to satisfy interfaces of ChunkCache
 2024-09-11 21:09:20 -07:00
chrislu
f23015ac5b weed mount caching 2024-09-10 10:30:05 -07:00
chrislu
93decd1114 remove unused 2024-09-10 10:29:25 -07:00
chrislu
69fcdd0840 adjust logging 2024-09-10 10:28:49 -07:00
zouyixiong
d8e18a8eb7
bug fixed to filer store elastic deleteEntry (#5988) 2024-09-09 02:15:25 -07:00
chrislu
18afdb15b6 Revert "weed mount, weed dav add option to force cache" 
This reverts commit 7367b976b0.
 2024-09-04 01:38:29 -07:00
chrislu
7367b976b0 weed mount, weed dav add option to force cache 2024-09-04 01:19:14 -07:00
chrislu
d49ecde535 rename functions 2024-08-01 23:27:23 -07:00
chrislu
4fee496b49 conditional delete 2024-08-01 13:00:54 -07:00
wyang
0581ce6096
fix delete chunk failed if volumeSever specified grpc.port (#5820) 
Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-07-26 11:14:19 -07:00
chrislu
aed2a18223 get, set and add path conf 2024-07-20 11:27:11 -07:00
chrislu
a8fa78b892 refactoring 2024-07-17 23:14:36 -07:00
vadimartynov
86d92a42b4
Added tls for http clients (#5766) 
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
 2024-07-16 23:14:09 -07:00
wusong
ce61a66b65
Fix mmap write fail (#5791) 
* [mount] fix GetAttr blocks count

Signed-off-by: wang wusong <wangwusong@virtaitech.com>

* [mount] fix mmap read

Signed-off-by: wang wusong <wangwusong@virtaitech.com>

---------

Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2024-07-16 07:59:55 -07:00
chrislu
4e7d8eb3f1 small refactoring 2024-07-11 14:50:11 -07:00
chrislu
2addcd1623 fix test 2024-07-11 07:53:02 -07:00
chrislu
590397229e fix heap init 2024-07-11 00:26:18 -07:00
chrislu
06e3dd306f clean up 2024-07-11 00:19:39 -07:00
chrislu
c59517e284 upgrade to github.com/redis/go-redis/v9 
fix https://github.com/seaweedfs/seaweedfs/issues/4532
 2024-07-11 00:15:19 -07:00
chrislu
33b11f92bc Squashed commit of the following: 
commit 465b47b8f72668dfc12eaa72befb232513c9e1db
Merge: f39272b54 9c440d472
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:56:51 2024 -0700

    Merge branch 'master' into merged-persisted-logs

commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:49:26 2024 -0700

    merge meta logs from multiple files
 2024-07-11 00:04:24 -07:00
chrislu
d29fccc521 skip checking if it is the root directory 2024-07-07 23:44:36 -07:00
chrislu
0fdf7eca48 avoid possible nil case during other error cases 2024-07-01 00:54:37 -07:00
chrislu
d8dfadb617 ParallelProcessDirectoryStructure 
need to use this for simpler parallel processing
 2024-06-28 15:02:46 -07:00
chrislu
c030cb3ce9 bootstrap filer from one peer 2024-06-28 14:57:20 -07:00
chrislu
cb67137a03 optionally open the leveldb in readonly mode 2024-06-25 21:07:04 -07:00
chrislu
9ac1023362 check bucket name in sql backend 
fix https://github.com/seaweedfs/seaweedfs/issues/5710
 2024-06-25 08:54:19 -07:00
chrislu
464611f614 optionally skip deleting file chunks 2024-06-15 11:39:48 -07:00
chrislu
843e778875 refactor 2024-06-15 09:41:08 -07:00
chrislu
677cfb8ad1 refactor 2024-06-15 09:39:49 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops (#5628) 
* Added context for the MasterClient's methods to avoid endless loops

* Returned WithClient function. Added WithClientCustomGetMaster function

* Hid unused ctx arguments

* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions

* Changed the context termination check in the tryConnectToMaster function

* Added a child context to the tryConnectToMaster function

* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
 2024-06-14 11:40:34 -07:00
chrislu
dbfbabac55 simplify 2024-06-09 12:14:10 -07:00
dependabot[bot]
722a14ef57
chore(deps): bump github.com/ydb-platform/ydb-go-sdk-auth-environ from 0.2.0 to 0.4.2 (#5621) 
* chore(deps): bump github.com/ydb-platform/ydb-go-sdk-auth-environ

Bumps [github.com/ydb-platform/ydb-go-sdk-auth-environ](https://github.com/ydb-platform/ydb-go-sdk-auth-environ) from 0.2.0 to 0.4.2.
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk-auth-environ/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk-auth-environ/compare/v0.2.0...v0.4.2)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk-auth-environ
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix compilation

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2024-05-27 12:36:00 -07:00
sb
a3a00d9499
Feature/mongodb security (#5602) 2024-05-18 12:19:10 -07:00
sb
3fae87632f
add filer configuration options for username/password and tls (#5601) 2024-05-18 09:02:03 -07:00
chrislu
f80fb1c02e Merge branch 'master' into mq-subscribe 2024-04-29 21:57:31 -07:00
dependabot[bot]
36a1cf0361
chore(deps): bump github.com/viant/ptrie from 0.3.1 to 1.0.1 (#5552) 
* chore(deps): bump github.com/viant/ptrie from 0.3.1 to 1.0.1

Bumps [github.com/viant/ptrie](https://github.com/viant/ptrie) from 0.3.1 to 1.0.1.
- [Release notes](https://github.com/viant/ptrie/releases)
- [Changelog](https://github.com/viant/ptrie/blob/master/CHANGELOG.md)
- [Commits](https://github.com/viant/ptrie/compare/v0.3.1...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/viant/ptrie
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix compilation

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2024-04-29 18:42:31 -07:00
chrislu
5e8d788311 Merge branch 'master' into mq-subscribe 2024-04-29 06:44:03 -07:00
Nikita Borzykh
0d04264494
Move file deletion logging to v2 verbisity (#5538) 2024-04-26 06:41:06 -07:00
chrislu
cc1c69f312 Merge branch 'master' into mq-subscribe 2024-04-08 11:03:14 -07:00
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server (#5480) 2024-04-08 07:27:00 -07:00
chrislu
ccdd9cd8de refactor 2024-04-02 16:25:43 -07:00
Nikita Borzykh
829c0da381
fix: adjust condition in prefixFilterEntries prevent infinite loop (#5440) 2024-03-30 14:47:00 -07:00
chrislu
36fec34c47 print only adapted url 
fix https://github.com/seaweedfs/seaweedfs/issues/5424
 2024-03-25 12:50:43 -07:00
Nikita Borzykh
9ca871af53
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries in etcd meta storage backend (#5416) 
Move ListDirectoryEntries logic to ListDirectoryPrefixedEntries
 2024-03-24 14:08:54 -07:00
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage (#5403) 2024-03-24 08:36:21 -07:00