mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-21 15:28:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,151 Commits 32 Branches 296 Tags 152 MiB
fd6c787c02
Commit Graph

1650 Commits

Author SHA1 Message Date
chrislu
e2a07d11d5 Revert "Check ShouldGrowVolumes before returning error in assign. (#5819)" 
This reverts commit 98d66338d0333cd955f7840c64ef95e3c4807a17.
 2024-07-26 11:21:50 -07:00
wyang
0581ce6096
fix delete chunk failed if volumeSever specified grpc.port (#5820) 
Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-07-26 11:14:19 -07:00
chrislu
ec9455dab7 same change as #5819 
follow up with https://github.com/seaweedfs/seaweedfs/pull/5819
 2024-07-26 11:06:47 -07:00
Ruoxi
d15966ae8e
Check ShouldGrowVolumes before returning error in assign. (#5819) 2024-07-26 11:04:38 -07:00
chrislu
9dd008f8f1 add version to filer 2024-07-21 20:13:51 -07:00
vadimartynov
86d92a42b4
Added tls for http clients (#5766) 
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
 2024-07-16 23:14:09 -07:00
Konstantin Lebedev
67edf1d014
[master] Do Automatic Volume Grow in background (#5781) 
* Do Automatic Volume Grow in backgound

* pass lastGrowCount to master

* fix build

* fix count to uint64
 2024-07-16 08:03:40 -07:00
Konstantin Lebedev
04f4b10884
fix: avoid timeout if datacenter does not exist in topology (#5772) 
* fix: avoid timeout if datacenter does not exist in topology

* fix: error msg

* fix: rm dublicate check

* fix: compare

* revert minor change
 2024-07-12 11:19:08 -07:00
小羽
cb5dae0c9c
Fix "no more writable volumes" error when volume grows and master leader changed at the same time (#5771) 2024-07-12 10:49:15 -07:00
chrislu
4e7d8eb3f1 small refactoring 2024-07-11 14:50:11 -07:00
chrislu
f3d708d3b6 add a test case 2024-07-07 23:44:36 -07:00
Konstantin Lebedev
f77eee667d
add s3test for sql (#5718) 
* add s3test for sql

* fix test test_bucket_listv2_delimiter_basic for s3

* fix action s3tests

* regen s3 api xsd

* rm minor s3 test test_bucket_listv2_fetchowner_defaultempty

* add docs

* without xmlns
 2024-07-04 11:00:41 -07:00
chrislu
7c06306857 fix breadcrumb 2024-07-03 17:40:55 -07:00
chrislu
62c29b2398 Revert "directory can paginate" 
This reverts commit 69003d0e90.
 2024-07-02 23:55:48 -07:00
chrislu
761a1555de fix ordering 2024-07-02 23:55:40 -07:00
chrislu
69003d0e90 directory can paginate 2024-07-02 21:36:03 -07:00
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const (#5726) 2024-07-01 01:00:39 -07:00
chrislu
ac6fd36c06 using fixed bootstrap from peer filer 2024-06-30 23:16:54 -07:00
chrislu
c030cb3ce9 bootstrap filer from one peer 2024-06-28 14:57:20 -07:00
chrislu
ded761ffc5 adjust visibility 2024-06-24 17:14:52 -07:00
chrislu
07f47776c3 minor 2024-06-23 11:47:26 -07:00
chrislu
464611f614 optionally skip deleting file chunks 2024-06-15 11:39:48 -07:00
chrislu
64fd233a59 DeleteUncommittedChunks 2024-06-15 09:55:22 -07:00
chrislu
677cfb8ad1 refactor 2024-06-15 09:39:49 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops (#5628) 
* Added context for the MasterClient's methods to avoid endless loops

* Returned WithClient function. Added WithClientCustomGetMaster function

* Hid unused ctx arguments

* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions

* Changed the context termination check in the tryConnectToMaster function

* Added a child context to the tryConnectToMaster function

* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
 2024-06-14 11:40:34 -07:00
chrislu
2a169dde9a minor 2024-06-14 09:17:46 -07:00
chrislu
b03c831ad2 more logs 2024-06-13 21:48:44 -07:00
MeenakshiSachdeva
1caa012a76
Fix Filer Sync Issue: 5455 (#5663) 2024-06-10 09:19:38 -07:00
steve.wei
2150289442
fix: Ensure that the clientAddress is unique (#5655) 2024-06-07 09:13:03 -07:00
Gaspare Iengo
dc6b750424
Fix panic (#5654) 2024-06-06 18:59:50 -07:00
steve.wei
d8da4bbaa7
Set the capacity of clientChan to 10000 (#5647) 2024-06-05 05:41:46 -07:00
LHHDZ
36b5b713ba
fix deadlock caused by message chan blocked (#5639) 2024-06-03 07:42:40 -07:00
chrislu
3e7a92061b pass along volume server grpc port 
fix https://github.com/seaweedfs/seaweedfs/issues/5617
 2024-05-29 10:41:33 -07:00
chrislu
364bb6c7b4 avoid ticker leak 2024-05-24 17:15:12 -07:00
shenxingwuying
ee25ada732
reduce ambiguity about use memory_sequencer (#5555) 2024-04-29 21:51:00 -07:00
chrislu
55976ae04a avoid repeated calls to heavy-weighted viper 2024-04-18 09:09:45 -07:00
chrislu
d9490c5e1f rename 2024-04-18 08:47:45 -07:00
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part (#5476) 
* try fix s3  test_multipart_get_part

* add passed s3 tests

* fix SeaweedFSUploadId

* rm spaces

* convert part request to range

* add passed s3 tests of multipart
 2024-04-14 10:41:32 -07:00
yudrywet
8833745fbc
chore: fix some typos in comments (#5497) 2024-04-14 07:42:40 -07:00
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk (#5458) 2024-04-11 04:47:21 -07:00
skycope
6dae685f9c
fix append disorder (#5492) 2024-04-10 19:21:44 -07:00
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server (#5480) 2024-04-08 07:27:00 -07:00
Konstantin Lebedev
df40908e57
fix panic 5435 (#5436) 2024-03-28 16:17:59 -07:00
chrislu
aed6fdffd2 fix nil volume data backend 
fix https://github.com/seaweedfs/seaweedfs/issues/5417
 2024-03-25 08:19:31 -07:00
jerebear12
7091269e65
fix get file metadata bug (#5394) 
* Removed problematic if statement

This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.

* Update filer_server_handers_read.go

* Updated filer responses to invalid requests
 2024-03-18 11:08:16 -07:00
chrislu
6a61b54f29 Merge branch 'mq-subscribe' 2024-03-16 11:42:23 -07:00
chrislu
27bb38228b only broad cast when there are waiting threads 2024-03-16 11:38:27 -07:00
Konstantin Lebedev
54ee73287d
[filer] replace an invalid methods in a metric with an invalid one (#5378) 
fix: replace an invalid methods in a metric with an invalid one for filer
 2024-03-14 08:19:09 -07:00
chrislu
6f615c62ae Merge branch 'master' into mq-subscribe 2024-03-12 09:19:06 -07:00
rustrover
ab70aa92da
remove repetitive words (#5364) 2024-03-09 07:35:47 -08:00
chrislu
34f2b600ac each log function adds a "done" return parameter 2024-03-07 10:50:09 -08:00
chrislu
832b60c13f pass errors 2024-03-07 10:47:06 -08:00
chrislu
8f79bb3987 Merge branch 'master' into mq-subscribe 2024-03-07 10:42:29 -08:00
chrislu
5c0f01c0eb purge cluster id 2024-03-07 10:42:12 -08:00
chrislu
2a7028373d Merge branch 'master' into mq-subscribe 2024-02-29 06:22:39 -08:00
jerebear12
85d6d5371b
Disable filer UI in configuration (#5297) 
* Add filer.ui.enabled configuration property

* Add filer.expose_directory_metadata to config

* Ammend commit

* Remove ShowUI reference

* Update all routes that allow directory metadata

* Add cmd flag to server.go
 2024-02-27 08:38:55 -08:00
chrislu
a52ce076fc Merge branch 'master' into mq-subscribe 2024-02-25 08:00:37 -08:00
Sébastien
c694cb4e7d
filer: reduce delete entry ErrNotFound logs (#5301) 2024-02-14 14:22:08 -08:00
Konstantin Lebedev
8fb978d6f6
[volume] requests that were canceled by the client are not an internal server problem(500 => 499) (#5304) 2024-02-14 14:18:56 -08:00
Chris Lu
fa72761bb8 fix on finding lock owner 2024-02-05 17:30:14 -08:00
Chris Lu
53612b770c Merge branch 'master' into mq-subscribe 2024-02-04 10:44:01 -08:00
chrislu
d41792461c lock returns host and owner 2024-02-02 15:54:57 -08:00
chrislu
d30150dde1 use a short-lived lock 2024-02-01 23:01:44 -08:00
chrislu
c1bb76d709 fix nil error 2024-01-30 08:21:42 -08:00
chrislu
ac50d8a822 re-lock if the lock owner is not found 2024-01-29 22:46:23 -08:00
Sébastien
0775d05a23
fix: http range request return status 500 (#5251) 
When volume server unavailable for at least one chunk; was returning status 206.

Split `StreamContent` in two parts,
- first prepare, to get chunk info and return stream function
- then write chunk, with that stream function

That allow to catch error in first step before setting response status code in `processRangeRequest`
 2024-01-29 10:35:52 -08:00
Konstantin Lebedev
0b71fae2e3
fix: s3tests test_get_object_ifnonematch_good (#5237) 2024-01-24 11:05:55 -08:00
chrislu
a8ddbf9ed2 Merge branch 'master' into mq-subscribe 2024-01-23 19:41:16 -08:00
chrislu
49fcb48e04 same logic as grpc assign 
fix https://github.com/seaweedfs/seaweedfs/issues/5213
 2024-01-23 19:39:59 -08:00
chrislu
263f1f3d04 Merge branch 'master' into mq-subscribe 2024-01-18 09:16:46 -08:00
chrislu
3002087541 http assign logic should be the same as grpc assign 
similar logic as bebbc9fe44
 2024-01-18 08:58:53 -08:00
Konstantin Lebedev
4e9ea1e628
chore: filer healthz handler check filer store (#5208) 2024-01-17 07:46:37 -08:00
Konstantin Lebedev
f9cf13fada
[filer] avoid return http ok for not allowed methods (#5209) 2024-01-17 07:17:07 -08:00
chrislu
fa59a5d67e read from disk if not in memory 2024-01-15 00:20:12 -08:00
chrislu
70a011a224 Merge branch 'master' into mq-subscribe 2024-01-11 20:58:57 -08:00
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics (#5188) 2024-01-10 10:05:27 -08:00
jerebear12
8c966ac23b
Removed problematic if statement (#5180) 
This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.
 2024-01-08 11:35:20 -08:00
chrislu
49428a303b add batch index for each memory buffer 2024-01-08 00:03:08 -08:00
chrislu
d9bd343055 Merge branch 'master' into mq-subscribe 2024-01-05 15:36:00 -08:00
Konstantin Lebedev
3b62b338a5
chore: add maxMB option for webdav (#5165) 2024-01-05 11:10:20 -08:00
Konstantin Lebedev
3e43778869
fix: webdav avoid create empty files (#5160) 2024-01-03 09:52:51 -08:00
Konstantin Lebedev
b832ddd1ef
fix: return etag with md5 in webdav responses (#5158) 2024-01-03 08:50:32 -08:00
chrislu
093fdc1621 subscriber can get assignments 2023-12-28 20:35:15 -08:00
Lars Lehtonen
edc0532b1d
weed/server: remove unneeded err from streamReadOneVolume() signature (#5143) 2023-12-28 11:45:13 -08:00
chrislu
bebbc9fe44 create volume grow request if the selected volume is close to full 2023-12-27 11:45:44 -08:00
jerebear12
06343f8976
Set allowed origins in config (#5109) 
* Add a way to use a JWT in an HTTP only cookie

If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.

* Added a way to specify allowed origins header from config

* Removed unecessary log

* Check list of domains from config or command flag

* Handle default wildcard and change name of config value to cors
 2023-12-20 16:21:11 -08:00
chrislu
5fe4c04a1a add missing error in the log 
related to https://github.com/seaweedfs/seaweedfs/issues/5084
 2023-12-07 08:13:42 -08:00
chrislu
756bcc032d adjust logs 2023-11-27 12:57:29 -08:00
Konstantin Lebedev
dd78397fea logging PickForWrite error 
https://github.com/seaweedfs/seaweedfs/issues/3886
 2023-11-27 12:56:15 -08:00
julor
2fa37e0287 Update filer_server_handlers_write_autochunk.go 2023-11-20 11:17:14 -08:00
julor
9e03b916d8 verify content-md5 in header 
if upload content by s3 client and set content-md5 in header,then will verify recived and saved content,if not equal delete thos chunks
 2023-11-20 11:17:14 -08:00
Lars Lehtonen
5db82e594a weed/server: fix dropped webdav error 2023-11-07 19:05:21 -08:00
Konstantin Lebedev
d693beea87
Replace an invalid method in a metric with an invalid one (#4987) 2023-11-06 10:51:47 -08:00
chrislu
a667f0ad99 fix error reporting 
related to https://github.com/seaweedfs/seaweedfs/issues/4978
 2023-11-05 07:17:16 -08:00
Nico D'Cotta
3834ad5c48
refactor webdav subdirectory, fixes #4967 (#4969) 
* refactor webdav subdirectory, fixes #4967

* fix bug where Name() was not called on delegate wrappedFileInfo
 2023-11-01 13:12:49 -07:00
SmsS4
ba11331cc1
Fix twice increase metric counter (#4961) 2023-10-30 08:05:27 -07:00
SmsS4
f95848ba7d
Add ErrorGetNotFound and ErrorGetInternal to volume server metrics (#4960) 2023-10-30 07:38:03 -07:00
zehweh
8b39bbbe2f
fix copying .vif file in VolumeCopy (#4943) 
closes #4934
fixes #2633
might fix #3528
 2023-10-24 09:54:24 -07:00
chrislu
cbc24c7b24 disconnect from old subscribers 2023-10-13 16:48:14 -07:00
Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file (#4881) 
* fix: disallow file name too long when writing a file

* bool LongerName to MaxFilenameLength

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-10-12 14:29:55 -07:00