vadimartynov
86d92a42b4
Added tls for http clients ( #5766 )
...
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
Konstantin Lebedev
67edf1d014
[master] Do Automatic Volume Grow in background ( #5781 )
...
* Do Automatic Volume Grow in backgound
* pass lastGrowCount to master
* fix build
* fix count to uint64
2024-07-16 08:03:40 -07:00
Konstantin Lebedev
04f4b10884
fix: avoid timeout if datacenter does not exist in topology ( #5772 )
...
* fix: avoid timeout if datacenter does not exist in topology
* fix: error msg
* fix: rm dublicate check
* fix: compare
* revert minor change
2024-07-12 11:19:08 -07:00
小羽
cb5dae0c9c
Fix "no more writable volumes" error when volume grows and master leader changed at the same time ( #5771 )
2024-07-12 10:49:15 -07:00
chrislu
4e7d8eb3f1
small refactoring
2024-07-11 14:50:11 -07:00
chrislu
f3d708d3b6
add a test case
2024-07-07 23:44:36 -07:00
Konstantin Lebedev
f77eee667d
add s3test for sql ( #5718 )
...
* add s3test for sql
* fix test test_bucket_listv2_delimiter_basic for s3
* fix action s3tests
* regen s3 api xsd
* rm minor s3 test test_bucket_listv2_fetchowner_defaultempty
* add docs
* without xmlns
2024-07-04 11:00:41 -07:00
chrislu
7c06306857
fix breadcrumb
2024-07-03 17:40:55 -07:00
chrislu
62c29b2398
Revert "directory can paginate"
...
This reverts commit 69003d0e90
2024-07-02 23:55:48 -07:00
chrislu
761a1555de
fix ordering
2024-07-02 23:55:40 -07:00
chrislu
69003d0e90
directory can paginate
2024-07-02 21:36:03 -07:00
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const ( #5726 )
2024-07-01 01:00:39 -07:00
chrislu
ac6fd36c06
using fixed bootstrap from peer filer
2024-06-30 23:16:54 -07:00
chrislu
c030cb3ce9
bootstrap filer from one peer
2024-06-28 14:57:20 -07:00
chrislu
ded761ffc5
adjust visibility
2024-06-24 17:14:52 -07:00
chrislu
07f47776c3
minor
2024-06-23 11:47:26 -07:00
chrislu
464611f614
optionally skip deleting file chunks
2024-06-15 11:39:48 -07:00
chrislu
64fd233a59
DeleteUncommittedChunks
2024-06-15 09:55:22 -07:00
chrislu
677cfb8ad1
refactor
2024-06-15 09:39:49 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops ( #5628 )
...
* Added context for the MasterClient's methods to avoid endless loops
* Returned WithClient function. Added WithClientCustomGetMaster function
* Hid unused ctx arguments
* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions
* Changed the context termination check in the tryConnectToMaster function
* Added a child context to the tryConnectToMaster function
* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
2024-06-14 11:40:34 -07:00
chrislu
2a169dde9a
minor
2024-06-14 09:17:46 -07:00
chrislu
b03c831ad2
more logs
2024-06-13 21:48:44 -07:00
MeenakshiSachdeva
1caa012a76
Fix Filer Sync Issue: 5455 ( #5663 )
2024-06-10 09:19:38 -07:00
steve.wei
2150289442
fix: Ensure that the clientAddress is unique ( #5655 )
2024-06-07 09:13:03 -07:00
Gaspare Iengo
dc6b750424
Fix panic ( #5654 )
2024-06-06 18:59:50 -07:00
steve.wei
d8da4bbaa7
Set the capacity of clientChan to 10000 ( #5647 )
2024-06-05 05:41:46 -07:00
LHHDZ
36b5b713ba
fix deadlock caused by message chan blocked ( #5639 )
2024-06-03 07:42:40 -07:00
chrislu
3e7a92061b
pass along volume server grpc port
...
fix https://github.com/seaweedfs/seaweedfs/issues/5617
2024-05-29 10:41:33 -07:00
chrislu
364bb6c7b4
avoid ticker leak
2024-05-24 17:15:12 -07:00
shenxingwuying
ee25ada732
reduce ambiguity about use memory_sequencer ( #5555 )
2024-04-29 21:51:00 -07:00
chrislu
55976ae04a
avoid repeated calls to heavy-weighted viper
2024-04-18 09:09:45 -07:00
chrislu
d9490c5e1f
rename
2024-04-18 08:47:45 -07:00
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part ( #5476 )
...
* try fix s3 test_multipart_get_part
* add passed s3 tests
* fix SeaweedFSUploadId
* rm spaces
* convert part request to range
* add passed s3 tests of multipart
2024-04-14 10:41:32 -07:00
yudrywet
8833745fbc
chore: fix some typos in comments ( #5497 )
2024-04-14 07:42:40 -07:00
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk ( #5458 )
2024-04-11 04:47:21 -07:00
skycope
6dae685f9c
fix append disorder ( #5492 )
2024-04-10 19:21:44 -07:00
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server ( #5480 )
2024-04-08 07:27:00 -07:00
Konstantin Lebedev
df40908e57
fix panic 5435 ( #5436 )
2024-03-28 16:17:59 -07:00
chrislu
aed6fdffd2
fix nil volume data backend
...
fix https://github.com/seaweedfs/seaweedfs/issues/5417
2024-03-25 08:19:31 -07:00
jerebear12
7091269e65
fix get file metadata bug ( #5394 )
...
* Removed problematic if statement
This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.
* Update filer_server_handers_read.go
* Updated filer responses to invalid requests
2024-03-18 11:08:16 -07:00
chrislu
6a61b54f29
Merge branch 'mq-subscribe'
2024-03-16 11:42:23 -07:00
chrislu
27bb38228b
only broad cast when there are waiting threads
2024-03-16 11:38:27 -07:00
Konstantin Lebedev
54ee73287d
[filer] replace an invalid methods in a metric with an invalid one ( #5378 )
...
fix: replace an invalid methods in a metric with an invalid one for filer
2024-03-14 08:19:09 -07:00
chrislu
6f615c62ae
Merge branch 'master' into mq-subscribe
2024-03-12 09:19:06 -07:00
rustrover
ab70aa92da
remove repetitive words ( #5364 )
2024-03-09 07:35:47 -08:00
chrislu
34f2b600ac
each log function adds a "done" return parameter
2024-03-07 10:50:09 -08:00
chrislu
832b60c13f
pass errors
2024-03-07 10:47:06 -08:00
chrislu
8f79bb3987
Merge branch 'master' into mq-subscribe
2024-03-07 10:42:29 -08:00
chrislu
5c0f01c0eb
purge cluster id
2024-03-07 10:42:12 -08:00
chrislu
2a7028373d
Merge branch 'master' into mq-subscribe
2024-02-29 06:22:39 -08:00
jerebear12
85d6d5371b
Disable filer UI in configuration ( #5297 )
...
* Add filer.ui.enabled configuration property
* Add filer.expose_directory_metadata to config
* Ammend commit
* Remove ShowUI reference
* Update all routes that allow directory metadata
* Add cmd flag to server.go
2024-02-27 08:38:55 -08:00
chrislu
a52ce076fc
Merge branch 'master' into mq-subscribe
2024-02-25 08:00:37 -08:00
Sébastien
c694cb4e7d
filer: reduce delete entry ErrNotFound logs ( #5301 )
2024-02-14 14:22:08 -08:00
Konstantin Lebedev
8fb978d6f6
[volume] requests that were canceled by the client are not an internal server problem(500 => 499) ( #5304 )
2024-02-14 14:18:56 -08:00
Chris Lu
fa72761bb8
fix on finding lock owner
2024-02-05 17:30:14 -08:00
Chris Lu
53612b770c
Merge branch 'master' into mq-subscribe
2024-02-04 10:44:01 -08:00
chrislu
d41792461c
lock returns host and owner
2024-02-02 15:54:57 -08:00
chrislu
d30150dde1
use a short-lived lock
2024-02-01 23:01:44 -08:00
chrislu
c1bb76d709
fix nil error
2024-01-30 08:21:42 -08:00
chrislu
ac50d8a822
re-lock if the lock owner is not found
2024-01-29 22:46:23 -08:00
Sébastien
0775d05a23
fix: http range request return status 500 ( #5251 )
...
When volume server unavailable for at least one chunk; was returning status 206.
Split `StreamContent` in two parts,
- first prepare, to get chunk info and return stream function
- then write chunk, with that stream function
That allow to catch error in first step before setting response status code in `processRangeRequest`
2024-01-29 10:35:52 -08:00
Konstantin Lebedev
0b71fae2e3
fix: s3tests test_get_object_ifnonematch_good ( #5237 )
2024-01-24 11:05:55 -08:00
chrislu
a8ddbf9ed2
Merge branch 'master' into mq-subscribe
2024-01-23 19:41:16 -08:00
chrislu
49fcb48e04
same logic as grpc assign
...
fix https://github.com/seaweedfs/seaweedfs/issues/5213
2024-01-23 19:39:59 -08:00
chrislu
263f1f3d04
Merge branch 'master' into mq-subscribe
2024-01-18 09:16:46 -08:00
chrislu
3002087541
http assign logic should be the same as grpc assign
...
similar logic as bebbc9fe44
2024-01-18 08:58:53 -08:00
Konstantin Lebedev
4e9ea1e628
chore: filer healthz handler check filer store ( #5208 )
2024-01-17 07:46:37 -08:00
Konstantin Lebedev
f9cf13fada
[filer] avoid return http ok for not allowed methods ( #5209 )
2024-01-17 07:17:07 -08:00
chrislu
fa59a5d67e
read from disk if not in memory
2024-01-15 00:20:12 -08:00
chrislu
70a011a224
Merge branch 'master' into mq-subscribe
2024-01-11 20:58:57 -08:00
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics ( #5188 )
2024-01-10 10:05:27 -08:00
jerebear12
8c966ac23b
Removed problematic if statement ( #5180 )
...
This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.
2024-01-08 11:35:20 -08:00
chrislu
49428a303b
add batch index for each memory buffer
2024-01-08 00:03:08 -08:00
chrislu
d9bd343055
Merge branch 'master' into mq-subscribe
2024-01-05 15:36:00 -08:00
Konstantin Lebedev
3b62b338a5
chore: add maxMB option for webdav ( #5165 )
2024-01-05 11:10:20 -08:00
Konstantin Lebedev
3e43778869
fix: webdav avoid create empty files ( #5160 )
2024-01-03 09:52:51 -08:00
Konstantin Lebedev
b832ddd1ef
fix: return etag with md5 in webdav responses ( #5158 )
2024-01-03 08:50:32 -08:00
chrislu
093fdc1621
subscriber can get assignments
2023-12-28 20:35:15 -08:00
Lars Lehtonen
edc0532b1d
weed/server: remove unneeded err from streamReadOneVolume() signature ( #5143 )
2023-12-28 11:45:13 -08:00
chrislu
bebbc9fe44
create volume grow request if the selected volume is close to full
2023-12-27 11:45:44 -08:00
jerebear12
06343f8976
Set allowed origins in config ( #5109 )
...
* Add a way to use a JWT in an HTTP only cookie
If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.
* Added a way to specify allowed origins header from config
* Removed unecessary log
* Check list of domains from config or command flag
* Handle default wildcard and change name of config value to cors
2023-12-20 16:21:11 -08:00
chrislu
5fe4c04a1a
add missing error in the log
...
related to https://github.com/seaweedfs/seaweedfs/issues/5084
2023-12-07 08:13:42 -08:00
chrislu
756bcc032d
adjust logs
2023-11-27 12:57:29 -08:00
Konstantin Lebedev
dd78397fea
logging PickForWrite error
...
https://github.com/seaweedfs/seaweedfs/issues/3886
2023-11-27 12:56:15 -08:00
julor
2fa37e0287
Update filer_server_handlers_write_autochunk.go
2023-11-20 11:17:14 -08:00
julor
9e03b916d8
verify content-md5 in header
...
if upload content by s3 client and set content-md5 in header,then will verify recived and saved content,if not equal delete thos chunks
2023-11-20 11:17:14 -08:00
Lars Lehtonen
5db82e594a
weed/server: fix dropped webdav error
2023-11-07 19:05:21 -08:00
Konstantin Lebedev
d693beea87
Replace an invalid method in a metric with an invalid one ( #4987 )
2023-11-06 10:51:47 -08:00
chrislu
a667f0ad99
fix error reporting
...
related to https://github.com/seaweedfs/seaweedfs/issues/4978
2023-11-05 07:17:16 -08:00
Nico D'Cotta
3834ad5c48
refactor webdav subdirectory, fixes #4967 ( #4969 )
...
* refactor webdav subdirectory, fixes #4967
* fix bug where Name() was not called on delegate wrappedFileInfo
2023-11-01 13:12:49 -07:00
SmsS4
ba11331cc1
Fix twice increase metric counter ( #4961 )
2023-10-30 08:05:27 -07:00
SmsS4
f95848ba7d
Add ErrorGetNotFound and ErrorGetInternal to volume server metrics ( #4960 )
2023-10-30 07:38:03 -07:00
zehweh
8b39bbbe2f
fix copying .vif file in VolumeCopy ( #4943 )
...
closes #4934
fixes #2633
might fix #3528
2023-10-24 09:54:24 -07:00
chrislu
cbc24c7b24
disconnect from old subscribers
2023-10-13 16:48:14 -07:00
Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file ( #4881 )
...
* fix: disallow file name too long when writing a file
* bool LongerName to MaxFilenameLength
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 14:29:55 -07:00
Konstantin Lebedev
edee91ef0e
fix has changes false if deleted result size eq zero ( #4909 )
...
* fix has changes false if deleted result size eq zero
https://github.com/seaweedfs/seaweedfs/issues/4171
* fix change respos status 202 to 304 if the file has already been deleted
https://github.com/seaweedfs/seaweedfs/issues/4171
* fix delete if set apply changes
https://github.com/seaweedfs/seaweedfs/issues/4171
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 08:15:20 -07:00
Konstantin Lebedev
2b3e39397e
fix: skipping checking active volumes with the same number of files at the moment ( #4893 )
...
* fix: skipping checking active volumes with the same number of files at the moment
https://github.com/seaweedfs/seaweedfs/issues/4140
* refactor with comments
https://github.com/seaweedfs/seaweedfs/issues/4140
* add TestShouldSkipVolume
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-09 09:57:26 -07:00
Yuval Yacoby
3fe00996b2
added healthz endpoint to filer ( #4899 )
2023-10-08 07:03:18 -07:00
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
...
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
d531b7cbbf
merge some previous missing changes
2023-09-19 14:10:54 -07:00
