Commit Graph

123 Commits

Author SHA1 Message Date
vadimartynov
86d92a42b4
Added tls for http clients (#5766)
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
vadimartynov
b796c21fa9
Added loadSecurityConfigOnce (#5792) 2024-07-16 09:15:55 -07:00
chrislu
3a82f5ffad ensure metadata follow a specific folder
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12 11:17:30 -07:00
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const (#5726) 2024-07-01 01:00:39 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops (#5628)
* Added context for the MasterClient's methods to avoid endless loops

* Returned WithClient function. Added WithClientCustomGetMaster function

* Hid unused ctx arguments

* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions

* Changed the context termination check in the tryConnectToMaster function

* Added a child context to the tryConnectToMaster function

* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
2024-06-14 11:40:34 -07:00
chrislu
364bb6c7b4 avoid ticker leak 2024-05-24 17:15:12 -07:00
binbinshi
30b48ad614
see_dat add print name (#4930) 2023-10-20 06:13:54 -07:00
wusong
670e2185e4
fix: fid mismatch (#4554)
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-07 12:15:19 -07:00
chrislu
5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
James Hartig
4c85da7844
Include meta in ReadAllNeedles (#3991)
This is useful for doing backups on the data so we can accurately store the
last modified time, the compression state, and verify the crc.

Previously we were doing VolumeNeedleStatus and then an HTTP request which
needlessly read from the dat file twice.
2022-11-20 20:19:41 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
870a30665b close responses 2022-08-31 00:24:17 -07:00
chrislu
4ffbda1c43 fix building for unmaintained 2022-08-17 14:45:38 -07:00
chrislu
eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu
13b9a52f80 fix deprecated functions 2022-07-29 01:34:39 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
guol-fnst
ac694f0c8f rename parameter and reuse functions
rename milestone to  watermark
2022-07-20 17:00:40 +08:00
creeew
02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
chrislu
a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
Nat Makarevitch
b72f0634d7
minor (typos...), done while reading around 2022-05-16 22:11:33 +08:00
chrislu
5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
225b019fe0 stream read multiple volumes in a volume server 2021-09-27 02:51:31 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
67949e25e9 add optional delay 2021-09-11 02:21:55 -07:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu
6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
byunghwa.yun
285fdd2dd5 Add the additional meta log 2021-07-29 17:43:17 +09:00
Chris Lu
cc7714fdbe logging changes to debug 2021-06-27 05:54:16 -07:00
Chris Lu
a852f8a3f5 some code to stress test the meta data events 2021-05-22 18:03:13 -07:00
Chris Lu
2586621362 minor 2021-05-22 01:02:31 -07:00
bingoohuang
cf552417a7 minFreeSpace refactored 2021-04-27 10:37:24 +08:00
Chris Lu
7635f6b9fa disk file avoid file.Stat() 2021-02-20 20:06:06 -08:00
Chris Lu
6daa932f5c refactoring to get master function, instead of passing master values directly
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
487e435679 adjust http max idle connections per host
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
bingoohuang
7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
Chris Lu
ec052f75d0 add example scripts 2020-11-17 13:14:03 -08:00
Chris Lu
283d584141 update to correct block size 2020-10-25 22:03:37 -07:00
Chris Lu
c693c98c15 write a lot of files 2020-10-25 19:28:11 -07:00
Chris Lu
3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu
f2723c1bc8 do not idx file format
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
75de7002ff adjust size 2020-09-11 11:43:13 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu
332caf0cd7 maintain the unmaintained 2020-08-18 17:23:01 -07:00