1
0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-28 11:57:54 +08:00
Commit Graph

1639 Commits

Author SHA1 Message Date
Konstantin Lebedev
f77eee667d
add s3test for sql ()
* add s3test for sql

* fix test test_bucket_listv2_delimiter_basic for s3

* fix action s3tests

* regen s3 api xsd

* rm minor s3 test test_bucket_listv2_fetchowner_defaultempty

* add docs

* without xmlns
2024-07-04 11:00:41 -07:00
chrislu
7c06306857 fix breadcrumb 2024-07-03 17:40:55 -07:00
chrislu
62c29b2398 Revert "directory can paginate"
This reverts commit 69003d0e90.
2024-07-02 23:55:48 -07:00
chrislu
761a1555de fix ordering 2024-07-02 23:55:40 -07:00
chrislu
69003d0e90 directory can paginate 2024-07-02 21:36:03 -07:00
Konstantin Lebedev
5ffacbb6ea
refactor all methods strings to const () 2024-07-01 01:00:39 -07:00
chrislu
ac6fd36c06 using fixed bootstrap from peer filer 2024-06-30 23:16:54 -07:00
chrislu
c030cb3ce9 bootstrap filer from one peer 2024-06-28 14:57:20 -07:00
chrislu
ded761ffc5 adjust visibility 2024-06-24 17:14:52 -07:00
chrislu
07f47776c3 minor 2024-06-23 11:47:26 -07:00
chrislu
464611f614 optionally skip deleting file chunks 2024-06-15 11:39:48 -07:00
chrislu
64fd233a59 DeleteUncommittedChunks 2024-06-15 09:55:22 -07:00
chrislu
677cfb8ad1 refactor 2024-06-15 09:39:49 -07:00
vadimartynov
8aae82dd71
Added context for the MasterClient's methods to avoid endless loops ()
* Added context for the MasterClient's methods to avoid endless loops

* Returned WithClient function. Added WithClientCustomGetMaster function

* Hid unused ctx arguments

* Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions

* Changed the context termination check in the tryConnectToMaster function

* Added a child context to the tryConnectToMaster function

* Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
2024-06-14 11:40:34 -07:00
chrislu
2a169dde9a minor 2024-06-14 09:17:46 -07:00
chrislu
b03c831ad2 more logs 2024-06-13 21:48:44 -07:00
MeenakshiSachdeva
1caa012a76
Fix Filer Sync Issue: 5455 () 2024-06-10 09:19:38 -07:00
steve.wei
2150289442
fix: Ensure that the clientAddress is unique () 2024-06-07 09:13:03 -07:00
Gaspare Iengo
dc6b750424
Fix panic () 2024-06-06 18:59:50 -07:00
steve.wei
d8da4bbaa7
Set the capacity of clientChan to 10000 () 2024-06-05 05:41:46 -07:00
LHHDZ
36b5b713ba
fix deadlock caused by message chan blocked () 2024-06-03 07:42:40 -07:00
chrislu
3e7a92061b pass along volume server grpc port
fix https://github.com/seaweedfs/seaweedfs/issues/5617
2024-05-29 10:41:33 -07:00
chrislu
364bb6c7b4 avoid ticker leak 2024-05-24 17:15:12 -07:00
shenxingwuying
ee25ada732
reduce ambiguity about use memory_sequencer () 2024-04-29 21:51:00 -07:00
chrislu
55976ae04a avoid repeated calls to heavy-weighted viper 2024-04-18 09:09:45 -07:00
chrislu
d9490c5e1f rename 2024-04-18 08:47:45 -07:00
Konstantin Lebedev
33537ae29f
[s3] fix s3 test_multipart_get_part ()
* try fix s3  test_multipart_get_part

* add passed s3 tests

* fix SeaweedFSUploadId

* rm spaces

* convert part request to range

* add passed s3 tests of multipart
2024-04-14 10:41:32 -07:00
yudrywet
8833745fbc
chore: fix some typos in comments () 2024-04-14 07:42:40 -07:00
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk () 2024-04-11 04:47:21 -07:00
skycope
6dae685f9c
fix append disorder () 2024-04-10 19:21:44 -07:00
Henco Appel
5c8e6014ba
fix: filer authenticate with with volume server () 2024-04-08 07:27:00 -07:00
Konstantin Lebedev
df40908e57
fix panic 5435 () 2024-03-28 16:17:59 -07:00
chrislu
aed6fdffd2 fix nil volume data backend
fix https://github.com/seaweedfs/seaweedfs/issues/5417
2024-03-25 08:19:31 -07:00
jerebear12
7091269e65
fix get file metadata bug ()
* Removed problematic if statement

This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.

* Update filer_server_handers_read.go

* Updated filer responses to invalid requests
2024-03-18 11:08:16 -07:00
chrislu
6a61b54f29 Merge branch 'mq-subscribe' 2024-03-16 11:42:23 -07:00
chrislu
27bb38228b only broad cast when there are waiting threads 2024-03-16 11:38:27 -07:00
Konstantin Lebedev
54ee73287d
[filer] replace an invalid methods in a metric with an invalid one ()
fix: replace an invalid methods in a metric with an invalid one for filer
2024-03-14 08:19:09 -07:00
chrislu
6f615c62ae Merge branch 'master' into mq-subscribe 2024-03-12 09:19:06 -07:00
rustrover
ab70aa92da
remove repetitive words () 2024-03-09 07:35:47 -08:00
chrislu
34f2b600ac each log function adds a "done" return parameter 2024-03-07 10:50:09 -08:00
chrislu
832b60c13f pass errors 2024-03-07 10:47:06 -08:00
chrislu
8f79bb3987 Merge branch 'master' into mq-subscribe 2024-03-07 10:42:29 -08:00
chrislu
5c0f01c0eb purge cluster id 2024-03-07 10:42:12 -08:00
chrislu
2a7028373d Merge branch 'master' into mq-subscribe 2024-02-29 06:22:39 -08:00
jerebear12
85d6d5371b
Disable filer UI in configuration ()
* Add filer.ui.enabled configuration property

* Add filer.expose_directory_metadata to config

* Ammend commit

* Remove ShowUI reference

* Update all routes that allow directory metadata

* Add cmd flag to server.go
2024-02-27 08:38:55 -08:00
chrislu
a52ce076fc Merge branch 'master' into mq-subscribe 2024-02-25 08:00:37 -08:00
Sébastien
c694cb4e7d
filer: reduce delete entry ErrNotFound logs () 2024-02-14 14:22:08 -08:00
Konstantin Lebedev
8fb978d6f6
[volume] requests that were canceled by the client are not an internal server problem(500 => 499) () 2024-02-14 14:18:56 -08:00
Chris Lu
fa72761bb8 fix on finding lock owner 2024-02-05 17:30:14 -08:00
Chris Lu
53612b770c Merge branch 'master' into mq-subscribe 2024-02-04 10:44:01 -08:00
chrislu
d41792461c lock returns host and owner 2024-02-02 15:54:57 -08:00
chrislu
d30150dde1 use a short-lived lock 2024-02-01 23:01:44 -08:00
chrislu
c1bb76d709 fix nil error 2024-01-30 08:21:42 -08:00
chrislu
ac50d8a822 re-lock if the lock owner is not found 2024-01-29 22:46:23 -08:00
Sébastien
0775d05a23
fix: http range request return status 500 ()
When volume server unavailable for at least one chunk; was returning status 206.

Split `StreamContent` in two parts,
- first prepare, to get chunk info and return stream function
- then write chunk, with that stream function

That allow to catch error in first step before setting response status code in `processRangeRequest`
2024-01-29 10:35:52 -08:00
Konstantin Lebedev
0b71fae2e3
fix: s3tests test_get_object_ifnonematch_good () 2024-01-24 11:05:55 -08:00
chrislu
a8ddbf9ed2 Merge branch 'master' into mq-subscribe 2024-01-23 19:41:16 -08:00
chrislu
49fcb48e04 same logic as grpc assign
fix https://github.com/seaweedfs/seaweedfs/issues/5213
2024-01-23 19:39:59 -08:00
chrislu
263f1f3d04 Merge branch 'master' into mq-subscribe 2024-01-18 09:16:46 -08:00
chrislu
3002087541 http assign logic should be the same as grpc assign
similar logic as bebbc9fe44
2024-01-18 08:58:53 -08:00
Konstantin Lebedev
4e9ea1e628
chore: filer healthz handler check filer store () 2024-01-17 07:46:37 -08:00
Konstantin Lebedev
f9cf13fada
[filer] avoid return http ok for not allowed methods () 2024-01-17 07:17:07 -08:00
chrislu
fa59a5d67e read from disk if not in memory 2024-01-15 00:20:12 -08:00
chrislu
70a011a224 Merge branch 'master' into mq-subscribe 2024-01-11 20:58:57 -08:00
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics () 2024-01-10 10:05:27 -08:00
jerebear12
8c966ac23b
Removed problematic if statement ()
This if statement was causing the value of option.AllowedOrigins to be always equal to "*". Now the values in the config file will be used when present. This allows for people who don't need this feature to not update their security.toml files.
2024-01-08 11:35:20 -08:00
chrislu
49428a303b add batch index for each memory buffer 2024-01-08 00:03:08 -08:00
chrislu
d9bd343055 Merge branch 'master' into mq-subscribe 2024-01-05 15:36:00 -08:00
Konstantin Lebedev
3b62b338a5
chore: add maxMB option for webdav () 2024-01-05 11:10:20 -08:00
Konstantin Lebedev
3e43778869
fix: webdav avoid create empty files () 2024-01-03 09:52:51 -08:00
Konstantin Lebedev
b832ddd1ef
fix: return etag with md5 in webdav responses () 2024-01-03 08:50:32 -08:00
chrislu
093fdc1621 subscriber can get assignments 2023-12-28 20:35:15 -08:00
Lars Lehtonen
edc0532b1d
weed/server: remove unneeded err from streamReadOneVolume() signature () 2023-12-28 11:45:13 -08:00
chrislu
bebbc9fe44 create volume grow request if the selected volume is close to full 2023-12-27 11:45:44 -08:00
jerebear12
06343f8976
Set allowed origins in config ()
* Add a way to use a JWT in an HTTP only cookie

If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie.

* Added a way to specify allowed origins header from config

* Removed unecessary log

* Check list of domains from config or command flag

* Handle default wildcard and change name of config value to cors
2023-12-20 16:21:11 -08:00
chrislu
5fe4c04a1a add missing error in the log
related to https://github.com/seaweedfs/seaweedfs/issues/5084
2023-12-07 08:13:42 -08:00
chrislu
756bcc032d adjust logs 2023-11-27 12:57:29 -08:00
Konstantin Lebedev
dd78397fea logging PickForWrite error
https://github.com/seaweedfs/seaweedfs/issues/3886
2023-11-27 12:56:15 -08:00
julor
2fa37e0287 Update filer_server_handlers_write_autochunk.go 2023-11-20 11:17:14 -08:00
julor
9e03b916d8 verify content-md5 in header
if upload content by s3 client and set content-md5 in header,then will verify recived and saved content,if not equal delete thos chunks
2023-11-20 11:17:14 -08:00
Lars Lehtonen
5db82e594a weed/server: fix dropped webdav error 2023-11-07 19:05:21 -08:00
Konstantin Lebedev
d693beea87
Replace an invalid method in a metric with an invalid one () 2023-11-06 10:51:47 -08:00
chrislu
a667f0ad99 fix error reporting
related to https://github.com/seaweedfs/seaweedfs/issues/4978
2023-11-05 07:17:16 -08:00
Nico D'Cotta
3834ad5c48
refactor webdav subdirectory, fixes ()
* refactor webdav subdirectory, fixes 

* fix bug where Name() was not called on delegate wrappedFileInfo
2023-11-01 13:12:49 -07:00
SmsS4
ba11331cc1
Fix twice increase metric counter () 2023-10-30 08:05:27 -07:00
SmsS4
f95848ba7d
Add ErrorGetNotFound and ErrorGetInternal to volume server metrics () 2023-10-30 07:38:03 -07:00
zehweh
8b39bbbe2f
fix copying .vif file in VolumeCopy ()
closes 
fixes 
might fix 
2023-10-24 09:54:24 -07:00
chrislu
cbc24c7b24 disconnect from old subscribers 2023-10-13 16:48:14 -07:00
Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file ()
* fix: disallow file name too long when writing a file

* bool LongerName to MaxFilenameLength

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 14:29:55 -07:00
Konstantin Lebedev
edee91ef0e
fix has changes false if deleted result size eq zero ()
* fix has changes false if deleted result size eq zero
https://github.com/seaweedfs/seaweedfs/issues/4171

* fix change respos status 202 to 304 if the file has already been deleted
https://github.com/seaweedfs/seaweedfs/issues/4171

* fix delete if set apply changes
https://github.com/seaweedfs/seaweedfs/issues/4171

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 08:15:20 -07:00
Konstantin Lebedev
2b3e39397e
fix: skipping checking active volumes with the same number of files at the moment ()
* fix: skipping checking active volumes with the same number of files at the moment
 https://github.com/seaweedfs/seaweedfs/issues/4140

* refactor with comments
https://github.com/seaweedfs/seaweedfs/issues/4140

* add TestShouldSkipVolume

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-09 09:57:26 -07:00
Yuval Yacoby
3fe00996b2
added healthz endpoint to filer () 2023-10-08 07:03:18 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs""
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
d531b7cbbf merge some previous missing changes 2023-09-19 14:10:54 -07:00
chrislu
27af11f1e8 Revert "Revert "Merge branch 'master' into sub""
This reverts commit 0bb97709d4.
2023-09-18 18:47:34 -07:00
chrislu
0bb97709d4 Revert "Merge branch 'master' into sub"
This reverts commit 4d414f54a2, reversing
changes made to 4827425146.
2023-09-18 16:13:20 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
2023-09-18 16:12:50 -07:00
chrislu
2e5aa06026 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-09-18 16:06:41 -07:00
chrislu
4d414f54a2 Merge branch 'master' into sub 2023-09-18 16:04:59 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ()
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2023-09-18 14:43:05 -07:00