julor
2fa37e0287
Update filer_server_handlers_write_autochunk.go
2023-11-20 11:17:14 -08:00
julor
9e03b916d8
verify content-md5 in header
...
if upload content by s3 client and set content-md5 in header,then will verify recived and saved content,if not equal delete thos chunks
2023-11-20 11:17:14 -08:00
Lars Lehtonen
5db82e594a
weed/server: fix dropped webdav error
2023-11-07 19:05:21 -08:00
Konstantin Lebedev
d693beea87
Replace an invalid method in a metric with an invalid one ( #4987 )
2023-11-06 10:51:47 -08:00
chrislu
a667f0ad99
fix error reporting
...
related to https://github.com/seaweedfs/seaweedfs/issues/4978
2023-11-05 07:17:16 -08:00
Nico D'Cotta
3834ad5c48
refactor webdav subdirectory, fixes #4967 ( #4969 )
...
* refactor webdav subdirectory, fixes #4967
* fix bug where Name() was not called on delegate wrappedFileInfo
2023-11-01 13:12:49 -07:00
SmsS4
ba11331cc1
Fix twice increase metric counter ( #4961 )
2023-10-30 08:05:27 -07:00
SmsS4
f95848ba7d
Add ErrorGetNotFound and ErrorGetInternal to volume server metrics ( #4960 )
2023-10-30 07:38:03 -07:00
zehweh
8b39bbbe2f
fix copying .vif file in VolumeCopy ( #4943 )
...
closes #4934
fixes #2633
might fix #3528
2023-10-24 09:54:24 -07:00
chrislu
cbc24c7b24
disconnect from old subscribers
2023-10-13 16:48:14 -07:00
Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file ( #4881 )
...
* fix: disallow file name too long when writing a file
* bool LongerName to MaxFilenameLength
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 14:29:55 -07:00
Konstantin Lebedev
edee91ef0e
fix has changes false if deleted result size eq zero ( #4909 )
...
* fix has changes false if deleted result size eq zero
https://github.com/seaweedfs/seaweedfs/issues/4171
* fix change respos status 202 to 304 if the file has already been deleted
https://github.com/seaweedfs/seaweedfs/issues/4171
* fix delete if set apply changes
https://github.com/seaweedfs/seaweedfs/issues/4171
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-12 08:15:20 -07:00
Konstantin Lebedev
2b3e39397e
fix: skipping checking active volumes with the same number of files at the moment ( #4893 )
...
* fix: skipping checking active volumes with the same number of files at the moment
https://github.com/seaweedfs/seaweedfs/issues/4140
* refactor with comments
https://github.com/seaweedfs/seaweedfs/issues/4140
* add TestShouldSkipVolume
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-09 09:57:26 -07:00
Yuval Yacoby
3fe00996b2
added healthz endpoint to filer ( #4899 )
2023-10-08 07:03:18 -07:00
chrislu
645ae8c57b
Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs ""
...
This reverts commit 8cb42c39
2023-09-25 09:35:16 -07:00
chrislu
d531b7cbbf
merge some previous missing changes
2023-09-19 14:10:54 -07:00
chrislu
27af11f1e8
Revert "Revert "Merge branch 'master' into sub""
...
This reverts commit 0bb97709d4
2023-09-18 18:47:34 -07:00
chrislu
0bb97709d4
Revert "Merge branch 'master' into sub"
...
This reverts commit 4d414f54a24827425146
2023-09-18 16:13:20 -07:00
chrislu
8cb42c39ad
Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs "
...
This reverts commit 2e5aa060264d414f54a2
2023-09-18 16:12:50 -07:00
chrislu
2e5aa06026
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-18 16:06:41 -07:00
chrislu
4d414f54a2
Merge branch 'master' into sub
2023-09-18 16:04:59 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 ( #4850 )
...
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0
Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone ) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases )
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md )
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0 )
---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* API changes
* go mod
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2023-09-18 14:43:05 -07:00
chrislu
b64fc44e8d
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2023-09-18 09:01:17 -07:00
chrislu
0d390ec882
fix nil
2023-09-18 08:52:35 -07:00
zemul
91cbaef02b
fix: mount proxyByFiler write error: wrong jwt
2023-09-18 07:44:50 -07:00
chrislu
a1cb49d078
adjust log
2023-09-16 23:00:28 -07:00
chrislu
b590a68e75
revert to non-streaming mode to assign file id
...
fix https://github.com/seaweedfs/seaweedfs/issues/4838#issuecomment-1722290685
2023-09-16 16:13:08 -07:00
chrislu
4650c4c65f
Revert "turn on streaming assign file id"
...
This reverts commit 733db2bc88
2023-09-16 16:09:20 -07:00
chrislu
89a1fd1751
Squashed commit of the following:
...
commit 48274251463b50139f687f685ce7ba436d99443bb771fefa370a851ec008908810370a851ec00b39941edc0b875f562779984b6c54cf2492a45499ba67e6ca299e4f985698cb470d44df1eb2da46d5504ae8383adbcba75271c9caf331194d6c18d86f4eb8e8624d1990456670905911853d
2023-09-16 15:06:16 -07:00
chrislu
4827425146
balancer works
2023-09-16 15:05:38 -07:00
Nico D'Cotta
796b7508f3
Implement SRV lookups for filer ( #4767 )
2023-08-24 07:08:56 -07:00
chrislu
733db2bc88
turn on streaming assign file id
2023-08-23 00:34:21 -07:00
chrislu
99f037b958
streaming assign file ids
2023-08-23 00:31:33 -07:00
chrislu
94b7e2a37c
add stream assign server side implementation
2023-08-22 09:59:04 -07:00
chrislu
ccedad5196
refactor files
2023-08-22 09:54:06 -07:00
wusong
33b9192536
Fix large file chunks deleted by mistake ( #4678 )
2023-07-17 07:52:14 -07:00
Konstantin Lebedev
2c076ebd96
avoid logging nil error ( #4659 )
...
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-07-10 23:38:52 -07:00
chrislu
a315490f7d
proxy to master uses http address
...
fix https://github.com/seaweedfs/seaweedfs/issues/4607
2023-07-04 11:45:21 -07:00
chrislu
f193942adc
lock on append metadata operation
2023-06-25 22:07:23 -07:00
chrislu
61c42f9991
adjust lock APIs
2023-06-25 20:30:20 -07:00
chrislu
5149b3d07b
filer can proxy to peer filer holding the lock
2023-06-25 15:28:01 -07:00
chrislu
868f7875d7
refactor
2023-06-25 14:30:58 -07:00
chrislu
464a71a373
add distributed lock manager
2023-06-25 14:14:40 -07:00
chrislu
3fd659df2a
add distributed lock manager
2023-06-25 00:58:21 -07:00
chrislu
8ec1bc2c99
remove unused cluster node leader
2023-06-19 18:19:13 -07:00
chrislu
94fbf02ea2
generate jwt token when looking up needle id
...
fix https://github.com/seaweedfs/seaweedfs/issues/4577
2023-06-17 00:11:23 -07:00
Konstantin Lebedev
25535e9c36
Delete volume is empty ( #4561 )
...
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559
* fix IsEmpty
* fix test
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 10:42:44 -07:00
柏杰
68feffa4e6
avoid overwriting non-nil uploadReaderToChunks.uploadErr with nil value ( #4560 )
2023-06-10 23:15:27 -07:00
Konstantin Lebedev
5ee04d20fa
Healthz check for deadlocks ( #4558 )
2023-06-09 09:42:48 -07:00
chrislu
0bffb3770f
support TTL when creating folders
...
related to https://github.com/seaweedfs/seaweedfs/discussions/4549
2023-06-05 23:29:25 -07:00
wusong
26f15d0079
Fix no more writable volumes by delay judgment ( #4548 )
...
* fix nomore writables volumes while disk free space is sufficient by time delay
* reset
---------
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-05 10:17:21 -07:00
wusong
5aec6da8a3
fix deadlock for filer upload ( #4527 )
...
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-03 13:38:27 -07:00
wusong
e7f4418855
add return for error handle ( #4502 )
2023-05-24 07:45:06 -07:00
chrislu
d999f1f0e2
update raft version
...
fix #4460
2023-05-09 22:54:23 -07:00
chrislu
4511edc871
update raft
2023-05-07 13:33:44 -07:00
zemul
e9fda774f4
[Filer] post add param:saveInside ( #4434 )
...
* fix:mount deadlock
* feat: filer http upload to metadata
* feat: /etc save inside
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-29 08:31:05 -07:00
wusong
4867aa03ab
Fix: http rename move dir to subdir ( #4432 )
...
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-04-26 20:54:36 -07:00
zemul
f352616b7d
Volume range read use sync.pool ( #4422 )
2023-04-24 07:00:48 -07:00
wusong
a408b46d95
compilation fail ( #4414 )
...
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-18 00:36:55 -07:00
chrislu
0a22eea55d
collect ec shard from multiple locations
...
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-17 22:56:21 -07:00
chrislu
ae9388723f
adjust error message
2023-04-17 22:01:39 -07:00
Konstantin Lebedev
130bc3e668
s3 fix get fake dir object key ( #4390 )
2023-04-11 07:36:22 -07:00
wusong
2b82d0f871
style: clerical error ( #4361 )
2023-03-31 07:36:21 -07:00
Konstantin Lebedev
de4545c28b
add validate config for raft ( #4332 )
2023-03-21 06:36:13 -07:00
chrislu
adb90bd252
avoid lower casing the command
...
fix https://github.com/seaweedfs/seaweedfs/pull/4321
2023-03-19 21:20:46 -07:00
chrislu
784daea1fa
fix volume not found if marked as read only
...
fix https://github.com/seaweedfs/seaweedfs/issues/4088
2023-03-19 18:30:13 -07:00
LHHDZ
d695119073
The filer does not set defaultReplication to the defaultReplication o… ( #4315 )
...
The filer does not set defaultReplication to the defaultReplication of the master (it is not necessary, because if the filer is not set, the configuration of the master will be used when calling Assign), otherwise, when the defaultReplication of the master is modified and all master nodes are restarted, the defaultReplication will not take effect (because the filer Keep the previous defaultReplication setting from the master)
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-03-15 22:10:39 -07:00
Rohit Chormale
59706c89fb
#4270 set http status code to 409 if dir already exists ( #4287 )
2023-03-08 07:45:08 -08:00
zemul
0bf56298d5
fix chunk.ModifiedTsNs ( #4264 )
...
* fix
* fix mtime s > ns
---------
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:24:36 -08:00
chrislu
8241c9a829
adjust ui pagination size
...
fix https://github.com/seaweedfs/seaweedfs/issues/4258
2023-02-28 08:31:26 -08:00
lfhy
1976ca9160
add -disk to filer command ( #4247 )
...
* add -disk to filer command
* add diskType to filer.grpc
* use filer.disk when filerWebDavOptions.disk is empty
* add filer.disk to weed server command.
---------
Co-authored-by: 三千院羽 <3000y@MacBook-Pro.lan>
2023-02-25 09:48:59 -08:00
Zachary Walters
ef2f741823
Updated the deprecated ioutil dependency ( #4239 )
2023-02-21 19:47:33 -08:00
Neo
d5f77706a8
volume:fix return if Replicate write error ( #4188 )
2023-02-06 22:34:56 -08:00
chrislu
c47f75eb5b
fix null content type caused exception in hdfs client
...
fix https://github.com/seaweedfs/seaweedfs/issues/4170
2023-02-01 02:38:05 -08:00
a
0d085ec175
make directory lister trigger if mime type is httpd/unix-directory ( #4169 )
2023-01-31 06:54:40 -08:00
chrislu
81fdf3651b
grpc connection to filer add sw-client-id header
2023-01-20 01:48:12 -08:00
chrislu
bfe5d910c6
use one readerCache for the whole file
2023-01-16 22:43:02 -08:00
Guo Lei
83e29faa55
use time.NewTicker instead of time.Tick. ( #4119 )
2023-01-09 06:47:08 -08:00
monchickey
3e2c9ea73d
Add image cropping. ( #4117 )
2023-01-06 09:28:07 -08:00
chrislu
7bdae5172e
batch delete EC needles
...
fix https://github.com/seaweedfs/seaweedfs/issues/4107
2023-01-03 22:05:26 -08:00
Chris Lu
d4566d4aaa
more solid weed mount ( #4089 )
...
* compare chunks by timestamp
* fix slab clearing error
* fix test compilation
* move oldest chunk to sealed, instead of by fullness
* lock on fh.entryViewCache
* remove verbose logs
* revert slat clearing
* less logs
* less logs
* track write and read by timestamp
* remove useless logic
* add entry lock on file handle release
* use mem chunk only, swap file chunk has problems
* comment out code that maybe used later
* add debug mode to compare data read and write
* more efficient readResolvedChunks with linked list
* small optimization
* fix test compilation
* minor fix on writer
* add SeparateGarbageChunks
* group chunks into sections
* turn off debug mode
* fix tests
* fix tests
* tmp enable swap file chunk
* Revert "tmp enable swap file chunk"
This reverts commit 985137ec47
2023-01-02 23:20:45 -08:00
Guo Lei
d8cfa1552b
support enable/disable vacuum ( #4087 )
...
* stop vacuum
* suspend/resume vacuum
* remove unused code
* rename
* rename param
2022-12-28 01:36:44 -08:00
lfhy
038bab3726
add -filer.path to webdav command ( #4061 )
2022-12-17 20:07:02 -08:00
chrislu
fc6b9e6e0c
volume server logs add url info
...
debug https://github.com/seaweedfs/seaweedfs/issues/3964
2022-12-11 13:15:38 -08:00
chrislu
38c63c429b
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-10 21:49:09 -08:00
chrislu
e1ca6308cb
add chunk etag when downloading from remote storage
...
fix https://github.com/seaweedfs/seaweedfs/issues/3987
2022-12-10 21:49:07 -08:00
wusong
549354e324
Fix hardlink counting ( #4042 )
...
Signed-off-by: wusong <wangwusong@virtaitech.com>
Signed-off-by: wusong <wangwusong@virtaitech.com>
Co-authored-by: wusong <wangwusong@virtaitech.com>
2022-12-08 10:50:57 -08:00
Jiffs Maverick
5f59d43c88
[filer] Fix uri escape during renaming ( #4039 )
2022-12-07 07:47:40 -08:00
famosss
15284e2bee
fix: Lost Content-Encoding metadata ( #4010 )
2022-11-23 22:52:34 -08:00
chrislu
70a4c98b00
refactor filer_pb.Entry and filer.Entry to use GetChunks()
...
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
Guo Lei
5b905fb2b7
Lazy loading ( #3958 )
...
* types packages is imported more than onece
* lazy-loading
* fix bugs
* fix bugs
* fix unit tests
* fix test error
* rename function
* unload ldb after initial startup
* Don't load ldb when starting volume server if ldbtimeout is set.
* remove uncessary unloadldb
* Update weed/command/server.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* Update weed/command/volume.go
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: guol-fnst <goul-fnst@fujitsu.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-11-14 00:19:27 -08:00
chrislu
aae82dca6f
file name escape uri
...
fix https://github.com/seaweedfs/seaweedfs/issues/3969
2022-11-13 23:01:56 -08:00
chrislu
ea2637734a
refactor filer proto chunk variable from mtime to modified_ts_ns
2022-10-28 12:53:19 -07:00
Eric Yang
51d462f204
ADHOC: volume fsck using append at ns ( #3906 )
...
* ADHOC: volume fsck using append at ns
* nit
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-10-24 22:09:38 -07:00
liubaojiang
25471d579a
add md5 header when UploadData to replication in ReplicatedWrite ( #3881 )
2022-10-19 01:34:14 -07:00
chrislu
de286fe662
shell: volume.move handles volume moved to cloud tier
...
fix https://github.com/seaweedfs/seaweedfs/issues/3803
2022-10-16 17:52:22 -07:00
Konstantin Lebedev
2f72103c83
avoid load volume file with BytesOffset mismatch ( #3841 )
...
* avoid load volume file with BytesOffset mismatch
https://github.com/seaweedfs/seaweedfs/issues/2966
* set BytesOffset if has not VolumeInfoFile
* typos fail => failed
* exit if bytesOffset mismatch
2022-10-14 00:18:09 -07:00
Konstantin Lebedev
1f7e52c63e
vacuum metrics and force sync dst files ( #3832 )
2022-10-13 00:51:20 -07:00
zemul
a05725aea6
filer: get directory metadata ( #3833 )
2022-10-12 07:14:49 -07:00
LHHDZ
d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… ( #3824 )
...
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-11 20:14:14 -07:00
