mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-21 06:57:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,152 Commits 32 Branches 296 Tags 152 MiB
167b50be88
Commit Graph

345 Commits

Author SHA1 Message Date
chrislu
ccf1795e6f wait a bit before getting the next volume id if the leader is recently elected 2024-11-23 19:58:45 -08:00
Konstantin Lebedev
a49d9e020c
[master] avoid crowded more writable for auto grow (#6214) 
avoid crowded more writable
https://github.com/seaweedfs/seaweedfs/issues/6121
 2024-11-21 00:57:42 -08:00
chrislu
6e388e29c9 correcting free volume count, factor it during ec encoding to ensure enough disk space available 
fix https://github.com/seaweedfs/seaweedfs/issues/6163
 2024-10-24 22:42:38 -07:00
chrislu
ae5bd0667a rename proto field from DestroyTime to expire_at_sec 
For TTL volume converted into EC volume, this change may leave the volumes staying.
 2024-10-24 21:35:11 -07:00
steve.wei
cfbe45c765
feat: add in-flight metric for s3/file/volume-server (#6120) 2024-10-14 12:10:05 -07:00
chrislu
35fd1e1c9a optimize memory usage for large number of volumes 
1. unwrap the map to avoid extra map object creation
2. fix ec shard counting in UpdateEcShards
 2024-10-10 10:00:30 -07:00
dsd
3b840c20e3
change math/rand => math/rand/v2 in volume_layout.go where is a perfo… (#6006) 2024-09-11 07:39:40 -07:00
Konstantin Lebedev
34bbaa2cdd
[master] process grow request with must grow (#5999) 
process grow request with must grow
 2024-09-09 23:45:02 -07:00
chrislu
ff3d46637d better logging for volume growth 2024-09-07 12:38:34 -07:00
chrislu
accba3070a refactor 2024-09-07 11:54:22 -07:00
Konstantin Lebedev
67a252ee8a
[master] refactor func ShouldGrowVolumes (#5884) 2024-09-04 08:16:44 -07:00
chrislu
3c0854e986 unnecessary skipping 2024-08-30 16:38:01 -07:00
chrislu
654b8210f7 parameter name 2024-08-30 14:52:07 -07:00
chrislu
8679870008 fix typo 2024-08-30 14:51:55 -07:00
chrislu
a4b25a642d math/rand => math/rand/v2 2024-08-29 09:52:21 -07:00
chrislu
ded5e084ea ensure none zero lastGrowCount 2024-08-27 09:03:11 -07:00
chrislu
4463296811 add parallel vacuuming 2024-08-21 22:53:54 -07:00
chrislu
b3696024d1 add warning for not enough copies when skipping vacuuming volumes 
fix https://github.com/seaweedfs/seaweedfs/issues/5906
 2024-08-20 09:39:35 -07:00
Riccardo Bertossa
6fe8639504
add http endpoint to get the size of a collection (#5910) 2024-08-19 07:44:45 -07:00
augustazz
0b00706454
EC volume supports expiration and displays expiration message when executing volume.list (#5895) 
* ec volume expire

* volume.list show DestroyTime

* comments

* code optimization

---------

Co-authored-by: xuwenfeng <xuwenfeng1@zto.com>
 2024-08-16 00:20:00 -07:00
wusong
6f58ab7e8b
[master] fix master panic (#5893) 2024-08-15 06:20:51 -07:00
Konstantin Lebedev
b2ffcdaab2
[master] do sync grow request only if absolutely necessary (#5821) 
* do sync grow request only if absolutely necessary
https://github.com/seaweedfs/seaweedfs/pull/5819

* remove check VolumeGrowStrategy Threshold on PickForWrite

* fix fmt.Errorf
 2024-07-30 13:21:35 -07:00
wyang
4b1f539ab8
fix allocate reduplicated volumeId to different volume (#5811) 
* fix allocate reduplicated volumeId to different volume

* only check barrier when read

---------

Co-authored-by: Yang Wang <yangwang@weride.ai>
 2024-07-26 21:48:36 -07:00
chrislu
9265be43c0 avoid nil 2024-07-21 21:01:29 -07:00
vadimartynov
86d92a42b4
Added tls for http clients (#5766) 
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
 2024-07-16 23:14:09 -07:00
Konstantin Lebedev
67edf1d014
[master] Do Automatic Volume Grow in background (#5781) 
* Do Automatic Volume Grow in backgound

* pass lastGrowCount to master

* fix build

* fix count to uint64
 2024-07-16 08:03:40 -07:00
Konstantin Lebedev
a53e406c99
[master] refactor HasGrowRequest to atomic bool (#5782) 
refactor HasGrowRequest to atomit bool
 2024-07-15 10:51:21 -07:00
Konstantin Lebedev
33964fa292
metrics stats of volume layout depends on the data center (#5775) 
stats volume layout depends on the data center
 2024-07-12 12:32:25 -07:00
Konstantin Lebedev
04f4b10884
fix: avoid timeout if datacenter does not exist in topology (#5772) 
* fix: avoid timeout if datacenter does not exist in topology

* fix: error msg

* fix: rm dublicate check

* fix: compare

* revert minor change
 2024-07-12 11:19:08 -07:00
小羽
e8537d7172
Different disk labels should not use the same DiskUsages instance while master received volume heatbeat (#5770) 2024-07-12 08:09:51 -07:00
Numblgw
73baf82f05
bugfix: unregister ec shards when volume server disconnected (#5697) 
bugfix unregister ec shards when volume server disconnected

Co-authored-by: liguowei <liguowei@xinye.com>
 2024-06-20 15:29:36 -07:00
chrislu
3e7a92061b pass along volume server grpc port 
fix https://github.com/seaweedfs/seaweedfs/issues/5617
 2024-05-29 10:41:33 -07:00
chrislu
d218fe54fa go fmt 2024-05-20 11:03:56 -07:00
chrislu
55976ae04a avoid repeated calls to heavy-weighted viper 2024-04-18 09:09:45 -07:00
chrislu
31f1f96038 improve perf a bit 2024-04-18 08:47:55 -07:00
Konstantin Lebedev
5189a09de0
[volume] Reduce the number of buffers for uploading one chunk (#5458) 2024-04-11 04:47:21 -07:00
Konstantin Lebedev
d5d8b8e2ae
fix panic at isAllWritable (#5457) 
fix panic
https://github.com/seaweedfs/seaweedfs/issues/5456
 2024-04-02 09:06:19 -07:00
Konstantin Lebedev
9c1e0f5811
[master] grow volumes if no writable volumes in current dataCenter (#5434) 
* grow volumes if no writable volumes in current dataCenter
https://github.com/seaweedfs/seaweedfs/issues/3886

* fix tests with volume grow

* automatic volume grow one volume

* add ErrorChunkAssign metrics
 2024-03-29 00:38:27 -07:00
Konstantin Lebedev
df40908e57
fix panic 5435 (#5436) 2024-03-28 16:17:59 -07:00
Konstantin Lebedev
dc9568fc0d
[master] add test for PickForWrite add metrics for volume layout (#5413) 2024-03-22 07:39:11 -07:00
Konstantin Lebedev
0f8e76bbd6
fix: clean metric MasterReplicaPlacementMismatch for unregister volume (#5239) 2024-01-25 00:23:24 -08:00
chrislu
025f8eb37b return dir assign counter 
fix https://github.com/seaweedfs/seaweedfs/issues/5231
 2024-01-23 19:48:42 -08:00
chrislu
49fcb48e04 same logic as grpc assign 
fix https://github.com/seaweedfs/seaweedfs/issues/5213
 2024-01-23 19:39:59 -08:00
Konstantin Lebedev
a7fc723ae0
chore: add status code for request_total metrics (#5188) 2024-01-10 10:05:27 -08:00
chrislu
bebbc9fe44 create volume grow request if the selected volume is close to full 2023-12-27 11:45:44 -08:00
chrislu
c6b1dc7058 remove unused code 2023-12-24 11:11:41 -08:00
chrislu
b62066d38f adding DiskType to /dir/status output 2023-10-12 21:39:11 -07:00
chrislu
645ae8c57b Revert "Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"" 
This reverts commit 8cb42c39
 2023-09-25 09:35:16 -07:00
chrislu
8cb42c39ad Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs" 
This reverts commit 2e5aa06026, reversing
changes made to 4d414f54a2.
 2023-09-18 16:12:50 -07:00
dependabot[bot]
a04bd4d26f
Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850) 
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0

Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0.
- [Release notes](https://github.com/rclone/rclone/releases)
- [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md)
- [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0)

---
updated-dependencies:
- dependency-name: github.com/rclone/rclone
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* API changes

* go mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
 2023-09-18 14:43:05 -07:00