chrislu
|
41dfe27102
|
Revert "remove max connection age"
This reverts commit b9b684194f.
|
2022-06-20 14:49:39 -07:00 |
|
chrislu
|
1d0c53ea56
|
remote storage: stop supporting hdfs as a remote storage
|
2022-06-20 14:15:59 -07:00 |
|
石昌林
|
3dd60529c5
|
some code optimizations
|
2022-06-17 19:07:39 +08:00 |
|
石昌林
|
78b3728169
|
add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply
# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply
# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply
# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply
# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
|
2022-06-15 21:07:55 +08:00 |
|
chrislu
|
d12f431d98
|
collect volume server status
|
2022-06-12 11:56:23 -07:00 |
|
Chris Lu
|
f43ec9f363
|
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud
|
2022-06-07 08:56:55 -07:00 |
|
Konstantin Lebedev
|
c07820178f
|
fix s3 tests
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
|
2022-06-07 14:43:10 +05:00 |
|
chrislu
|
4fd5f96598
|
filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
|
2022-06-06 00:39:35 -07:00 |
|
creeew
|
02ae102731
|
fix filer.sync missing source srv uploaded files to target when target down
|
2022-06-02 01:28:47 +08:00 |
|
chrislu
|
682382648e
|
collect cluster node start time
|
2022-05-30 16:23:52 -07:00 |
|
chrislu
|
a2b101a737
|
subscribe metadata between a range
|
2022-05-30 15:04:19 -07:00 |
|
chrislu
|
ca16fbf0ef
|
shell: cluster.ps display filer group
|
2022-05-29 16:37:14 -07:00 |
|
Chris Lu
|
8efe55f981
|
Merge pull request #3059 from guol-fnst/avoid_dup_vol
avoid loading duplicated volume directory
|
2022-05-17 23:48:57 -07:00 |
|
guol-fnst
|
8e2426a879
|
revert changes of pb file
orignal protoc-gen-go-grpc verion is 1.1.0
|
2022-05-18 11:52:14 +08:00 |
|
guol-fnst
|
b12944f9c6
|
fix naming convention
notify volume server of duplicate directoris
improve searching efficiency
|
2022-05-17 15:41:49 +08:00 |
|
guol-fnst
|
de6aa9cce8
|
avoid duplicated volume directory
|
2022-05-16 19:33:51 +08:00 |
|
chrislu
|
866fed1eab
|
avoid grpc name conflict
fix https://github.com/chrislusf/seaweedfs/issues/3055
|
2022-05-15 21:31:29 -07:00 |
|
chrislu
|
29198720f2
|
s3: add grpc server to accept configuration changes
|
2022-05-15 00:43:37 -07:00 |
|
chrislu
|
8bc16f1149
|
rename
|
2022-05-06 23:38:22 -07:00 |
|
chrislu
|
94635e9b5c
|
filer: add filer group
|
2022-05-01 21:59:16 -07:00 |
|
chrislu
|
4bd6bea429
|
fix test
|
2022-05-01 09:02:01 -07:00 |
|
Konstantin Lebedev
|
1e35b4929f
|
shell vacuum volume by collection and volume id
|
2022-04-18 18:40:58 +05:00 |
|
chrislu
|
b4be56bb3b
|
add timing info during ping operation
|
2022-04-16 12:45:49 -07:00 |
|
Konstantin Lebedev
|
35bc67f030
|
fix conflicts
|
2022-04-07 18:52:13 +05:00 |
|
Konstantin Lebedev
|
f5246b748d
|
Merge branch 'new_master' into hashicorp_raft
# Conflicts:
# weed/pb/master_pb/master.pb.go
|
2022-04-07 18:50:27 +05:00 |
|
Konstantin Lebedev
|
85d80fd36d
|
fix removing old raft server
|
2022-04-07 15:31:37 +05:00 |
|
Konstantin Lebedev
|
357aa818fe
|
add raft shell cmds
|
2022-04-06 15:23:53 +05:00 |
|
chrislu
|
bc888226fc
|
erasure coding: tracking encoded/decoded volumes
If an EC shard is created but not spread to other servers, the masterclient would think this shard is not located here.
|
2022-04-05 19:03:02 -07:00 |
|
chrislu
|
958f880b70
|
mount: add grpc method to adjust quota
|
2022-04-02 15:14:37 -07:00 |
|
chrislu
|
daca2d22a5
|
use original server address string as map key
|
2022-04-01 17:34:42 -07:00 |
|
chrislu
|
bbbbbd70a4
|
master supports grpc ping
|
2022-04-01 16:50:58 -07:00 |
|
chrislu
|
743ad690b6
|
filer supports grpc ping
|
2022-04-01 16:44:58 -07:00 |
|
chrislu
|
800cbc004c
|
volume server adds ping function
|
2022-04-01 16:37:06 -07:00 |
|
chrislu
|
21e0898631
|
refactor: change masters from a slice to a map
|
2022-03-26 13:33:17 -07:00 |
|
chrislu
|
a0bad1c997
|
remove any go mod changes
This reverts commit 6c7f7d6887.
|
2022-03-21 23:04:00 -07:00 |
|
chrislu
|
6c7f7d6887
|
Revert "Merge pull request #2782 from SadmiB/upstream"
This reverts commit a644b7236a, reversing
changes made to 349257f822.
|
2022-03-21 23:00:50 -07:00 |
|
Chris Lu
|
0d692a6254
|
Merge branch 'master' into upstream
|
2022-03-21 11:02:04 -07:00 |
|
SadmiB
|
d12540c9f2
|
Add contabo api client
|
2022-03-21 17:16:49 +01:00 |
|
chrislu
|
4042fdf3bb
|
rename to skipCheckParentDir
related to https://github.com/chrislusf/seaweedfs/pull/2761
It's better to default to false.
|
2022-03-16 23:55:31 -07:00 |
|
chrislu
|
202a29d014
|
refactoring
|
2022-02-25 01:17:26 -08:00 |
|
chrislu
|
91d6785cf3
|
define metadata action types
|
2022-02-25 00:54:16 -08:00 |
|
chrislu
|
be3fc77391
|
mount2: use consistent inode
|
2022-02-25 00:53:27 -08:00 |
|
chrislu
|
ceaf993a27
|
mount2: add rdev
|
2022-02-24 14:51:25 -08:00 |
|
chrislu
|
b9b684194f
|
remove max connection age
following https://github.com/grpc/grpc-go/issues/3170#issuecomment-552517779
|
2022-01-27 02:28:22 -08:00 |
|
chrislu
|
b1063162b6
|
display bucket quota
|
2022-01-21 00:55:04 -08:00 |
|
chrislu
|
fec8428fd8
|
POSIX: different inode for same named different file types
|
2022-01-12 11:51:13 -08:00 |
|
chrislu
|
6a12520a96
|
fix logging
|
2022-01-10 01:00:11 -08:00 |
|
chrislu
|
5c87fcc6d2
|
add client id for all metadata listening clients
|
2021-12-30 00:23:57 -08:00 |
|
chrislu
|
9f9ef1340c
|
use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
|
2021-12-26 00:15:03 -08:00 |
|
banjiaojuhao
|
083bf3a137
|
filer server: add "datacenter, rack and datanode" for path specific configuration
|
2021-12-23 23:25:05 +08:00 |
|
banjiaojuhao
|
08336be92e
|
filer server: allow upload file to specific dataNode
|
2021-12-22 21:57:26 +08:00 |
|
chrislu
|
a2d3f89c7b
|
add lock messages
|
2021-12-10 13:24:38 -08:00 |
|
chrislu
|
b70cb3e0b2
|
upgrade protoc to 3.17.3
$brew install protobuf
$ protoc --version
libprotoc 3.17.3
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
|
2021-12-05 00:42:25 -08:00 |
|
Chris Lu
|
e0fc2898e9
|
auto updated filer peer list
|
2021-11-06 14:23:35 -07:00 |
|
Chris Lu
|
4b9c42996a
|
refactor grpc API
|
2021-11-05 18:11:40 -07:00 |
|
Chris Lu
|
5ea86ef1da
|
Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()"
This reverts commit af71ae11aa.
|
2021-11-05 17:52:15 -07:00 |
|
Chris Lu
|
af71ae11aa
|
master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()
|
2021-11-03 01:09:48 -07:00 |
|
Chris Lu
|
5160eb08f7
|
shell: optionally read filer address from master
|
2021-11-02 23:38:45 -07:00 |
|
Chris Lu
|
24858507cc
|
rename API to avoid confusion
|
2021-10-30 19:27:25 -07:00 |
|
Chris Lu
|
5435027ff0
|
volume copy: stream out copying progress and avoid grpc request timeout
fix https://github.com/chrislusf/seaweedfs/issues/2386
|
2021-10-24 02:52:56 -07:00 |
|
Chris Lu
|
3be3c17f59
|
volume vacuum: avoid timeout with streaming progress report
fix https://github.com/chrislusf/seaweedfs/issues/2396
|
2021-10-24 01:55:34 -07:00 |
|
Chris Lu
|
cca62fdb30
|
mount: streaming renaming folders
|
2021-10-17 04:22:42 -07:00 |
|
Chris Lu
|
004e56c1a6
|
save updated lastTsNs
|
2021-10-17 02:01:20 -07:00 |
|
Chris Lu
|
0a856241fe
|
avoid int bigger than math.MaxInt32
fix https://github.com/chrislusf/seaweedfs/issues/2363
|
2021-10-07 21:12:57 -07:00 |
|
Chris Lu
|
225b019fe0
|
stream read multiple volumes in a volume server
|
2021-09-27 02:51:31 -07:00 |
|
Chris Lu
|
c4d7ee6c5c
|
volume server: read all files in a volume
|
2021-09-27 01:45:32 -07:00 |
|
Chris Lu
|
52fe86df45
|
use default 10000 for grpc port
|
2021-09-20 14:05:59 -07:00 |
|
Chris Lu
|
2789d10342
|
go fmt
|
2021-09-14 10:37:06 -07:00 |
|
Chris Lu
|
e5fc35ed0c
|
change server address from string to a type
|
2021-09-12 22:47:52 -07:00 |
|
Chris Lu
|
0128239c0f
|
handle ipv6 addresses
|
2021-09-07 16:43:54 -07:00 |
|
Chris Lu
|
889a39a24f
|
ipv6
|
2021-09-07 02:30:25 -07:00 |
|
Chris Lu
|
2b1feb732c
|
remote.cache supports replication
|
2021-09-06 18:30:44 -07:00 |
|
Chris Lu
|
c218ef20c7
|
filer.remote.sync: automatically detect the primary remote storage
|
2021-09-06 15:10:55 -07:00 |
|
Chris Lu
|
0cda61d539
|
return error if not found
|
2021-09-05 23:43:44 -07:00 |
|
Chris Lu
|
e93d4935e3
|
add other replica locations when assigning volumes
|
2021-09-05 23:32:25 -07:00 |
|
Chris Lu
|
0652805236
|
cloud drive: add createBucket() deleteBucket()
|
2021-09-03 22:30:55 -07:00 |
|
Chris Lu
|
d1a4e19a3f
|
volume: copy file also copies modification time
to ensure ttl can work well
|
2021-09-01 02:42:57 -07:00 |
|
Chris Lu
|
8e125339d5
|
line wrap
|
2021-08-31 23:30:28 -07:00 |
|
Chris Lu
|
68ea99190f
|
add additionalPathPrefixes
|
2021-08-31 23:26:26 -07:00 |
|
Chris Lu
|
43fd11278e
|
support follow additional path prefixes
|
2021-08-31 23:23:08 -07:00 |
|
Chris Lu
|
1dfcdc53b7
|
cloud drive: add storj, filebase
|
2021-08-30 17:43:14 -07:00 |
|
Chris Lu
|
701cc133f3
|
cloud drive: s3 add option for v4 signature
|
2021-08-30 17:28:33 -07:00 |
|
Chris Lu
|
05f32376eb
|
add cluster id in filer configuration response
|
2021-08-29 21:02:10 -07:00 |
|
Chris Lu
|
001a472057
|
cloud mount: remote storage support hdfs
|
2021-08-29 18:41:29 -07:00 |
|
Chris Lu
|
a22f37b01c
|
remove tencent region
|
2021-08-28 02:23:03 -07:00 |
|
Chris Lu
|
05a648bb96
|
refactor: separating out remote.proto
|
2021-08-26 15:18:34 -07:00 |
|
Chris Lu
|
c08ac536ed
|
cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
|
2021-08-25 17:34:29 -07:00 |
|
Chris Lu
|
a7a914f120
|
cloud drive: add support for Baidu BOS
|
2021-08-24 23:46:33 -07:00 |
|
Chris Lu
|
a19c728034
|
cloud drive: add support for Tencent COS
|
2021-08-24 23:19:45 -07:00 |
|
Chris Lu
|
19a81d25af
|
cloud drive: add support for Aliyun OSS
|
2021-08-24 23:14:24 -07:00 |
|
Chris Lu
|
47d775cf68
|
cloud drive: add support for BackBlaze
|
2021-08-24 22:30:06 -07:00 |
|
Chris Lu
|
e9ebe24f2e
|
cloud drive: add support for Azure
|
2021-08-24 01:18:30 -07:00 |
|
Chris Lu
|
00c4e06caa
|
cloud drive: s3 configurable force path style
|
2021-08-23 03:30:41 -07:00 |
|
Chris Lu
|
2836a58d87
|
cloud drive: S3 supports storage class
|
2021-08-23 02:18:59 -07:00 |
|
Chris Lu
|
258063de26
|
cloud drive: add google cloud storage
|
2021-08-23 00:29:27 -07:00 |
|
Chris Lu
|
3ada61c875
|
rename
|
2021-08-14 21:50:35 -07:00 |
|
Chris Lu
|
c34747c79d
|
rename, fix wrong logic.
|
2021-08-14 21:46:34 -07:00 |
|
Chris Lu
|
08258a819d
|
fix mistake
|
2021-08-14 05:10:30 -07:00 |
|
Chris Lu
|
4909bd9684
|
gRpc connection error on filer when no volume left #2243
fix https://github.com/chrislusf/seaweedfs/issues/2243
grpc do not cache connections only when connection problem happens.
Normal error results should not close the shared grpc connection.
|
2021-08-14 05:06:44 -07:00 |
|
Chris Lu
|
5a0f92423e
|
use grpc and jwt
|
2021-08-12 21:40:33 -07:00 |
|