Commit Graph

181 Commits

Author SHA1 Message Date
Chris Lu
3be061994f skip connection reset error
fix https://github.com/chrislusf/seaweedfs/issues/1971

this is because the connections are pooled but the volume server has reset the connection
2021-04-07 00:54:13 -07:00
Chris Lu
319743d330 go fmt 2021-04-02 02:22:59 -07:00
Chris Lu
7e8edc3c4a refactoring 2021-04-02 01:10:24 -07:00
Chris Lu
cefe66f159 dedicated upload processor
avoid thundering effect of overloading volume servers
2021-04-01 02:21:40 -07:00
Chris Lu
1f984d2645 refactor buffer pool 2021-04-01 02:20:00 -07:00
Chris Lu
3b2a9c98ef fix nil 2021-03-07 11:45:35 -08:00
Chris Lu
df72dc206d stats collect chunk upload retry count 2021-03-07 11:26:15 -08:00
Chris Lu
44bdfb2d15 filer: avoid encryption and compression at the same time
fix https://github.com/chrislusf/seaweedfs/issues/1828
2021-02-22 12:22:49 -08:00
bingoohuang
352ac2f271 Merge remote-tracking branch 'origin/master' 2021-02-18 14:05:51 +08:00
bingoohuang
eab6e31d34 use backticks instead of double quotes to avoid escaped additionally in regex 2021-02-18 14:05:28 +08:00
Chris Lu
3575d41009 go fmt 2021-02-17 20:57:08 -08:00
Chris Lu
6daa932f5c refactoring to get master function, instead of passing master values directly
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
dd9f3a0104 add sleep between upload retries 2021-02-17 20:55:11 -08:00
Chris Lu
487e435679 adjust http max idle connections per host
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
Chris Lu
821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu
00707ec00f mount: outsideContainerClusterMode proxy through filer
Running mount outside of the cluster would not need to expose all the volume servers to outside of the cluster. The chunk read and write will go through the filer.
2021-01-24 19:01:58 -08:00
Chris Lu
4b5adc60b4 upload result changes to the right output variable 2021-01-03 01:44:22 -08:00
Chris Lu
3433accb1b weed upload: add TTL 2020-12-30 15:46:07 -08:00
Chris Lu
1bf22c0b5b go fmt 2020-12-16 09:14:05 -08:00
Chris Lu
0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu
e9cd798bd3 adding volume type 2020-12-13 00:58:58 -08:00
Chris Lu
0da7ecfd29 go fmt 2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc remove zstd
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Chris Lu
dc304342b2 fs.configure: configurable volume growth 2020-11-17 01:00:02 -08:00
Chris Lu
b04375a3d1 use alternative assign request 2020-11-17 00:36:21 -08:00
Chris Lu
0ea5c087ce go fmt 2020-11-15 16:59:28 -08:00
Chris Lu
95c0de285d refactoring 2020-11-15 16:58:48 -08:00
Chris Lu
a29abd8600 add a {X-,}Idempotency-Key header for http POST
fix https://github.com/chrislusf/seaweedfs/issues/1592

according to https://github.com/golang/go/issues/19943
2020-11-11 12:56:53 -08:00
Chris Lu
8708e7a64d print out data size 2020-11-11 12:51:44 -08:00
Chris Lu
0ecff9521d avoid []byte reuse 2020-11-02 23:27:54 -08:00
Chris Lu
031aa70da6 Revert "revert bytebufferpool, seems problematic"
This reverts commit e0fcab47c0.
2020-11-02 19:11:25 -08:00
Chris Lu
e0fcab47c0 revert bytebufferpool, seems problematic 2020-11-02 13:49:38 -08:00
Chris Lu
c057dd1f64 add back upload retries 2020-10-30 02:16:34 -07:00
Chris Lu
19772d70d7 print for debugging 2020-10-24 00:12:02 -07:00
Chris Lu
ef9c32ea0d adjust compression threshold 2020-10-23 23:05:55 -07:00
Chris Lu
29c9fa2ef2 byte buffer for uploading 2020-10-23 11:29:51 -07:00
Chris Lu
2579edbc60 fix bug found by tests 2020-10-23 10:05:40 -07:00
Chris Lu
e0c8507d93 memory pool to reduce memory allocation 2020-10-23 00:41:40 -07:00
Chris Lu
c31b254248 mount: shortcut when there is only one chunk 2020-10-20 23:48:29 -07:00
Chris Lu
f64252023e Revert "a better byte buffer pool"
This reverts commit 59c6422777.
2020-10-20 23:21:54 -07:00
Chris Lu
59c6422777 a better byte buffer pool 2020-10-20 22:53:58 -07:00
Chris Lu
35f6518c36 weed upload: usePublicUrl was not being used on big files
fix https://github.com/chrislusf/seaweedfs/issues/1492
2020-09-25 02:41:38 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu
50f7e2e89f add log messages 2020-08-26 09:51:20 -07:00
Chris Lu
5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu
0b301b0b47 consume all response body in order to release requests 2020-08-23 21:32:29 -07:00
Chris Lu
003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu
83cad3da79 add retry file upload 2020-08-11 20:30:11 -07:00
Chris Lu
20e2ac1add filer: store md5 metadata for files uploaded by filer
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00