Commit Graph

48 Commits

Author SHA1 Message Date
Konstantin Lebedev
f53cff045f checks disk file exist 2022-03-10 18:58:56 +05:00
Chris Lu
0c8dea9de8 go fmt 2021-11-02 23:39:16 -07:00
Chris Lu
c857cc7286 cloud tier: remove tagging since not all s3 vendors support this 2021-10-29 12:39:19 -07:00
Chris Lu
d04cdcf40d s3 header add user agent 2021-10-29 12:28:24 -07:00
Chris Lu
f58ea6a2ee add source name to error message 2021-10-01 02:19:30 -07:00
Chris Lu
7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu
c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu
00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu
f0cc130849 do not force path style for better compatibility 2021-08-23 03:09:41 -07:00
Chris Lu
400de380f4 volume server: support tcp direct put/get/delete 2021-03-05 02:29:38 -08:00
Chris Lu
d680676d45 skip already loaded backends 2021-03-01 00:47:03 -08:00
Chris Lu
2cca07b44c cloud tier to non-AWS s3 gateways 2021-02-28 18:59:09 -08:00
Chris Lu
7635f6b9fa disk file avoid file.Stat() 2021-02-20 20:06:06 -08:00
Chris Lu
a2383b3b12 Revert "Revert "avoid file.Stat()""
This reverts commit 7ef4c24f18.
2021-02-20 12:42:09 -08:00
Chris Lu
7ef4c24f18 Revert "avoid file.Stat()"
This reverts commit 98c93ca465.
2021-02-20 12:39:33 -08:00
Chris Lu
98c93ca465 avoid file.Stat()
avoid one Syscall, but did not help on performance though
2021-02-20 09:44:17 -08:00
Chris Lu
a1210d1e8d Revert "volume: avoid file.stat(), file.seek() if possible during writes"
This reverts commit c78409a598.
2021-02-19 03:56:27 -08:00
Chris Lu
c78409a598 volume: avoid file.stat(), file.seek() if possible during writes 2021-02-18 23:22:54 -08:00
Chris Lu
cfb9342a15 avoid concurrent map updates to viper 2021-01-12 02:28:13 -08:00
Chris Lu
d522df0ecd adjust log level 2020-10-04 01:35:32 -07:00
Chris Lu
627b081b67 adjust logs 2020-08-16 16:32:22 -07:00
Chris Lu
7764e0465c refactoring 2020-04-12 21:00:55 -07:00
Chris Lu
c8ca234773 refactoring 2020-04-11 14:27:25 -07:00
Chris Lu
81797a059a volume: sync volume file right before compaction
fix https://github.com/chrislusf/seaweedfs/issues/1237
2020-03-19 23:54:52 -07:00
Chris Lu
709f231e23 tiered storage: add s3 endpoint for private s3 implementation
fix https://github.com/chrislusf/seaweedfs/issues/1238
2020-03-19 21:13:56 -07:00
Chris Lu
d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu
2000284435 rename volume tier info to volume info 2019-12-28 11:21:49 -08:00
Chris Lu
48d28d3eb2 tier: support remote file attributes and remember the file extension 2019-12-25 21:37:24 -08:00
Chris Lu
eecad6062a go fmt 2019-12-25 09:56:42 -08:00
Chris Lu
d960b3474a tier storage: support downloading the remote dat files 2019-12-25 09:53:13 -08:00
Chris Lu
356bd1b629 adjust text 2019-12-10 22:09:29 -08:00
Chris Lu
10bd3c6b4b refactoring 2019-12-08 19:44:16 -08:00
Chris Lu
2b8e20f122 remove unused functions 2019-12-08 18:33:25 -08:00
Chris Lu
70648d35ad go fmt 2019-12-02 20:49:58 -08:00
Chris Lu
caae543a9f fix test 2019-12-02 15:54:24 -08:00
Chris Lu
ec8de250e2 tiered storage: can copy to s3, read from s3
master not aware tiered volume yet, file assigning is not working yet
2019-12-02 15:08:31 -08:00
Chris Lu
0da7b894cc pass backend config from master to volume servers 2019-11-29 01:05:09 -08:00
Chris Lu
61bc1d6ffd tweaking 2019-11-28 18:47:51 -08:00
Chris Lu
f60154f330 master load backend storage config from master.toml 2019-11-28 18:33:18 -08:00
Chris Lu
0f9ba84274 s3 2019-11-27 03:09:45 -08:00
Chris Lu
e4c63ff2f8 go fmt 2019-11-22 14:51:09 -08:00
Chris Lu
9711a6ffaa WIP 2019-11-18 19:24:37 -08:00
Chris Lu
c6e8225a20 adjust memory mapped file size
related to https://github.com/chrislusf/seaweedfs/pull/1087
2019-11-12 21:05:48 -08:00
Lei Liu
46755ea1e1 fix master maintenance logic
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-11-12 14:46:10 +08:00
Chris Lu
85f8649320 refactor memory mapped file into backend storage 2019-11-09 00:10:59 -08:00
Chris Lu
4b5ba4927f adjust NewDiskFile() paramaters 2019-10-29 22:37:36 -07:00
Chris Lu
19b6a16003 changed from os.file to backend.DataStorageBackend 2019-10-29 00:35:16 -07:00
Chris Lu
46ed2ca902 refactoring 2019-10-24 23:41:32 -07:00