Konstantin Lebedev
25643cfbc6
fix: panic: assignment to entry in nil map on S3Sink.CreateEntry ( #5406 )
2024-03-21 08:12:31 -07:00
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink ( #5352 )
...
* fix: install cronie
* chore: refactor configure S3Sink
* chore: refactor cinfig
* add filer-backup compose file
* fix: X-Amz-Meta-Mtime and resolve with comments
* fix: attr mtime
* fix: MaxUploadPartst is reduced to the maximum allowable
* fix: env and force set max MaxUploadParts
* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
2024-03-07 08:35:51 -08:00
chrislu
cc0c8c5f81
simplify
2022-08-27 00:21:57 -07:00
chrislu
87b70a6809
clean up
2022-08-27 00:09:04 -07:00
chrislu
c839ce1b19
s3 sink use s3 upload manager
...
fix https://github.com/seaweedfs/seaweedfs/issues/3531
2022-08-26 23:47:12 -07:00
chrislu
e3f40d538d
cleaner code
2022-08-20 17:51:30 -07:00
chrislu
11f99836c3
filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0)
...
fix https://github.com/seaweedfs/seaweedfs/issues/3468
2022-08-19 23:00:56 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
elee
954ad98e0d
set canned acl on replication create
2022-02-27 04:49:31 -06:00
Chris Lu
7ce97b59d8
go fmt
2021-09-01 02:45:42 -07:00
Chris Lu
c08ac536ed
cloud drive: add support for Wasabi
...
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu
00c4e06caa
cloud drive: s3 configurable force path style
2021-08-23 03:30:41 -07:00
Chris Lu
f0cc130849
do not force path style for better compatibility
2021-08-23 03:09:41 -07:00
Chris Lu
45bffc92a8
filer.backup: fix cloud sinks when updating entry
2021-05-26 14:53:11 -07:00
Chris Lu
984fdd6192
always use non bucket prefixing url
2021-02-28 16:20:13 -08:00
Chris Lu
678c54d705
data sink: add incremental mode
2021-02-28 16:19:03 -08:00
Chris Lu
387ab6796f
filer: cross cluster synchronization
2020-09-09 11:21:23 -07:00
Chris Lu
eb7929a971
rename filer2 to filer
2020-09-01 00:21:19 -07:00
Chris Lu
c647deace1
file size support set file length
...
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu
97d97f3528
go code can read and write chunk manifest
2020-07-19 17:59:43 -07:00
Chris Lu
4c498e73de
filer replication: add s3 endpoint
2020-04-07 17:49:00 -07:00
Chris Lu
ae2ee379c0
consistent 64bit size
2020-03-22 01:37:46 -07:00
HongyanShen
81610ed006
fix : #1226
2020-03-11 14:37:14 +08:00
Chris Lu
97ab8a1976
remove ctx if possible
2020-02-25 22:23:59 -08:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
d335f04de6
support env variables to overwrite toml file
2020-01-29 09:09:55 -08:00
Chris Lu
9711a6ffaa
WIP
2019-11-18 19:24:37 -08:00
Chris Lu
b3b42bc947
replicate need to include new entry path
2019-04-16 00:44:31 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
Chris Lu
8dfac6a4cf
working b2 sink
2018-11-04 11:58:59 -08:00
Chris Lu
3674ad9f8e
go fmt
2018-11-01 01:12:21 -07:00
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
2018-10-31 01:11:19 -07:00
Chris Lu
08266b7256
go fmt
2018-10-11 00:08:13 -07:00
Chris Lu
a5b5887efc
add filer replication to aws s3
2018-10-06 17:10:15 -07:00
Chris Lu
f8c2704d2b
reformat
2018-10-06 13:04:33 -07:00
Chris Lu
56c5c7b1b6
add google cloud storage
2018-10-04 01:14:44 -07:00
Chris Lu
e8ef501f02
add s3 replication sink
2018-10-03 23:36:52 -07:00