Commit Graph

72 Commits

Author SHA1 Message Date
chrislu
00f87e5bb5 remove unused 2024-06-28 14:54:39 -07:00
mervynzhang
df400e6c71
Concurrency works better (#4663)
Co-authored-by: mervyn.zhang <mervyn.zhang@sap.com>
2023-07-12 23:04:54 -07:00
mervynzhang
1ebb549f77
support swift (#4480) 2023-05-19 06:39:25 -07:00
Muhammad Hallaj bin Subery
9bd422d2c9
adding support for B2 region (#4177)
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
aronneagu
180853a2c9
Replace dashes with underscores in x-amz-meta headers (#3965) 2022-11-10 07:09:53 -08:00
Konstantin Lebedev
5431c445cd
fix filer.remote.sync to azure with ContentType (#3949)
* fix filer.remote.sync to azure with ContentType

* fix pass X-Amz-Meta to X-Ms-Meta
2022-11-04 09:10:33 -07:00
chrislu
4193dafce1 azure metadata: skip metadata prefixed with "X-"
fix https://github.com/seaweedfs/seaweedfs/issues/3875
2022-11-02 21:42:02 -07:00
chrislu
9920d65bc0 gateway to remote object store: adjust upload concurrency 2022-08-26 23:47:37 -07:00
chrislu
eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
1d0c53ea56 remote storage: stop supporting hdfs as a remote storage 2022-06-20 14:15:59 -07:00
chrislu
61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
chrislu
bff1ccc1de fix compilation 2022-05-11 00:52:15 -07:00
chrislu
ad01c63b84 conditionally skip hdfs related code 2022-04-21 01:43:01 -07:00
justin
d02f13c2d1 remove Redundant type conversion and use strings.TrimSuffix to enhance readability 2022-04-06 14:58:09 +08:00
chrislu
a0bad1c997 remove any go mod changes
This reverts commit 6c7f7d6887.
2022-03-21 23:04:00 -07:00
chrislu
6c7f7d6887 Revert "Merge pull request #2782 from SadmiB/upstream"
This reverts commit a644b7236a, reversing
changes made to 349257f822.
2022-03-21 23:00:50 -07:00
SadmiB
d12540c9f2 Add contabo api client 2022-03-21 17:16:49 +01:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
04663c3611 remote.mount: print out metadata sync errors 2021-11-06 11:29:50 -07:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
53b9b521c9 adjust error message 2021-09-04 13:46:06 -07:00
Chris Lu
da49d25950 auto list of storage types 2021-09-04 00:18:21 -07:00
Chris Lu
bbc77f7af4 fix compilation 2021-09-03 22:56:59 -07:00
Chris Lu
0652805236 cloud drive: add createBucket() deleteBucket() 2021-09-03 22:30:55 -07:00
Chris Lu
83cd0fc739 cloud drive: add list buckets 2021-09-03 20:42:02 -07:00
Chris Lu
fbfc90fd1e adjust formatting remote location 2021-09-03 18:52:37 -07:00
Chris Lu
bca4a9de78 simplify 2021-09-02 23:09:24 -07:00
Chris Lu
958125bd02 conforming to http user agent common practice 2021-09-02 22:55:35 -07:00
Chris Lu
7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu
3bd48c4f29 filer.remote.sync: exit when directory is unmounted
this will not propagate the deletions back to the cloud
2021-09-01 01:29:22 -07:00
Chris Lu
3faaa6e360 ensure cached client with updated storage conf 2021-09-01 01:27:45 -07:00
Chris Lu
1dfcdc53b7 cloud drive: add storj, filebase 2021-08-30 17:43:14 -07:00
Chris Lu
701cc133f3 cloud drive: s3 add option for v4 signature 2021-08-30 17:28:33 -07:00
Chris Lu
eacaa44dc2 refactor 2021-08-29 19:13:48 -07:00
Chris Lu
a31f2907f0 cloud drive: filer.remote.sync supports remove folder 2021-08-29 18:46:28 -07:00
Chris Lu
001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu
05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu
c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu
a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu
c6c97bd83d add default env variables 2021-08-24 23:25:36 -07:00
Chris Lu
a19c728034 cloud drive: add support for Tencent COS 2021-08-24 23:19:45 -07:00
Chris Lu
19a81d25af cloud drive: add support for Aliyun OSS 2021-08-24 23:14:24 -07:00
Chris Lu
47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00
Chris Lu
e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu
a78d0227cd adjust package name 2021-08-23 23:19:31 -07:00
Chris Lu
12631a3f5b cloud drive: gcs simplify a little bit 2021-08-23 14:43:01 -07:00