Commit Graph

79 Commits

Author SHA1 Message Date
Chris Lu
1c65656fb4 s3: add option to fsync buckets 2020-04-11 23:37:10 -07:00
Chris Lu
dc08e4098f add etag only for PUT or large chunked uploads 2020-04-08 09:13:26 -07:00
Chris Lu
ec2eb8bc48 add If-None-Match and If-Modified-Since
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu
c0f0fdb3ba refactoring 2020-03-23 00:01:34 -07:00
Chris Lu
8a899992f2 filer: fix ttl parsing
fix https://github.com/chrislusf/seaweedfs/issues/1225
2020-03-09 00:16:10 -07:00
Chris Lu
e04c1a1361 filer: remove chunk manifest file support
This is not needed for filer
2020-03-08 17:03:27 -07:00
Chris Lu
9b3109a5d8 filer: processing all response headers, no pass through to volume server
* filer calculate MD5 etag
* filer handle response headers, instread of pass it to volume servers
2020-03-08 15:42:44 -07:00
Chris Lu
afb20de14c breaks dependency loop 2020-03-07 17:01:39 -08:00
Chris Lu
0375ce2c2e filer: set mime type from volume server 2020-03-07 07:25:15 -08:00
Chris Lu
ea1169dc80 filer cipher: single chunk http POST and PUT and read 2020-03-07 06:08:08 -08:00
Chris Lu
6ab7368ef2 filer: dynamically create bucket under /buckets folder 2020-02-24 22:28:45 -08:00
Chris Lu
382ff218d3 filer: set file size, streaming chunk file uploading
fix https://github.com/chrislusf/seaweedfs/issues/1193
2020-02-03 17:04:06 -08:00
Chris Lu
d4e75a0d18 filer: option to create file only if it is new, O_EXCL 2020-01-22 11:42:40 -08:00
Chris Lu
86d4b18eb5 filer: optional recursive deletion
fix https://github.com/chrislusf/seaweedfs/issues/1176
2019-12-31 11:52:54 -08:00
Chris Lu
36ddca9d1f filer: return not found 404 status if not found
fix https://github.com/chrislusf/seaweedfs/issues/1160
2019-12-18 21:04:40 -08:00
Chris Lu
bbb6ebc3c0 filer: DeleteFolderChildren for deleting large folders 2019-12-13 00:23:05 -08:00
Chris Lu
1ddfbff2d3 deletion add option skipChunkDeletion 2019-12-11 14:58:22 -08:00
Chris Lu
f06b65a6fd filer: EXPERT add shouldDeleteChunks parameter for fast deletion 2019-12-11 07:50:53 -08:00
Yoni Nakache
9e45702334
Filer: Enable delete entries without flag volume
same as happens on the s3 API on delete bucket,
to reduce the IO requests on the system, you can just clear recursively folders without flagging them in volume server (they will be removed on delete bucket/drop collection)
this will remove the user ability to retrieve data under these folders
2019-12-11 11:30:19 +02:00
Lei Liu
2700ac6d1f filer: Add attr.mode in query string
Signed-off-by: Lei Liu <liul.stone@gmail.com>
2019-11-27 13:57:29 +08:00
Lei Liu
3cc084269c master api: return http 404 when volumeId not exist
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
2019-09-29 14:23:39 +08:00
Chris Lu
ae53f63680 filer: recursive deletion optionally ignoring any errors
fix https://github.com/chrislusf/seaweedfs/issues/1062
2019-09-11 20:26:20 -07:00
Chris Lu
1d1b355f9e filer: set file name when uploading to a directory
fix https://github.com/chrislusf/seaweedfs/issues/1005
2019-07-11 04:20:03 -07:00
Chris Lu
359563e95a refactoring 2019-06-25 09:49:27 -07:00
Chris Lu
6c01fb6d2d go fmt 2019-06-23 01:57:51 -07:00
Chris Lu
d1cd8f8c5b add metrics, refactoring 2019-06-22 22:53:52 -07:00
Chris Lu
8b43679ae3 refactoring 2019-06-15 12:21:44 -07:00
Chris Lu
a11525fe4e filer: adds basic metrics pushing to Prometheus gateway 2019-06-13 02:01:54 -07:00
Chris Lu
338e6d60a5 refactor: prepare for snapshotting 2019-04-16 09:55:37 -07:00
Chris Lu
e85048bcdc http exhaust and close response body 2019-04-14 23:28:24 -07:00
Chris Lu
c7e7b6229f detect mime type 2019-03-27 14:25:18 -07:00
Chris Lu
cece860bfd add context to all filer APIs 2019-03-15 15:55:34 -07:00
Chris Lu
55c85f3a66 adding context 2019-03-15 15:26:09 -07:00
Chris Lu
77b9af531d adding grpc mutual tls 2019-02-18 12:11:52 -08:00
Chris Lu
74fb237727 benchmark can work in secure mode 2019-02-15 00:09:19 -08:00
Chris Lu
28a41fda3e assign new file id even on filer PUT operations 2019-01-02 12:58:26 -08:00
Chris Lu
2fb740a2b9 fix upload path 2018-12-17 00:21:34 -08:00
Chris Lu
7b7f4215aa filer update existing entry 2018-12-17 00:20:00 -08:00
Chris Lu
f6d8525d1d working S3 multipart uploads 2018-09-12 00:46:12 -07:00
Chris Lu
164091c269 add s3 multipart upload 2018-09-09 16:25:43 -07:00
Chris Lu
888eb2abb5 filer read write all via locations from MasterClient 2018-07-28 14:51:36 -07:00
Chris Lu
e4b7e31902 add back recursively delete a folder 2018-07-24 22:33:26 -07:00
Chris Lu
8f240276db s3 add HEAD DELETE 2018-07-21 18:49:47 -07:00
Chris Lu
bfa87b4f50 adjust log level 2018-07-21 18:47:23 -07:00
Chris Lu
12e6ccc581 remove unused code 2018-07-21 17:54:14 -07:00
Chris Lu
702d7ac424 delegating old chunk purging to CreateEntry
1. filer add file user id and group id
2. auto-chunking set the file name correctly
3. delegating old chunk purging to CreateEntry
2018-07-21 17:47:59 -07:00
Chris Lu
8480008a9a add s3 upload, and removing mono and multi part upload analyzer
removing mono and multi part upload analyzer, which were used just to determine the file name
2018-07-21 10:39:02 -07:00
Chris Lu
80d80daf64 set filer2.ErrNotFound for not found entry 2018-07-20 02:14:18 -07:00
Chris Lu
c7228fefa7 add bucket creation and deletion
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
2018-07-19 01:21:44 -07:00
Chris Lu
842dab07b4 weed mount can request to prioritize to write to a data center 2018-07-14 13:36:28 -07:00