mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-28 13:31:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,655 Commits 32 Branches 296 Tags 148 MiB
b64fc44e8d
Commit Graph

109 Commits

Author SHA1 Message Date
chrislu
0bffb3770f support TTL when creating folders 
related to https://github.com/seaweedfs/seaweedfs/discussions/4549
 2023-06-05 23:29:25 -07:00
zemul
e9fda774f4
[Filer] post add param:saveInside (#4434) 
* fix:mount deadlock

* feat: filer http upload to metadata

* feat: /etc save inside

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2023-04-29 08:31:05 -07:00
Rohit Chormale
59706c89fb
#4270 set http status code to 409 if dir already exists (#4287) 2023-03-08 07:45:08 -08:00
Chris Lu
d4566d4aaa
more solid weed mount (#4089) 
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
 2023-01-02 23:20:45 -08:00
famosss
15284e2bee
fix: Lost Content-Encoding metadata (#4010) 2022-11-23 22:52:34 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks() 
for later locking on reading chunks
 2022-11-15 06:33:36 -08:00
LHHDZ
d21e2f523d
split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824) 
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2022-10-11 20:14:14 -07:00
LHHDZ
3de1e19780
s3: sync bucket info from filer (#3759) 2022-09-29 12:29:01 -07:00
Konstantin Lebedev
5307cadb32
[s3] Replace code 500 to 499 for upload with ErrUnexpectedEOF (#3597) 2022-09-05 07:47:43 -07:00
chrislu
f8fa430257 filer: retryable data chunk upload 2022-08-20 19:15:44 -07:00
chrislu
689b4ecdcc remove unused collection and replication from upload result 2022-08-20 18:14:57 -07:00
Konstantin Lebedev
22181dd018
refactor FilerRequest metrics (#3402) 
* refactor FilerRequest metrics

* avoid double count proxy

* defer to
 2022-08-04 01:44:54 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
chrislu
27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu
79b8e6a8c3 add a place holder for later merge consecutive chunks 2022-04-06 00:28:48 -07:00
zzq09494
9f16df736e fix-filer: calculation error of the method skipCheckParentDirEntry 2022-03-19 09:15:42 +08:00
chrislu
4042fdf3bb rename to skipCheckParentDir 
related to https://github.com/chrislusf/seaweedfs/pull/2761

It's better to default to false.
 2022-03-16 23:55:31 -07:00
zzq09494
40b0033fa7 go fmt 2022-03-17 14:19:48 +08:00
zzq09494
81cce4b4c3 filer: support uploading file without needEnsureParentDir 2022-03-17 10:53:47 +08:00
zzq09494
a6a8892255 Revert "filer: support uploading file without needEnsureParentDir" 
This reverts commit a93c4947ba.
 2022-03-17 10:27:17 +08:00
zzq09494
a93c4947ba filer: support uploading file without needEnsureParentDir 2022-03-17 10:18:23 +08:00
chrislu
bb0b784544 minor 2022-03-07 00:04:59 -08:00
banjiaojuhao
71f3046841 filer: add back isAppend function 2022-03-07 15:41:07 +08:00
chrislu
f1713c96ae avoid possible runtime error: index out of range [0] with length 0 2022-03-05 21:14:31 -08:00
banjiaojuhao
6ab09e9071 filer_http: support uploading file with offset 2022-02-22 00:15:00 +08:00
chrislu
7bf14452f7 s3: handle Response-Content-Disposition 2021-12-12 23:05:40 -08:00
Chris Lu
88ff8fc27b ensure uploaded chunks are deleted on error 2021-11-29 00:28:26 -08:00
Chris Lu
bea26549f2 avoid empty extended attribute value 2021-11-28 23:50:52 -08:00
Konstantin Lebedev
c78220a7f2 fix object tagging https://github.com/chrislusf/seaweedfs/issues/2389 2021-10-20 16:01:06 +05:00
Chris Lu
de1ec050e8 fier/s3: save "Content-Disposition" to extended properties 
related to https://github.com/chrislusf/seaweedfs/issues/2371
 2021-10-12 00:04:28 -07:00
Chris Lu
e9760f261e minor 2021-09-13 00:31:06 -07:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu
fb8036385a s3: save metadata during put-object 
fix https://github.com/chrislusf/seaweedfs/issues/2092
 2021-05-29 14:14:30 -07:00
Konstantin Lebedev
69c768870b - object write cache control 
- object write expires
 2021-05-24 15:43:55 +05:00
Chris Lu
519b0e1e49 filer: upload to a directory without "/" suffix 
fix https://github.com/chrislusf/seaweedfs/issues/1988
 2021-04-12 11:56:56 -07:00
Chris Lu
8e404a1433 go fmt 2021-04-02 02:22:26 -07:00
Chris Lu
b5880334fc refactor 2021-03-30 21:07:34 -07:00
Chris Lu
a801332b0d filer: return 409 if file conflicts with a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1938
 2021-03-24 01:03:11 -07:00
Chris Lu
b1a86cf808 s3: copy object to itself 
fix https://github.com/chrislusf/seaweedfs/issues/1922
 2021-03-19 01:31:49 -07:00
Chris Lu
df72dc206d stats collect chunk upload retry count 2021-03-07 11:26:15 -08:00
Chris Lu
aa17311063 s3: set filename when uploading file chunks 2021-02-17 20:54:53 -08:00
Chris Lu
821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu
f8dbb03bdd filer: append operation returns final file size 2021-01-23 04:19:49 -08:00
Chris Lu
f98817cfe6 filer: support appending to a file 2021-01-20 13:40:32 -08:00
Chris Lu
1b8e3da295 refactoring 2021-01-20 12:59:54 -08:00
Chris Lu
1efb51ba84 filer: change to saveToFilerLimit from cacheToFilerLimit 
short circuit saving small files to volume server
 2021-01-10 23:14:46 -08:00
Peter Cai
cd2a9865ed filer: respect Content-Type on PUT 
This is needed for the S3 gateway to conform to the S3 protocol -- the
server should always respect the Content-Type set by client.
 2020-12-31 11:32:45 +08:00
Chris Lu
0a067944cc filer: add retries during volume moving 
fix https://github.com/chrislusf/seaweedfs/issues/1704
 2020-12-25 02:32:55 -08:00