mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-12 00:19:14 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,661 Commits 32 Branches 296 Tags 150 MiB
735e65be17
Commit Graph

147 Commits

Author SHA1 Message Date
Chris Lu
063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu
ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu
51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu
c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu
aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu
f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu
839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu
618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu
208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu
1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu
abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu
9d46c7bc78 rename 2020-08-17 11:12:10 -07:00
Chris Lu
f5837b7000 report error first 2020-08-16 23:49:10 -07:00
Chris Lu
2ac27616bc fix possible out of range bytes 
avoid buff out of range resp.Data = buff[:totalRead]
 2020-08-16 23:47:34 -07:00
Chris Lu
22e5132b3a adjust log level 2020-08-16 16:25:11 -07:00
Chris Lu
ee0f92a6be reduce memory allocation 2020-08-16 16:24:40 -07:00
Chris Lu
4a77f0820a clean up logs 2020-08-16 01:37:50 -07:00
Chris Lu
aec7f32b02 fix reader_at 2020-08-16 00:49:08 -07:00
Chris Lu
003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu
c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu
edfa73782f adjust log level 2020-08-14 00:22:21 -07:00
Chris Lu
20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu
d40de39e75 FUSE: do not change crtime, uid, gid on save 2020-07-24 10:13:45 -07:00
Chris Lu
8dfeba8023 FUSE: skip changing to empty uid and gid during flush 
Skip uid and gid changes if request uid and gid are zero

mongodb lock file, and many interim files, has empty uid and gid

I0724 10:05:51 93643 filehandle.go:173] /db/diagnostic.data/metrics.interim fh 1333342842031408359 flush Flush [ID=0x3 Node=0x50 Uid=0 Gid=0 Pid=178] 0x10 fl=0x0 lk=0x0
 2020-07-24 10:06:43 -07:00
Chris Lu
97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
Chris Lu
9033a7d369 removed async option 2020-06-28 10:18:32 -07:00
Chris Lu
1bb8cae65d reverting and working 2020-06-28 10:14:17 -07:00
Chris Lu
e8e0d629af ensure correct file mode 2020-06-24 23:26:54 -07:00
Chris Lu
628b27ef3b purge old cache implementation 2020-06-11 01:50:00 -07:00
Chris Lu
4598473251 fix possible EOF problem 2020-06-09 18:04:40 -07:00
Chris Lu
5d0e1d8d74 also writes to local meta cache 
before waiting for subscribed meta events
 2020-04-22 15:40:47 -07:00
Chris Lu
9cacaf5eca better detect file mime type 2020-04-14 11:32:31 -07:00
Chris Lu
13ab1fdaf0 mount: use file default permission 0666 2020-04-10 02:23:23 -07:00
Chris Lu
56ec8c4513 fix to avoid nil file an dirty pages 2020-04-09 21:43:05 -07:00
Chris Lu
006f78b70b make a copy of request data 
to avoid concurrency issues
 2020-04-08 22:31:19 -07:00
Chris Lu
6630541399 ensure correct file size when opening existing files 2020-04-08 12:50:59 -07:00
Chris Lu
9dc0b1df8f refactoring to compile for windows OS 
windows os does not like to work with fuse
 2020-03-29 21:07:55 -07:00
Chris Lu
826bc0b7e3 FUSE: add chunk cache for recently accessed file chunks 2020-03-28 13:43:31 -07:00
Chris Lu
d1439c5bd3 fix FUSE read for large files 
FUSE expects ReadAt do not return partial filled buffer with a nil error.
 2020-03-27 04:50:51 -07:00
Chris Lu
7f0d87b206 tree structured fs cache 
FsCache for FsNode, wrapping fs.Node
 2020-03-26 00:08:14 -07:00
Chris Lu
ae2ee379c0 consistent 64bit size 2020-03-22 01:37:46 -07:00
Chris Lu
65d2ea9fb0 FUSE mount: stream read data with buffer 
fix https://github.com/chrislusf/seaweedfs/issues/1244
 2020-03-22 01:00:36 -07:00
Chris Lu
97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu
892e726eb9 avoid reusing context object 
fix https://github.com/chrislusf/seaweedfs/issues/1182
 2020-02-25 21:50:12 -08:00
Chris Lu
6ab7368ef2 filer: dynamically create bucket under /buckets folder 2020-02-24 22:28:45 -08:00
Chris Lu
2f6bb57979 logging 2020-01-27 00:54:21 -08:00
Chris Lu
72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu
c48fc8b4de grpc send error via response instead of grpc error 2020-01-25 09:17:19 -08:00
Chris Lu
4433c7bdca logs 2020-01-25 07:34:09 -08:00
Chris Lu
24f62ebe76 minor changes 2020-01-24 10:02:53 -08:00
Chris Lu
9d93c83654 mount: return IO error for mkdir 2020-01-24 01:41:31 -08:00
Chris Lu
c936a12afa mount: update the file size so file Attr() can read 
if file Attr() during file writes, the OS, at least for Mac, may try to resend the data to write, which usually are empty.
 2020-01-23 21:59:58 -08:00
Chris Lu
c2e589f202 mount: better combines connected intervals to write to volume servers 2020-01-22 23:00:04 -08:00
Chris Lu
6b48d246a5 mount: read data that is just written 
able read on data not flushed
multiple file open shares the same file handle

fix https://github.com/chrislusf/seaweedfs/issues/1182 on linux
 2020-01-22 13:42:03 -08:00
bs106
ce79a31448 Fix missing leading dot in extension 2019-12-17 09:47:12 +00:00
Chris Lu
33da30d2b1 tweak 2019-12-14 11:04:20 -08:00
Chris Lu
5956dfd08d mount: add umask option 
related to https://github.com/chrislusf/seaweedfs/issues/978
 2019-07-24 00:03:05 -07:00
Chris Lu
a14bd31f5b go fmt 2019-06-22 20:05:25 -07:00
Chris Lu
ff97acae62 more logs 2019-06-21 12:14:40 -07:00
Chris Lu
11be0b5e91 more logging 2019-06-20 23:45:30 -07:00
Chris Lu
4b0f084b5d delete chunks only when file writing is successful 2019-06-20 21:58:35 -07:00
Chris Lu
eebc44cce3 refactoring 2019-06-20 21:57:17 -07:00
Chris Lu
b30c14b631 webdav: can read now 2019-05-03 00:24:35 -07:00
Chris Lu
c7e7b6229f detect mime type 2019-03-27 14:25:18 -07:00
Chris Lu
55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu
77b9af531d adding grpc mutual tls 2019-02-18 12:11:52 -08:00
Chris Lu
bd32108a90 disable async file deletion 2019-01-05 22:51:50 -08:00
Chris Lu
d9f9aca453 refactoring 2019-01-05 15:16:39 -08:00
Chris Lu
525db94d9a async file chunk deletion 2019-01-01 02:33:57 -08:00
Chris Lu
0388d421d2 caching visible intervals for read 
speeds up 4x in single thread mode
speeds up 30% in 32 threads mode
 2018-12-30 00:51:44 -08:00
Chris Lu
d7bf6c5b91 switching_to_seaweedfs_fuse 2018-12-29 10:31:36 -08:00
Chris Lu
308ac1d0d2 reuse buffer by sync pool 2018-12-28 03:27:48 -08:00
Chris Lu
3ac54792e1 paginate when filer deleting and FUSE mount renaming 2018-12-16 23:20:08 -08:00
Chris Lu
1bfb96f34d optimization for reading whole chunk with gzip encoding 2018-12-07 01:57:55 -08:00
Chris Lu
86dd933596 go fmt 2018-11-23 00:26:15 -08:00
Chris Lu
4fcfc9410f cleanup 2018-11-18 11:51:38 -08:00
Chris Lu
2e6a3c7b16 reduce memory usage 2018-11-14 22:48:54 -08:00
Chris Lu
11b0a9cb76 fix possible error when reusing file handles 2018-11-13 23:53:17 -08:00
Chris Lu
578f8a6dec fix empty file reading on linux 2018-09-23 10:01:00 -07:00
Chris Lu
7bb62b9f21 delay file creation without chunks from dir.Create to file.Flush 2018-09-22 00:11:46 -07:00
Chris Lu
747377142b fix file permission during writes 2018-09-07 13:57:06 -07:00
Chris Lu
75e749039b fix data writes error when consecutive calls overlaps with previous writes 2018-09-07 13:11:43 -07:00
Chris Lu
7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu
c8234a5af6 ensure using none nil attributes 
fix https://github.com/chrislusf/seaweedfs/issues/674
 2018-07-05 23:16:34 -07:00
Chris Lu
6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu
430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu
0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu
c4b92e17d0 fix isPerfectAppend 2018-05-28 22:45:52 -07:00
Chris Lu
d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu
07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu
458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu
ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu
0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu
7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu
d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu
849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu
536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00