mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-30 07:09:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,968 Commits 32 Branches 296 Tags 148 MiB
61ef2d8658
Commit Graph

431 Commits

Author SHA1 Message Date
Chris Lu
d89774cb7f cache grpc client connections to improve performance 2018-11-14 23:31:39 -08:00
Chris Lu
2e6a3c7b16 reduce memory usage 2018-11-14 22:48:54 -08:00
Chris Lu
11b0a9cb76 fix possible error when reusing file handles 2018-11-13 23:53:17 -08:00
Chris Lu
8178fc9973 adjust logging 2018-11-11 00:07:46 -08:00
Chris Lu
cbd94b18a5 improve "ls -al" performance for large directory 2018-11-08 07:37:34 -08:00
Chris Lu
ed8efb5aef much improved "ls -al" performance 2018-11-07 11:35:13 -08:00
Chris Lu
ff4189ab25 go fmt 2018-11-04 11:59:08 -08:00
Chris Lu
9db53cf256 use the configurable chunk size 2018-10-14 00:18:52 -07:00
Chris Lu
3ddcd87098 increase weed mount IO perforance 2018-10-14 00:10:17 -07:00
Chris Lu
08266b7256 go fmt 2018-10-11 00:08:13 -07:00
Chris Lu
f7ff84f17f Revert "smalller block size for better accuracy" 
This reverts commit 97ecd20fc9.
 2018-09-27 09:42:24 -07:00
Chris Lu
97ecd20fc9 smalller block size for better accuracy 2018-09-27 09:39:17 -07:00
Chris Lu
f53632a84b add block size for linux du 
du already working on Mac
 2018-09-27 09:33:36 -07:00
Chris Lu
60c1ada4c1 remove filer grpc API GetEntryAttributes 2018-09-25 09:18:06 -07:00
Chris Lu
578f8a6dec fix empty file reading on linux 2018-09-23 10:01:00 -07:00
Chris Lu
200d9b9ac2 add file name to log 2018-09-23 09:48:55 -07:00
Chris Lu
7d6b2a4740 add ETag to upload results and chunks 2018-09-22 22:12:21 -07:00
Chris Lu
402ab598b6 go fmt 2018-09-22 00:12:10 -07:00
Chris Lu
7bb62b9f21 delay file creation without chunks from dir.Create to file.Flush 2018-09-22 00:11:46 -07:00
Chris Lu
97575e7185 do not set attributes if the file is still open 2018-09-16 12:37:06 -07:00
Chris Lu
267201ff44 handle non continuous writes 
fix #728
 2018-09-10 02:39:41 -07:00
Chris Lu
109385e955 simplifying logic to avoid handling non continuous writes 
fix https://github.com/chrislusf/seaweedfs/issues/728
 2018-09-10 02:21:57 -07:00
Chris Lu
296acc4a0a go fmt 2018-09-09 16:26:11 -07:00
Chris Lu
747377142b fix file permission during writes 2018-09-07 13:57:06 -07:00
Chris Lu
75e749039b fix data writes error when consecutive calls overlaps with previous writes 2018-09-07 13:11:43 -07:00
Chris Lu
f177cec509 avoid slice out of bounds 
avoid this problem

2018/09/04 16:27:14 fuse: panic in handler for Write [ID=0x27c0d Node=0x2 Uid=0 Gid=0 Pid=0] 0x1 131072 @10607788032 fl=WriteCache lock=0 ffl=OpenReadOnly: runtime error: slice bounds out of range
goroutine 211141 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x10d3e60, 0xc00014be30, 0xc00052fef8, 0xc00052fe77)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:857 +0x1ac
panic(0xe2d080, 0x17f62b0)
	/home/travis/.gimme/versions/go/src/runtime/panic.go:513 +0x1b9
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).saveToStorage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0xc0005dc000, 0x20000, 0x1000000, 0x276720000, 0xc0003feaa0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:142 +0x8ec
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).saveExistingPagesToStorage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:107 +0x6c
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).AddPage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0x278460000, 0xc011966050, 0x20000, 0x20fb0, 0x6fc23ac00, 0x4a817c800, 0x0, ...)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:70 +0x8f
github.com/chrislusf/seaweedfs/weed/filesys.(*FileHandle).Write(0xc000548410, 0x10d7ba0, 0xc0003fcc00, 0xc00014be30, 0xc011946af8, 0x47fa01, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/filehandle.go:141 +0x245
bazil.org/fuse/fs.(*Server).handleRequest(0xc0002cc0c0, 0x10d7ba0, 0xc0003fcc00, 0x10cb020, 0xc000394140, 0xc0000acac0, 0x10d3e60, 0xc00014be30, 0xc00052fef8, 0x10ca6a0, ...)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:1265 +0x1599
bazil.org/fuse/fs.(*Server).serve(0xc0002cc0c0, 0x10d3e60, 0xc00014be30)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:878 +0x410
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0002cc0c0, 0x10d3e60, 0xc00014be30)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:425 +0x6e
created by bazil.org/fuse/fs.(*Server).Serve
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:423 +0x321
 2018-09-05 02:17:04 -07:00
Chris Lu
0655151b8c go fmt 2018-07-22 01:15:11 -07:00
Chris Lu
6319d84f42 s3 API add ListObjectsV1 2018-07-22 01:14:36 -07:00
Chris Lu
7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu
13e5541e17 FUSE can change file or folder attributes 
FUSE can change file or folder attributes
 2018-07-19 02:17:36 -07:00
Chris Lu
7abfab8e77 add feature to mount a specific filer path to local directory 2018-07-14 21:09:21 -07:00
Chris Lu
842dab07b4 weed mount can request to prioritize to write to a data center 2018-07-14 13:36:28 -07:00
Chris Lu
c8234a5af6 ensure using none nil attributes 
fix https://github.com/chrislusf/seaweedfs/issues/674
 2018-07-05 23:16:34 -07:00
Chris Lu
77fc8c5914 keep alive for gRpc calls 2018-07-03 19:07:55 -07:00
Chris Lu
d2a811eef2 fix compilation 2018-06-11 23:29:09 -07:00
Chris Lu
5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu
f1273073fc switch to fs.NodeRequestLookuper 
in order to set the entry valid duration
 2018-06-07 00:07:37 -07:00
Chris Lu
cec1d97035 recursively move files and directories 2018-06-06 23:39:30 -07:00
Chris Lu
ae23e46313 better error log 2018-06-06 23:06:19 -07:00
Chris Lu
e755540be9 fix caching during directory listing 2018-06-06 22:55:59 -07:00
Chris Lu
daabdfe357 remove nodemap, fix directory listing cache 2018-06-06 22:48:51 -07:00
Chris Lu
5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu
b3447f4375 adjust logging 2018-06-06 02:21:36 -07:00
Chris Lu
6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu
299312c805 use separate filer grpc port 2018-06-05 23:37:41 -07:00
Chris Lu
a218eaf1f0 fix log 2018-05-30 22:09:24 -07:00
Chris Lu
430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu
0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu
4e3ea49cff properly working local write buffer 2018-05-29 01:21:21 -07:00
Chris Lu
c4b92e17d0 fix isPerfectAppend 2018-05-28 22:45:52 -07:00
Chris Lu
74332e1a61 minor 2018-05-28 14:32:16 -07:00
Chris Lu
be0e88a606 fix chunk size limit for default 0 2018-05-28 13:44:27 -07:00
Chris Lu
5c4480ec6c add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug 2018-05-28 13:42:25 -07:00
Chris Lu
8ab7dd9d08 weed mount add options for collection and replication 2018-05-28 13:24:48 -07:00
Chris Lu
d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu
07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu
458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu
6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu
ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu
6d1bcd4b8c use existing attributes instead of fetching from filer 2018-05-25 01:22:31 -07:00
Chris Lu
0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu
7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu
9f4f8de9ad skip printout 2018-05-24 23:20:12 -07:00
Chris Lu
d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu
849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu
536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu
873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu
7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu
9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu
f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu
7ca5052942 create files correctly! 2018-05-19 13:51:44 -07:00
Chris Lu
793dd81ca2 skip permission checking when creating dir or files 2018-05-19 12:40:24 -07:00
Chris Lu
e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu
6bf31467c7 adding empty fsync 
less error now. But still:

vi on write: E514: write error (file system full?)
cp: Input/output error
 2018-05-16 00:54:27 -07:00
Chris Lu
b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu
58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu
9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu
43a69d20bf change filer API to gRPC 2018-05-08 01:59:43 -07:00
Chris Lu
be58993f47 ensure the same Node is returned 2018-05-05 23:50:34 -07:00
Chris Lu
b857cf9d9b format 2018-05-05 23:39:29 -07:00
Chris Lu
456738ba64 refactoring fuse 2018-05-05 22:47:16 -07:00