mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-19 21:47:50 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,793 Commits 32 Branches 296 Tags 152 MiB
776308c4eb
Commit Graph

607 Commits

Author SHA1 Message Date
Chris Lu
a5b5887efc add filer replication to aws s3 2018-10-06 17:10:15 -07:00
Chris Lu
84d4a2c8a0 add filer replication to google cloud storage 2018-10-06 13:08:38 -07:00
Chris Lu
f8c2704d2b reformat 2018-10-06 13:04:33 -07:00
Chris Lu
14858f6224 add logging 2018-10-06 13:01:38 -07:00
Chris Lu
04da4c8094 add logging 2018-10-06 13:01:29 -07:00
Chris Lu
4a8ef198d7 add logging 2018-10-06 13:00:33 -07:00
Chris Lu
56c5c7b1b6 add google cloud storage 2018-10-04 01:14:44 -07:00
Chris Lu
e8ef501f02 add s3 replication sink 2018-10-03 23:36:52 -07:00
Chris Lu
56a5d5af8d ignore duplicated key error if the store has duplicated primary key prevention 
fix https://github.com/chrislusf/seaweedfs/issues/741
 2018-09-30 00:49:52 -07:00
Chris Lu
98384b62fd adjust description 2018-09-28 01:58:34 -07:00
Chris Lu
f11d531d45 adjust logging 2018-09-28 01:54:10 -07:00
Chris Lu
06c43286cd skip error when deletion failed 2018-09-28 00:10:49 -07:00
Chris Lu
746017ea4f releasing 0.97 2018-09-27 21:51:41 -07:00
Chris Lu
f7ff84f17f Revert "smalller block size for better accuracy" 
This reverts commit 97ecd20fc9.
 2018-09-27 09:42:24 -07:00
Chris Lu
97ecd20fc9 smalller block size for better accuracy 2018-09-27 09:39:17 -07:00
Chris Lu
f53632a84b add block size for linux du 
du already working on Mac
 2018-09-27 09:33:36 -07:00
Chris Lu
a362261972 avoid possible racing issue if leveldb or boltdb is used. 
remove the to-be-generated files during commit

fix https://github.com/chrislusf/seaweedfs/issues/738
 2018-09-26 18:45:51 -07:00
Chris Lu
31ed352ab6 replication handle cases when entry already exists 2018-09-25 09:27:03 -07:00
Chris Lu
60c1ada4c1 remove filer grpc API GetEntryAttributes 2018-09-25 09:18:06 -07:00
Chris Lu
578f8a6dec fix empty file reading on linux 2018-09-23 10:01:00 -07:00
Chris Lu
200d9b9ac2 add file name to log 2018-09-23 09:48:55 -07:00
Chris Lu
a60767c43a add more instructions 2018-09-23 09:26:39 -07:00
Chris Lu
12ba8b9799 generate useable filer.toml 2018-09-23 09:26:25 -07:00
Chris Lu
b1b8c4ed32 join via filepath 2018-09-23 01:46:50 -07:00
Chris Lu
b16d47030e save and use last processed kafka offsets 2018-09-23 01:34:40 -07:00
Chris Lu
9fe24991d5 refactoring 2018-09-23 00:40:36 -07:00
Chris Lu
7d6b2a4740 add ETag to upload results and chunks 2018-09-22 22:12:21 -07:00
Chris Lu
420f068376 add more instructions 2018-09-22 22:12:06 -07:00
Chris Lu
d3e1f20dcc adjust log level 2018-09-22 22:11:49 -07:00
Chris Lu
13d6a74eb2 consolidate file chunks during create 2018-09-22 22:11:37 -07:00
Chris Lu
fdc8a267de remove unused function 2018-09-22 22:11:13 -07:00
Chris Lu
9a3d46f8e3 handle duplicated replication when retry happens 2018-09-22 11:14:04 -07:00
Chris Lu
01ceace18e adjust sink options 2018-09-22 00:53:52 -07:00
Chris Lu
402ab598b6 go fmt 2018-09-22 00:12:10 -07:00
Chris Lu
7bb62b9f21 delay file creation without chunks from dir.Create to file.Flush 2018-09-22 00:11:46 -07:00
Chris Lu
db69ce89f0 go fmt 2018-09-21 01:56:43 -07:00
Chris Lu
a6cfaba018 able to sync the changes 2018-09-21 01:54:29 -07:00
Chris Lu
a8cd7df44a let filer choose default replication 2018-09-20 09:00:15 -07:00
Chris Lu
d5149e592e add logging on kafka reading 2018-09-19 22:31:45 -07:00
Chris Lu
d9b32db8d7 adjust uploads to a hidden folder 2018-09-19 22:03:16 -07:00
Chris Lu
f5471bcebf add bucket name in the redirection 2018-09-19 22:01:41 -07:00
Chris Lu
25fb6f9a46 fix compilation 2018-09-17 02:23:21 -07:00
Chris Lu
779641e9d4 adjust replicated entry name 2018-09-17 01:37:24 -07:00
Chris Lu
b7a375f5aa set large limit for "weed filer.export" directory listing 2018-09-17 00:42:36 -07:00
Chris Lu
4e1f1a0b92 set large limit for "weed mount" directory listing 2018-09-17 00:41:33 -07:00
Chris Lu
788acdf527 add WIP filer.replicate 2018-09-17 00:27:56 -07:00
Chris Lu
865a017936 fix when if buffer is not aligned 2018-09-17 00:27:40 -07:00
Chris Lu
97575e7185 do not set attributes if the file is still open 2018-09-16 12:37:06 -07:00
Chris Lu
70e67b7c6d add notification log 2018-09-16 11:20:36 -07:00
Chris Lu
5c6e3f457f adjust log 2018-09-16 11:20:08 -07:00
Chris Lu
bc03233364 refactoring 2018-09-16 01:37:35 -07:00
Chris Lu
d923ba2206 renaming msgqueue to notification 2018-09-16 01:18:30 -07:00
Chris Lu
bea4f6ca14 go fmt 2018-09-12 01:00:57 -07:00
Chris Lu
f628953c33 cleanup tmp upload directories 2018-09-12 01:00:51 -07:00
Chris Lu
f6d8525d1d working S3 multipart uploads 2018-09-12 00:46:12 -07:00
Chris Lu
439e88b5a6 fix format error 2018-09-11 13:01:51 -07:00
Chris Lu
267201ff44 handle non continuous writes 
fix #728
 2018-09-10 02:39:41 -07:00
Chris Lu
109385e955 simplifying logic to avoid handling non continuous writes 
fix https://github.com/chrislusf/seaweedfs/issues/728
 2018-09-10 02:21:57 -07:00
Chris Lu
296acc4a0a go fmt 2018-09-09 16:26:11 -07:00
Chris Lu
164091c269 add s3 multipart upload 2018-09-09 16:25:43 -07:00
Chris Lu
9b3bf0e46c fix "weed backup" rerunning 
"weed backup" rerunning will already have ReplicaPlacement set, while version is not set.
 2018-09-09 02:48:58 -07:00
Chris Lu
bc8d34143d Revert "fix "weed backup" rerunning" 
This reverts commit 1937be2bb2.
 2018-09-09 02:38:30 -07:00
Chris Lu
1937be2bb2 fix "weed backup" rerunning 
"weed backup" rerunning will already have ReplicaPlacement set, while version is not set.
 2018-09-09 02:31:12 -07:00
Chris Lu
d7a4b12d2a fix build error 2018-09-07 14:21:10 -07:00
Chris Lu
83ca595472 adjust version number 2018-09-07 14:19:48 -07:00
Chris Lu
747377142b fix file permission during writes 2018-09-07 13:57:06 -07:00
Chris Lu
dd37f0540f add extended attributes 2018-09-07 13:12:52 -07:00
Chris Lu
75e749039b fix data writes error when consecutive calls overlaps with previous writes 2018-09-07 13:11:43 -07:00
Chris Lu
f177cec509 avoid slice out of bounds 
avoid this problem

2018/09/04 16:27:14 fuse: panic in handler for Write [ID=0x27c0d Node=0x2 Uid=0 Gid=0 Pid=0] 0x1 131072 @10607788032 fl=WriteCache lock=0 ffl=OpenReadOnly: runtime error: slice bounds out of range
goroutine 211141 [running]:
bazil.org/fuse/fs.(*Server).serve.func2(0x10d3e60, 0xc00014be30, 0xc00052fef8, 0xc00052fe77)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:857 +0x1ac
panic(0xe2d080, 0x17f62b0)
	/home/travis/.gimme/versions/go/src/runtime/panic.go:513 +0x1b9
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).saveToStorage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0xc0005dc000, 0x20000, 0x1000000, 0x276720000, 0xc0003feaa0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:142 +0x8ec
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).saveExistingPagesToStorage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:107 +0x6c
github.com/chrislusf/seaweedfs/weed/filesys.(*ContinuousDirtyPages).AddPage(0xc0000aca80, 0x10d7ba0, 0xc0003fcc00, 0x278460000, 0xc011966050, 0x20000, 0x20fb0, 0x6fc23ac00, 0x4a817c800, 0x0, ...)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/dirty_page.go:70 +0x8f
github.com/chrislusf/seaweedfs/weed/filesys.(*FileHandle).Write(0xc000548410, 0x10d7ba0, 0xc0003fcc00, 0xc00014be30, 0xc011946af8, 0x47fa01, 0x0)
	/home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/filesys/filehandle.go:141 +0x245
bazil.org/fuse/fs.(*Server).handleRequest(0xc0002cc0c0, 0x10d7ba0, 0xc0003fcc00, 0x10cb020, 0xc000394140, 0xc0000acac0, 0x10d3e60, 0xc00014be30, 0xc00052fef8, 0x10ca6a0, ...)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:1265 +0x1599
bazil.org/fuse/fs.(*Server).serve(0xc0002cc0c0, 0x10d3e60, 0xc00014be30)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:878 +0x410
bazil.org/fuse/fs.(*Server).Serve.func1(0xc0002cc0c0, 0x10d3e60, 0xc00014be30)
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:425 +0x6e
created by bazil.org/fuse/fs.(*Server).Serve
	/home/travis/gopath/src/bazil.org/fuse/fs/serve.go:423 +0x321
 2018-09-05 02:17:04 -07:00
Chris Lu
d614716ff2 fix compilation error 2018-09-04 00:43:46 -07:00
Chris Lu
ce6a2140a2 preparing to support S3 multipart uploads 2018-09-04 00:42:44 -07:00
Chris Lu
7f760f16b0 prepare for 0.95 2018-09-03 20:52:52 -07:00
Chris Lu
35bafd9861 remove unused function 2018-09-03 16:50:32 -07:00
Chris Lu
7ad5a90127 go fmt 2018-09-03 16:47:00 -07:00
Chris Lu
2ed6d8cca6 refactoring filer rm 2018-09-03 13:16:26 -07:00
Chris Lu
03f852c799 refactoring 2018-09-03 13:03:16 -07:00
Chris Lu
0b0ece9649 add place holders for multiplarts upload 2018-09-03 11:38:10 -07:00
Chris Lu
0adb2ba6b8 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2018-09-03 00:29:02 -07:00
Chris Lu
06b81c063a fix possible connection counting error 2018-09-03 00:28:58 -07:00
zhangmingfeng
aafe8d84cb
Update volume_server_handlers_read.go 
change ext to lower-case
 2018-09-03 11:01:35 +08:00
Chris Lu
88f1d32cc4 add s3ChunkedReader 
fix https://github.com/chrislusf/seaweedfs/issues/718
 2018-09-02 14:20:47 -07:00
Chris Lu
76cbe8bf33 instant notification of new volumes added or deleted 2018-08-24 01:26:56 -07:00
Chris Lu
5ccf8e8078 reformat 2018-08-24 00:30:03 -07:00
Chris Lu
ac793a3c5a clean remove all left over files when deleting a collection 2018-08-23 23:33:16 -07:00
Chris Lu
d361a1ddf5 fix id String() instead 2018-08-23 22:44:30 -07:00
Chris Lu
694f93de80 parse needle id correctly 
fix https://github.com/chrislusf/seaweedfs/issues/713
 2018-08-23 20:48:43 -07:00
Chris Lu
37e4e022fd prepare for 0.94 release 2018-08-23 00:02:13 -07:00
Chris Lu
6e3f4d1079 add working filer.export command 2018-08-23 00:02:04 -07:00
Chris Lu
98b8f8649d adding filer.export prototype 2018-08-22 23:54:22 -07:00
Chris Lu
2b4277ec18 fix nil bug with filer.toml 2018-08-22 23:54:00 -07:00
Chris Lu
ae4e654d2a add example configuration for default redis create-cluster setup 2018-08-22 18:23:19 -07:00
Chris Lu
b23883b3b6 fix a bug on error reporting 2018-08-22 18:19:26 -07:00
Chris Lu
1f030af06f adjusting startup logging 2018-08-22 13:59:04 -07:00
Chris Lu
bb92fd1db0 fix test error 2018-08-19 18:52:50 -07:00
Chris Lu
53d171cf5b releasing 0.93 2018-08-19 18:45:26 -07:00
Chris Lu
906472b21d filer add upload and drag&dorp 2018-08-19 18:42:40 -07:00
Chris Lu
f22d6b87e1 go fmt 2018-08-19 15:42:50 -07:00
Chris Lu
748870b03f add "weed scaffold" command 2018-08-19 15:36:30 -07:00
Chris Lu
95b997fb35 go fmt 2018-08-19 15:18:37 -07:00
Chris Lu
f827ada811 merge notification config with filer.toml 2018-08-19 15:17:55 -07:00