mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-14 10:39:24 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,339 Commits 32 Branches 296 Tags 150 MiB
b67ebe67a6
Commit Graph

4890 Commits

Author SHA1 Message Date
chrislu
3a6eb8ca5f default bind to one ip address 
fix https://github.com/chrislusf/seaweedfs/issues/1937
 2022-03-11 14:02:39 -08:00
Konstantin Lebedev
f53cff045f checks disk file exist 2022-03-10 18:58:56 +05:00
chrislu
197ade6aeb Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-09 22:26:55 -08:00
chrislu
011a41b561 add back writes to swap file when too many in memory chunks are used. 2022-03-09 22:26:51 -08:00
zzq09494
2ea18fca48 fix:support some databases which not support 'IF NOT EXISTS' keyword of 'create table command' 2022-03-10 13:37:34 +08:00
banjiaojuhao
f28dbbe5c5 [bugfix] filer: 1. Delete uploaded chunks when upload failed. 2. Report error when upload is interrupted by user. 2022-03-10 11:40:39 +08:00
chrislu
e2b07737da Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-08 23:02:33 -08:00
chrislu
dc204dd137 fix nil entry 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1d340b4]

goroutine 130523 [running]:
github.com/chrislusf/seaweedfs/weed/filer.FileSize(...)
        /code/seaweedfs/weed/filer/filechunks.go:26
github.com/chrislusf/seaweedfs/weed/mount.(*WFS).Lookup(0xc000866d80, 0x1, 0xc002897f40, {0xc004b00980, 0x39}, 0x1ec19e0)
        /code/seaweedfs/weed/mount/weedfs_dir_lookup.go:59 +0x654
github.com/hanwen/go-fuse/v2/fuse.doLookup(0xc00033c000, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/opcode.go:333 +0x6b
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc000ab2420, 0xc00033c000)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:483 +0x1f3
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc000ab2420, 0x0)
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:456 +0x110
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        /code/go/pkg/mod/github.com/hanwen/go-fuse/v2@v2.1.0/fuse/server.go:323 +0x534
 2022-03-08 23:02:30 -08:00
banjiaojuhao
f7f2a597dd minor 2022-03-08 16:22:55 +08:00
banjiaojuhao
d61bea9038 [bugfix] filer: In file modification, old chunks will be mis-deleted when they are merged(Manifestized). 2022-03-08 16:22:55 +08:00
chrislu
3aeee3d748 ensure releasing file handle 2022-03-07 14:01:24 -08:00
chrislu
8136384473 remove debug message 2022-03-07 11:22:26 -08:00
banjiaojuhao
b9ff7723dd [bugfix] filer: nil pointer dereference 2022-03-07 23:26:25 +08:00
chrislu
da3d330616 s3 and filer transport using unix domain socket instead of tcp 2022-03-07 02:00:14 -08:00
chrislu
0cb17b45b1 refactoring 2022-03-07 01:59:01 -08:00
chrislu
6d3db4445b buffer for all range requests 2022-03-07 01:56:47 -08:00
chrislu
f3bcbeb60a a little optimization 2022-03-07 00:24:59 -08:00
chrislu
f7ee60996c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-07 00:08:09 -08:00
chrislu
6f3ec989a7 fix manifest batch size 2022-03-07 00:07:53 -08:00
Chris Lu
1e7fcef581
Merge pull request #2729 from banjiaojuhao/filer_metadata-resolve-manifest 
filer: support get metadata with resolved manifest chunk
 2022-03-07 00:05:46 -08:00
chrislu
bb0b784544 minor 2022-03-07 00:04:59 -08:00
banjiaojuhao
bfcc9ca808 filer: support metadata with resolved manifest chunk 2022-03-07 15:47:51 +08:00
banjiaojuhao
71f3046841 filer: add back isAppend function 2022-03-07 15:41:07 +08:00
chrislu
0ba4e4cd23 2.93 2022-03-06 18:54:12 -08:00
chrislu
46a28b8819 mount: adjust disk space based on quota 2022-03-06 17:22:49 -08:00
chrislu
ede6ce44c6 fix test 2022-03-06 17:09:55 -08:00
chrislu
21ef152423 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-03-06 17:04:24 -08:00
chrislu
f3442e36e6 mount: quota adjust error type to be syscall.ENOSPC 2022-03-06 17:04:21 -08:00
Chris Lu
89cd0c27a8
Merge pull request #2728 from kmlebedev/set_default_leveldb2 2022-03-06 07:26:30 -08:00
Konstantin Lebedev
cf444ebd07 Set default leveldb2 enabled 
avoid Filer store is enabled for both leveldb2 and mysql
 2022-03-06 18:27:25 +05:00
banjiaojuhao
a07c93f4dd filer: reset bytesBuffer before use 2022-03-06 21:07:36 +08:00
chrislu
c7e8ac18f0 mount: quota for one mounted collection 
related to https://github.com/seaweedfs/seaweedfs-csi-driver/issues/48
 2022-03-06 02:44:40 -08:00
chrislu
b7c992f410 add flag to enforce quota 2022-03-05 22:10:43 -08:00
chrislu
f1713c96ae avoid possible runtime error: index out of range [0] with length 0 2022-03-05 21:14:31 -08:00
chrislu
da76af187f mount: avoid possible index out of bounds error 2022-03-04 22:36:01 -08:00
chrislu
6e49e75a5b use logical number of files and sizes for statistics and quota 2022-03-04 18:47:44 -08:00
chrislu
f51e20028a mount: avoid comma in mount options 
fix https://github.com/chrislusf/seaweedfs/issues/2719
 2022-03-03 03:42:29 -08:00
chrislu
dc0f48682c unused 2022-03-03 03:41:35 -08:00
chrislu
28b8974a3a mount: fix directory pagination when using midnight commander 2022-03-03 02:59:31 -08:00
chrislu
a96d4254e9 filer, s3, volume server: a bit memory optimization 2022-03-02 20:15:28 -08:00
chrislu
6fbbc78574 stream reading a whole chunk 2022-03-02 13:50:46 -08:00
chrislu
784583afc6 avoid pool memory allocation if too large 2022-03-02 13:50:28 -08:00
chrislu
ba14307319 2.92 2022-02-28 15:22:19 -08:00
chrislu
fcf3714443 mount: add back support for filer.path 2022-02-28 12:16:53 -08:00
chrislu
80c017907b filer.backup: fix backing up encrypted chunks 
I have done filer.backup test:
replication.toml:
[sink.local]
enabled = true
directory = "/srv/test"
___
system@dat1:/srv/test$ weed filer.backup -filer=app1:8888 -filerProxy
I0228 12:39:28 19571 filer_replication.go:129] Configure sink to local
I0228 12:39:28 19571 filer_backup.go:98] resuming from 2022-02-28 12:04:20.210984693 +0100 CET
I0228 12:39:29 19571 filer_backup.go:113] backup app1:8888 progressed to 2022-02-28 12:04:20.211726749 +0100 CET 0.33/sec

system@dat1:/srv/test$ ls -l
total 16
drwxr-xr-x 2 system system 4096 Feb 28 12:39 a
-rw-r--r-- 1 system system   48 Feb 28 12:39 fu.txt
-rw-r--r-- 1 system system   32 Feb 28 12:39 _index.html
-rw-r--r-- 1 system system   68 Feb 28 12:39 index.php
system@dat1:/srv/test$ cat fu.txt
?	?=?^??`?f^};?{4?Z%?X0=??rV????|"?1??踪~??
system@dat1:/srv/test$
On the active mount on the target server it's:
system@app1:/srv/app$ ls -l
total 2
drwxrwxr-x 1 system system  0 Feb 28 12:04 a
-rw-r--r-- 1 system system 20 Feb 28 12:04 fu.txt
-rw-r--r-- 1 system system  4 Feb 28 12:04 _index.html
-rw-r--r-- 1 system system 40 Feb 28 12:04 index.php
system@app1:/srv/app$ cat fu.txt
This is static boy!
Filer was started with: weed filer master="app1:9333,app2:9333,app3:9333" -encryptVolumeData
It seems like it's still encrypted?
 2022-02-28 10:07:06 -08:00
chrislu
aad62ee148 mount: mark directory uncached if forgotten 2022-02-28 02:08:24 -08:00
chrislu
554e239097 release file handle 2022-02-28 01:23:14 -08:00
chrislu
3639fedd01 mount: fix fsync opened and renamed files 2022-02-28 00:34:17 -08:00
chrislu
63a9d8f01d ensure inodes are not duplicating unless hardlinked 2022-02-27 23:13:49 -08:00
chrislu
de77d00c81 correctly clean up for a file 2022-02-27 23:12:28 -08:00
chrislu
18543c6e8b minor 2022-02-27 23:11:09 -08:00
chrislu
09cd00f356 2.91 2022-02-27 04:03:39 -08:00
chrislu
941ced60a4 download 2 chunks if at the beginning of a file 2022-02-27 03:57:24 -08:00
chrislu
f9d9eed0c9 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-27 03:41:35 -08:00
chrislu
d602d68fd1 remove dead code 2022-02-27 03:41:32 -08:00
Chris Lu
6c07af6014
Merge pull request #2715 from gfx-labs/acl-pr 2022-02-27 03:27:32 -08:00
chrislu
c3792c8352 remove dead code 2022-02-27 03:03:19 -08:00
chrislu
aa9eef81e6 retire mount v1 2022-02-27 02:57:27 -08:00
eddy-gfx
fb940dd807
Merge branch 'chrislusf:master' into acl-pr 2022-02-27 04:52:08 -06:00
elee
881a0fe806 ensure compatibility 2022-02-27 04:50:59 -06:00
elee
954ad98e0d set canned acl on replication create 2022-02-27 04:49:31 -06:00
chrislu
6e1ab97988 use debug option to see operations 2022-02-27 02:02:30 -08:00
chrislu
2112d99140 mount2: add back readonly mode 2022-02-27 01:13:32 -08:00
chrislu
9ef5bb20f6 mount2: invalidate fuse cache for replaced inode 2022-02-27 00:00:23 -08:00
chrislu
4ddcbaab57 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-26 23:20:47 -08:00
chrislu
551d00d51a prefetch other chunks when stream reading 2022-02-26 23:20:45 -08:00
Chris Lu
e1d3dd5e18
Merge pull request #2712 from guo-sj/correct_comments 2022-02-26 09:16:12 -08:00
guosj
82cad5e330
correct comments 2022-02-26 22:44:26 +08:00
chrislu
7b1a713d2a remove dead code 2022-02-26 03:23:15 -08:00
chrislu
3a58b7bac6 a little safer 2022-02-26 03:23:06 -08:00
chrislu
708e14fcfa avoid possible too big memory allocation 2022-02-26 03:22:41 -08:00
chrislu
3345a50d9b prefetch 2 chunks 2022-02-26 03:06:17 -08:00
chrislu
86ce69f709 remove logs 2022-02-26 03:00:20 -08:00
chrislu
b2a148cb4c use file size as max range 2022-02-26 03:00:08 -08:00
chrislu
2ab0ad24a3 use memory pool 2022-02-26 02:59:19 -08:00
chrislu
28b395bef4 better control for reader caching 2022-02-26 02:16:47 -08:00
chrislu
3ad5fa6f6f chunk cache adds function ReadChunkAt 2022-02-25 21:55:04 -08:00
chrislu
fc7a4957ea fix mount2 options 2022-02-25 21:22:44 -08:00
chrislu
72c0233938 less logs 2022-02-25 15:34:24 -08:00
chrislu
101e6d80b0 mount2: listXattr return ok if xattr is empty 2022-02-25 14:38:56 -08:00
chrislu
03466f955e rename: delete source entry metadata only, skipping hard links 2022-02-25 02:57:54 -08:00
chrislu
8080fe4cc1 logs 2022-02-25 02:56:23 -08:00
chrislu
e423548673 rename: pass along entry metadata 2022-02-25 02:53:37 -08:00
chrislu
e8110bb54c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-25 01:17:28 -08:00
chrislu
202a29d014 refactoring 2022-02-25 01:17:26 -08:00
chrislu
91d6785cf3 define metadata action types 2022-02-25 00:54:16 -08:00
chrislu
be3fc77391 mount2: use consistent inode 2022-02-25 00:53:27 -08:00
Chris Lu
9873bae115
Merge pull request #2706 from guo-sj/fix_log_info 2022-02-24 22:48:19 -08:00
guosj
121b31f750 fix incorrect log information 2022-02-25 13:41:20 +08:00
chrislu
ceaf993a27 mount2: add rdev 2022-02-24 14:51:25 -08:00
chrislu
95717d1006 simpler output 2022-02-24 13:50:08 -08:00
chrislu
b05962b90e rename: handle hard links 2022-02-24 02:59:00 -08:00
chrislu
e31ec04f4f mount2: POSIX deleted opened file nlink should be 0 2022-02-24 01:59:37 -08:00
chrislu
419e355e9e Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 01:31:07 -08:00
chrislu
91f0481f4e mount2: SetAttr set mode correctly 2022-02-24 01:31:04 -08:00
Chris Lu
97a4b66df7
Merge pull request #2704 from guo-sj/fix_bugs_in_return_value 
fix return value in storage/volume_vacuum.go:444
 2022-02-24 00:49:29 -08:00
chrislu
abe7214c1f Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 00:37:59 -08:00
chrislu
b93d57da31 mount2: dir read opened file 2022-02-24 00:37:57 -08:00
guosj
3e7aa1caf5 fix return value in storage/volume_vacuum.go:444 2022-02-24 15:54:36 +08:00
Chris Lu
6834df77a0
Merge pull request #2701 from guo-sj/fix_bugs_in_return_value 2022-02-23 19:23:37 -08:00