mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-12-05 03:29:03 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,218 Commits 32 Branches 296 Tags 149 MiB
921535001a
Commit Graph

4855 Commits

Author SHA1 Message Date
chrislu
91d6785cf3 define metadata action types 2022-02-25 00:54:16 -08:00
chrislu
be3fc77391 mount2: use consistent inode 2022-02-25 00:53:27 -08:00
Chris Lu
9873bae115
Merge pull request #2706 from guo-sj/fix_log_info 2022-02-24 22:48:19 -08:00
guosj
121b31f750 fix incorrect log information 2022-02-25 13:41:20 +08:00
chrislu
ceaf993a27 mount2: add rdev 2022-02-24 14:51:25 -08:00
chrislu
95717d1006 simpler output 2022-02-24 13:50:08 -08:00
chrislu
b05962b90e rename: handle hard links 2022-02-24 02:59:00 -08:00
chrislu
e31ec04f4f mount2: POSIX deleted opened file nlink should be 0 2022-02-24 01:59:37 -08:00
chrislu
419e355e9e Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 01:31:07 -08:00
chrislu
91f0481f4e mount2: SetAttr set mode correctly 2022-02-24 01:31:04 -08:00
Chris Lu
97a4b66df7
Merge pull request #2704 from guo-sj/fix_bugs_in_return_value 
fix return value in storage/volume_vacuum.go:444
 2022-02-24 00:49:29 -08:00
chrislu
abe7214c1f Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-24 00:37:59 -08:00
chrislu
b93d57da31 mount2: dir read opened file 2022-02-24 00:37:57 -08:00
guosj
3e7aa1caf5 fix return value in storage/volume_vacuum.go:444 2022-02-24 15:54:36 +08:00
Chris Lu
6834df77a0
Merge pull request #2701 from guo-sj/fix_bugs_in_return_value 2022-02-23 19:23:37 -08:00
guosj
26f3ab8d4b fix a return bug in func (c *commandVacuum) Do 2022-02-24 09:04:38 +08:00
chrislu
320637dc7a use "mv.from" for moving files 2022-02-23 15:34:42 -08:00
chrislu
c29bc9a367 fix error handling 2022-02-23 15:34:25 -08:00
Chris Lu
da58c748bc
Merge pull request #2698 from guo-sj/fix_bugs_in_return_value 2022-02-23 00:41:30 -08:00
Chris Lu
bd092d8318
Merge pull request #2697 from guo-sj/fix_bugs_in_return_value 2022-02-23 00:30:52 -08:00
guosj
d68c27f82d fix another return value bug 2022-02-23 16:21:25 +08:00
guosj
8f9aa0cddd fix bugs in return value 2022-02-23 16:17:48 +08:00
Tuan Vuong
d2ec62656d initialize master address in iam options 2022-02-23 12:01:54 +07:00
banjiaojuhao
6ab09e9071 filer_http: support uploading file with offset 2022-02-22 00:15:00 +08:00
chrislu
497ebbbd45 2.90 2022-02-20 22:00:13 -08:00
chrislu
56da3494cb leveldb3: adjust memory allocation for each bucket 2022-02-20 13:40:13 -08:00
banjiaojuhao
e6126cef62 filer_web: support moving entry 2022-02-20 23:56:23 +08:00
chrislu
7bc67399e4 listing for filer stores without prefixed query: break if no more progress 2022-02-19 10:12:51 -08:00
chrislu
248c0c8087 minor 2022-02-19 09:54:09 -08:00
chrislu
4ee0a6f47b filer store: reduce one possible listing operation 2022-02-19 00:43:42 -08:00
chrislu
b3594278c9 optimize a bit 2022-02-18 23:36:10 -08:00
chrislu
61811dc2f1 comments 2022-02-18 22:14:40 -08:00
chrislu
daa27b2119 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-18 20:55:10 -08:00
chrislu
9014d00fd0 Revert "s3: listObjectParts return ErrNoSuchUpload if does not exist" 
This reverts commit 6cf2e7d493.
 2022-02-18 20:54:54 -08:00
banjiaojuhao
4c30934cd9 filer: support get file entry 2022-02-18 22:52:26 +08:00
chrislu
63062ed7f0 mount2: fix unlink 2022-02-18 01:10:53 -08:00
chrislu
93e12d5f30 fix build 2022-02-18 00:48:00 -08:00
chrislu
d62370d4e0 add todo 2022-02-18 00:47:15 -08:00
chrislu
e8ce30fdc5 mount2: adjust file mode 2022-02-18 00:47:02 -08:00
chrislu
f9d33f70b0 return fuse.Status when looking up by inode 2022-02-18 00:45:43 -08:00
chrislu
b9cf4f12fc Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-16 21:33:51 -08:00
chrislu
49b84b6e2a list entries while reading from remote 2022-02-16 21:32:15 -08:00
Chris Lu
38dcaaa76e
Merge pull request #2683 from guo-sj/fix_fsconfigure_bug 
fix minor bug in commandFsConfigure.Do
 2022-02-16 18:16:47 -08:00
guosj
ca121ecd66 fix minor bug in commandFsConfigure.Do 2022-02-17 09:57:52 +08:00
chrislu
65a19e3abc fix listing with correct inode 2022-02-16 17:01:39 -08:00
chrislu
6ac066d1dc count lookup or not 2022-02-16 16:49:03 -08:00
chrislu
a129bda7d9 sync data first before stopping 2022-02-16 09:11:34 -08:00
chrislu
118d0e01a8 less logs 2022-02-16 08:45:07 -08:00
chrislu
a6bc67c34c less logs 2022-02-16 08:38:51 -08:00
chrislu
2facd65998 fix second listing 2022-02-16 08:16:27 -08:00
chrislu
1013fc90d8 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-16 07:57:11 -08:00
chrislu
22739c653a clean up 2022-02-16 07:57:08 -08:00
Chris Lu
3cb19cf6db
Merge pull request #2680 from kmlebedev/volume_healthz 2022-02-16 04:43:44 -08:00
Konstantin Lebedev
9ea09cc41c healthz check to avoid drain pod with last replicas 2022-02-16 14:18:36 +05:00
chrislu
65bfeafb5a add back mkdir 2022-02-16 01:09:21 -08:00
chrislu
37e8fce841 clean up cache on exit 2022-02-16 00:39:21 -08:00
chrislu
1560ec7e26 remove unused code 2022-02-16 00:37:24 -08:00
chrislu
3cbce878f2 mount2: fix directory pagination 2022-02-15 22:42:10 -08:00
chrislu
df51e0c042 mongodb: remove ErrNoDocuments checking since it is only for creating single record, not for querying 2022-02-15 16:18:36 -08:00
chrislu
98cce6a150 mongodb: ensure closing cursor 2022-02-15 16:15:44 -08:00
Chris Lu
aa7fc299b8
Merge pull request #2676 from banjiaojuhao/add_filer_store-redis_lua 
FilerStore: add redis_lua
 2022-02-15 12:10:34 -08:00
banjiaojuhao
fc3b75f2f8 weed/operation/delete_content.go: nil pointer dereference 2022-02-15 23:07:03 +08:00
banjiaojuhao
b5ec346700 FilerStore: add redis_lua 2022-02-15 20:54:57 +08:00
chrislu
17ac5244c3 mount2: avoid double listing directories 2022-02-15 01:44:17 -08:00
chrislu
222798d926 mount2: fix for read dir plus on linux 2022-02-15 00:29:17 -08:00
chrislu
4e72863ba5 mount2 add debug mode 2022-02-15 00:26:30 -08:00
Chris Lu
64afbc5235
Merge pull request #2640 from Radtoo/fsck_with_delete 
Added basic deletion capabilities to fsck.
 2022-02-14 23:00:01 -08:00
chrislu
4e181db21a mount: default disable cache 
* Prevent cases as https://github.com/seaweedfs/seaweedfs-csi-driver/issues/43
* Improve read write benchmarks
* Improve AI training performance. Most of the files are just read once.
 2022-02-14 20:42:33 -08:00
chrislu
ebc22625b4 less noisy on mac 2022-02-14 16:38:01 -08:00
chrislu
377bf31445 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-14 13:48:51 -08:00
chrislu
e8420aaed7 fix building for windows freebsd 2022-02-14 13:48:48 -08:00
Chris Lu
d3ee621fce
Merge pull request #2661 from garenchan/ck-dev1 2022-02-14 10:08:37 -08:00
Konstantin Lebedev
526094d2da StopTimeout 30 sec 2022-02-14 21:42:27 +05:00
Konstantin Lebedev
275e9a4e86 reduce to default http server KillTimeout and StopTimeout 2022-02-14 21:38:24 +05:00
garenchan
bd032eabe7 [UPDATE] Make heartbeat interval and election timeout of masters configurable. 2022-02-14 21:09:07 +08:00
chrislu
ff666104c4 fix GOOS 2022-02-14 03:14:05 -08:00
chrislu
05724a68d4 skip other OS 2022-02-14 02:59:51 -08:00
chrislu
4244ef8b72 fix building on linux 2022-02-14 02:59:27 -08:00
chrislu
b9c2bff931 clean up 2022-02-14 02:14:26 -08:00
chrislu
a990cd29cd Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-14 01:52:25 -08:00
chrislu
6a40fd1c65 2.89 2022-02-14 01:52:16 -08:00
Chris Lu
305418fac5
Merge pull request #2667 from kmlebedev/syncReplicatedWrite 
atomically write to replicas
 2022-02-14 01:48:21 -08:00
Chris Lu
aa13168b4d
Merge pull request #2668 from chrislusf/mount2 
Mount2
 2022-02-14 01:41:09 -08:00
chrislu
fe57a2e770 file set attribute 2022-02-14 01:36:10 -08:00
Konstantin Lebedev
0ed76a0556 clearly 2022-02-14 14:10:06 +05:00
chrislu
dbeeda8123 listen for metadata updates 2022-02-14 01:09:31 -08:00
Konstantin Lebedev
36013f63ed https://github.com/chrislusf/seaweedfs/issues/2648 2022-02-14 13:59:12 +05:00
chrislu
7286e525ad support write 2022-02-13 23:27:11 -08:00
chrislu
2b955c1713 support read 2022-02-13 22:50:44 -08:00
chrislu
f3c1e00521 rename 2022-02-13 19:16:56 -08:00
chrislu
bb9919b07a add open release, refactor 2022-02-13 19:14:34 -08:00
Eng Zer Jun
b92df1654c
test: use T.TempDir to create temporary test directory 
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-02-14 10:38:13 +08:00
chrislu
072f923a9b add doc 2022-02-13 16:56:35 -08:00
chrislu
3d0e9e5197 supports renaming 2022-02-13 16:34:57 -08:00
chrislu
417b59b893 doc 2022-02-13 14:37:47 -08:00
chrislu
24290fed9d add comments 2022-02-13 06:02:21 -08:00
chrislu
6200b6abb1 avoid fatal message in some edge cases 2022-02-13 05:59:10 -08:00
chrislu
6a921e15f3 forget() factor in nlookup 2022-02-13 05:49:29 -08:00
chrislu
f8af0f93d9 support link 2022-02-13 05:29:43 -08:00
chrislu
0381338d40 fix import 2022-02-13 04:23:06 -08:00
chrislu
a1ef0e48a9 doc 2022-02-13 04:22:02 -08:00
chrislu
be40ff6048 add symlink 2022-02-13 03:50:16 -08:00
chrislu
3d93570979 supports forget 2022-02-13 03:31:47 -08:00
chrislu
6a42cb6b0b supports mknod, unlink 2022-02-13 03:09:24 -08:00
chrislu
813b868b9a add rmdir 2022-02-13 01:43:11 -08:00
chrislu
e85ca10a1a add mkdir 2022-02-13 01:34:19 -08:00
chrislu
21046c6a28 split files 2022-02-13 01:05:30 -08:00
chrislu
a4c9223b9d support xattr 2022-02-13 00:58:46 -08:00
chrislu
c81833a192 add directory setAttr 2022-02-12 23:08:56 -08:00
chrislu
5c48c23235 remove println 2022-02-12 22:45:07 -08:00
chrislu
7cfbf1e85f fix ok status 2022-02-12 22:41:45 -08:00
chrislu
4c75fd5f9c sync format 2022-02-12 22:41:29 -08:00
chrislu
b0a5193e32 working 2022-02-12 22:21:30 -08:00
chrislu
661a34e23d Merge branch 'master' into mount2 2022-02-12 17:58:32 -08:00
chrislu
5e814afe88 blocks count 2022-02-12 17:58:12 -08:00
Radtoo
724ce04b1c We picked the second flag's name. 2022-02-13 00:53:35 +01:00
chrislu
5a0a709016 it runs, but directory listing output is not showing up 2022-02-12 05:27:16 -08:00
chrislu
866981d8ac rename 2022-02-12 02:49:15 -08:00
chrislu
72faae91e1 implement read directory and read directory plus 2022-02-12 02:48:44 -08:00
chrislu
a10c28ba82 simplify 2022-02-12 01:59:36 -08:00
chrislu
f4d88862c4 can attr root directory 2022-02-12 01:54:16 -08:00
chrislu
180445f5a8 change to use fuse file system 2022-02-11 21:35:09 -08:00
chrislu
45a0fda9bd need to follow https://github.com/hanwen/go-fuse/blob/master/fuse/api.go 2022-02-11 03:12:52 -08:00
chrislu
f87da798a4 to be re-written following fuse virtual file system 2022-02-11 03:09:30 -08:00
chrislu
9a913457dd supports stats 2022-02-10 23:23:47 -08:00
chrislu
b6143de52a mount with name 2022-02-10 22:43:55 -08:00
chrislu
7a0c35674c clean up previously mounted folder 2022-02-10 20:46:53 -08:00
chrislu
c3f9d9fa2e initial setup 2022-02-10 20:32:13 -08:00
Radtoo
fbe7ed7927 Revert the the flag to "reallyDeleteFromVolume". 2022-02-10 20:01:48 +01:00
chrislu
21aaa4c1f1 ec.encode: calculate free ec slots based on (maxVolumeCount-volumeCount) 
fix https://github.com/chrislusf/seaweedfs/issues/2642
 2022-02-08 01:51:13 -08:00
chrislu
13f6ec1c4e test checking ec distribution 2022-02-08 01:50:05 -08:00
chrislu
f34c2ff7c5 use embed txt 2022-02-08 01:14:27 -08:00
chrislu
c8c7c10c3f volume.tier.move: avoid double counting 
related to https://github.com/chrislusf/seaweedfs/issues/2637
 2022-02-08 00:57:35 -08:00
chrislu
f18803424a volume.balance: add delay during tight loop 
fix https://github.com/chrislusf/seaweedfs/issues/2637
 2022-02-08 00:53:55 -08:00
chrislu
9860405974 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-07 23:08:57 -08:00
chrislu
85c1615b43 filer read empty file may cause OOM in some cases 
fix https://github.com/chrislusf/seaweedfs/issues/2641
 2022-02-07 23:08:54 -08:00
Chris Lu
a2ac540ecc
Merge pull request #2645 from guol-fnst/fix_Precedence 
fix preconditions
 2022-02-07 18:32:16 -08:00
root
7f0c793083 fix preconditions according to https://tools.ietf.org/id/draft-ietf-httpbis-p4-conditional-26.html#preconditions 2022-02-08 10:13:19 +08:00
chrislu
b1cff07ab0 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-02-07 11:38:01 -08:00
chrislu
81f86c381d s3: avoid nil response 
fix https://github.com/chrislusf/seaweedfs/issues/2636
 2022-02-07 11:37:59 -08:00
Chris Lu
f46763d74f
Merge pull request #2644 from kmlebedev/vacum_fsync 
updated needle with fsync
 2022-02-07 11:33:58 -08:00
Konstantin Lebedev
ef541972f8 updated needle with fsync 2022-02-08 00:10:53 +05:00
Chris Lu
1b7fb3ce84
Merge pull request #2643 from kmlebedev/graceful_volume_stop 2022-02-07 09:58:26 -08:00
Konstantin Lebedev
fb97e234c9 skips compact if store is stopping 2022-02-07 20:16:15 +05:00
chrislu
9405eaefdb filer.sync: fix replicating partially updated file 
Run two servers with volumes and fillers:
server -dir=Server1alpha -master.port=11000 -filer -filer.port=11001 -volume.port=11002
server -dir=Server1sigma -master.port=11006 -filer -filer.port=11007 -volume.port=11008

Run Active-Passive filler.sync:
filer.sync -a localhost:11007 -b localhost:11001 -isActivePassive

Upload file to 11007 port:
curl -F file=@/Desktop/9.xml "http://localhost:11007/testFacebook/"

If we request a file on two servers now, everything will be correct, even if we add data to the file and upload it again:
curl "http://localhost:11007/testFacebook/9.xml"
EQUALS
curl "http://localhost:11001/testFacebook/9.xml"

However, if we change the already existing data in the file (for example, we change the first line in the file, reducing its length), then this file on the second server will not be valid and will not be equivalent to the first file

Снимок экрана 2022-02-07 в 14 21 11

This problem occurs on line 202 in the filer_sink.go file. In particular, this is due to incorrect mapping of chunk names in the DoMinusChunks function. The names of deletedChunks do not match the chunks of existingEntry.Chunks, since the first chunks come from another server and have a different addressing (name) compared to the addressing on the server where the file is being overwritten.

Deleted chunks are not actually deleted on the server to which the file is replicated.
 2022-02-07 03:46:28 -08:00
Radtoo
fbb14e0ea8 Adding separate toggle to purge absent vols 
While this toggle is basically required to clean out entries for deleted volumes, having a separate description + toggling this separately seems like a good idea so people get a chance to check if their volumes are all mounted/connected as expected.

Also renamed forcePurge to just purge.
 2022-02-06 23:46:52 +01:00
Radtoo
6b17f45da2 Also delete paths for which a volume is entirely absent. 2022-02-06 23:22:04 +01:00
Radtoo
fa0cfdfb7f Added basic deletion capabilities to fsck. 2022-02-06 17:05:58 +01:00
chrislu
433fde4b18 move error to a separate file 
This file contains metric names for all errors
The naming convention is ErrorSomeThing = "error.some.thing"
 2022-02-04 22:57:51 -08:00
Chris Lu
a23fcb9a7c
Merge pull request #2634 from kmlebedev/errorMetrics 
error metrics for filer and store
 2022-02-04 22:35:13 -08:00