mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-01-21 00:13:43 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,159 Commits 42 Branches 297 Tags 158 MiB
413cb3d389
Commit Graph

5344 Commits

Author SHA1 Message Date
Guo Lei
3df84cabf6
Merge branch 'chrislusf:master' into weed_update 2022-06-15 15:28:40 +08:00
guol-fnst
695e8c7555 fix minor bug 2022-06-15 15:26:04 +08:00
zhihao.qu
4d0d1848c6 fix(filer.sync): modify clientName format : from -> to 2022-06-15 13:33:20 +08:00
yulai.li
b90d213a87 Change ui.deleteDir default to true 2022-06-15 11:37:37 +08:00
zhihao.qu
42d04c581b feat(filer.sync): add metricsServer in filer.sync. 
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
 2022-06-15 11:33:18 +08:00
chrislu
d4ef06cdcf 3.11 2022-06-14 12:14:52 -07:00
chrislu
b347b2fb54 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-14 11:16:51 -07:00
chrislu
36d2756978 volume: avoid writing too much data for range requests in large files 
related to https://github.com/chrislusf/seaweedfs/issues/3178
 2022-06-14 11:16:47 -07:00
yulai.li
28cda854eb Add filer command line parameter to let Filer UI show/hide directory delete button 2022-06-15 00:30:49 +08:00
zhihao.qu
14d82c3dea feat(filer.sync): add offset to path. 2022-06-14 19:46:02 +08:00
guol-fnst
78fc9cdfa0 add "copied from https://github.com/restic/restic/tree/master/internal/selfupdate" 2022-06-14 19:10:54 +08:00
guol-fnst
322dbcf372 add update feature 2022-06-14 19:01:57 +08:00
Chris Lu
b22ca85fbb
Merge pull request #3169 from shichanglin5/fix_auth 
fix: When there is no access permission configured before startup, th…
 2022-06-12 23:40:42 -07:00
shichanglin5
f01dd27752 fix: When there is no access permission configured before startup, the authentication does not take effect after configuring the permission after startup 2022-06-13 13:23:26 +08:00
chrislu
e437ec5a68 3.10 2022-06-12 19:30:27 -07:00
chrislu
d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
chrislu
444ac21050 go fmt 2022-06-11 09:51:11 -07:00
chrislu
a74e926063 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-11 09:51:02 -07:00
chrislu
61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
Chris Lu
0a8b3948db
Merge pull request #3165 from blacktear23/reload-bug-fix 2022-06-10 07:47:43 -07:00
yulai.li
83f8cdd5cb Bug Fix: force reload filer page when upload/delete/rename/create directory 2022-06-10 22:39:21 +08:00
guol-fnst
eabf4d4e64 fix typo 2022-06-10 22:03:11 +08:00
Chris Lu
36757d28a8
Merge pull request #3163 from kmlebedev/metric_replica_placement_mismatch 
stats master_replica_placement_mismatch
 2022-06-10 06:44:16 -07:00
chrislu
9f8b72a54d Revert "Merge pull request #3159 from shichanglin5/_duplicateUUID" 
This reverts commit 37da689319, reversing
changes made to 00d53c34c4.
 2022-06-10 06:38:17 -07:00
Konstantin Lebedev
fb57aa431c stats master_replica_placement_mismatch 2022-06-10 15:30:40 +05:00
shichanglin5
a63559ee6c fix: fix command prompt information 2022-06-10 12:46:10 +08:00
Chris Lu
37da689319
Merge pull request #3159 from shichanglin5/_duplicateUUID 
perf: Optimized volume handling duplicateUUID logic to avoid quitting…
 2022-06-09 18:01:02 -07:00
shichanglin5
f5b0c04b14 perf: Optimized volume handling duplicateUUID logic to avoid quitting when volume is actualy normal 
Under normal circumstances, there will be no problems, but when the
master is debugged in the local environment, the volume client cannot
communicate with the master normally, so the sendHeartBeat logic is
restarted, and a new connection is created to report the heartbeat. If
the master has not cleared the uuid of the volume at this time, then The
master will respond to volume duplicateUUIDS, and the volume service
will exit, but in fact the uuid of the volume is not duplicated
 2022-06-09 20:41:16 +08:00
zhihao.qu
cd5cca36a4 feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges 2022-06-09 10:53:19 +08:00
Chris Lu
4a5135961f
Merge pull request #3154 from kmlebedev/s3_test_invalid_range 2022-06-08 01:45:40 -07:00
Konstantin Lebedev
49f058da85 fix s3 tests: 
ranged_request_invalid_range
ranged_request_empty_object
 2022-06-08 13:25:13 +05:00
chrislu
8a1166c021 auto remove previous leftover socket 2022-06-08 00:32:09 -07:00
chrislu
633a4aed18 filer: fix customizable local socket file name 
fix https://github.com/chrislusf/seaweedfs/issues/3147
 2022-06-08 00:15:28 -07:00
chrislu
acbc0286e2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-07 22:33:38 -07:00
chrislu
ebd6097e6f fix: sequence read rate is slow when use filer.ChunkReadAt.ReadAt #3074 
related to https://github.com/chrislusf/seaweedfs/pull/3074
 2022-06-07 22:33:35 -07:00
Chris Lu
f43ec9f363
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud 2022-06-07 08:56:55 -07:00
Konstantin Lebedev
c07820178f fix s3 tests 
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
 2022-06-07 14:43:10 +05:00
chrislu
78be3b0f6c fix compilation 2022-06-06 10:09:21 -07:00
chrislu
4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
chrislu
cbf46de5f4 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-06 00:15:40 -07:00
chrislu
8402af0882 filer: detect ttl based on fs configure setting 2022-06-06 00:15:37 -07:00
ningfd
f32142f6f5 add disableXAttr in mount option 2022-06-06 14:09:01 +08:00
chrislu
4a046e4de7 3.09 2022-06-05 19:00:24 -07:00
chrislu
11f1e7996b fix test 2022-06-05 18:50:41 -07:00
chrislu
d65bb2c6df mount: file handle locks entry better 
related to https://github.com/chrislusf/seaweedfs/issues/2952
 2022-06-05 18:15:06 -07:00
chrislu
746092a60b add a comment 2022-06-05 16:44:07 -07:00
chrislu
056c480eb0 volume: checksum remove the hashing step in Value() 2022-06-05 15:24:02 -07:00
chrislu
ecef844dfc stream read large files 2022-06-05 11:54:04 -07:00
chrislu
48cca4e54f remove unused logic 2022-06-04 19:11:06 -07:00
chrislu
685643d506 needle read into a writer 2022-06-04 19:10:52 -07:00
chrislu
663bc5dc23 add needle reading in chunks 2022-06-04 18:15:39 -07:00
chrislu
18b29f709c refactoring, split file 2022-06-03 23:31:14 -07:00
chrislu
39c3cb04af Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-03 08:45:32 -07:00
chrislu
f97acdd489 volume.fix.replication fix retry logic 
fix https://github.com/chrislusf/seaweedfs/issues/3136
 2022-06-03 08:45:29 -07:00
creeew
02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
Konstantin Lebedev
b86628f85d fix show empty dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-06-01 20:10:52 +05:00
chrislu
6793bc853c help message when in simulation mode 2022-05-31 14:48:46 -07:00
chrislu
89948a373b fix error reporting for "Need to a hdd disk type" 
related to https://github.com/chrislusf/seaweedfs/issues/3128
 2022-05-31 12:43:55 -07:00
chrislu
8a49240d64 3.08 2022-05-31 11:57:41 -07:00
chrislu
376b005ad3 fix wrong logic 2022-05-31 11:55:58 -07:00
chrislu
fbd99d53c1 3.07 2022-05-31 00:36:24 -07:00
chrislu
27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu
f4a6da6cb2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 22:47:29 -07:00
chrislu
596c3860ca use final destination to resolve fs configuration 
related to https://github.com/chrislusf/seaweedfs/issues/3075
 2022-05-30 22:47:26 -07:00
Chris Lu
ca01ce0524
Merge pull request #3122 from chrislusf/filer-sync-with-peers 
Filer bootstrap from peers
 2022-05-30 21:38:31 -07:00
chrislu
6adc42147f fresh filer store bootstrap from the oldest peer 2022-05-30 21:27:48 -07:00
chrislu
490e0850bf filer.meta.tail add one example of using untilTimeAgo 2022-05-30 20:24:59 -07:00
liubaojiang
f0ee3e6f21 reduce the scope of inFlightUploadDataLimitCond lock 2022-05-31 09:49:07 +08:00
chrislu
c59068d0f3 refactor 2022-05-30 16:28:36 -07:00
chrislu
682382648e collect cluster node start time 2022-05-30 16:23:52 -07:00
chrislu
968ca95b49 filer.meta.tail: support untilTimeAgo for a range 2022-05-30 16:16:23 -07:00
chrislu
f214dfb1f5 stop when in memory log is done 2022-05-30 15:25:21 -07:00
chrislu
aece35a64f stop when on disk log is done 2022-05-30 15:20:51 -07:00
chrislu
a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu
8902fa6ff6 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 13:28:32 -07:00
Chris Lu
730c9cf601
Merge pull request #3114 from kmlebedev/nextcloud 
Create folders in s3 via nextcloud
 2022-05-30 13:10:10 -07:00
chrislu
5b9347c938 typo 2022-05-30 12:37:41 -07:00
Konstantin Lebedev
af562e1a20 rm log info 2022-05-30 23:26:26 +05:00
Konstantin Lebedev
62e5e3822c list self dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-05-30 18:09:08 +05:00
Konstantin Lebedev
26f3646961 show fake dirs 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-05-30 15:03:27 +05:00
chrislu
b8f3db0d46 s3: keep auth enabled in case identities are set to empty 
fix https://github.com/chrislusf/seaweedfs/issues/3084
 2022-05-29 19:00:03 -07:00
chrislu
81d6159290 volume: report error if a volume has nil data backend 
fix https://github.com/chrislusf/seaweedfs/issues/3105
 2022-05-29 16:59:30 -07:00
chrislu
ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
chrislu
9219664505 reset sync offset if peer filer resets 
fix https://github.com/chrislusf/seaweedfs/issues/3089
 2022-05-29 13:11:24 -07:00
Konstantin Lebedev
54356211c2 list empty folder 2022-05-27 13:17:18 +05:00
Kaiwalya Joshi
4598581741
fix: Minor spelling fixes on FilerStorePathTranslator. 2022-05-25 01:11:02 -07:00
chrislu
5b8b022985 remove unused parameter 2022-05-24 18:52:04 -07:00
Robert Coelho
0e6e72d462 mount: ReadDir return EIO on EnsureVisited err 2022-05-24 13:02:59 -04:00
Robert Coelho
1fabbe8a25 mount: cleanup ReadDir rewrite's branches to not assume offsets 2022-05-24 12:48:32 -04:00
Robert Coelho
cb422d96f7 mount: rewrite ReadDir to respect input.Offset to fix partial results 2022-05-24 12:16:58 -04:00
Robert Coelho
05602167f4 mount: do not mark meta_cache when there is an err 2022-05-24 12:13:15 -04:00
chrislu
7e25a2d416 reduce busy waiting when reading metadata logs 2022-05-24 00:23:53 -07:00
chrislu
2f846777bb 3.06 2022-05-23 01:21:48 -07:00
chrislu
a5cdcc7e63 fix reading files larger than 20GB due to manifest resolving bug 2022-05-23 01:16:10 -07:00
chrislu
59cc3c0bfd fix reading files larger than 20GB due to manifest resolving bug 
this is introduced since Mar 08 2022
 2022-05-23 01:14:56 -07:00
chrislu
acc318e12b 3.05 2022-05-22 21:43:59 -07:00
liubaojiang
076e48a676 add inflight upload data wait timeout 2022-05-21 10:38:08 +08:00
liubaojiang
71b2e6223e add inFlightUploadDataLimitCond signal 2022-05-20 15:27:06 +08:00
liubaojiang
1a41691b4c exclude replication from the concurrentUploadLimitMB 2022-05-20 14:40:05 +08:00
Chris Lu
cb64983c6b
Merge pull request #3079 from ztinpn/hotfix_fsync 2022-05-19 10:33:42 -07:00
tianzhang
66747ee9c9 hotfix_fsync fix fsync 2022-05-19 23:17:17 +08:00
Chris Lu
8efe55f981
Merge pull request #3059 from guol-fnst/avoid_dup_vol 
avoid loading duplicated volume directory
 2022-05-17 23:48:57 -07:00
guol-fnst
8e2426a879 revert changes of pb file 
orignal protoc-gen-go-grpc verion is 1.1.0
 2022-05-18 11:52:14 +08:00
guol-fnst
b12944f9c6 fix naming convention 
notify volume server of duplicate directoris
improve searching efficiency
 2022-05-17 15:41:49 +08:00
guol-fnst
076595fbdd just exit in case of duplicated volume directories were loaded 2022-05-17 15:41:49 +08:00
guol-fnst
8fab39e775 rename UUID file 
fix typo
move locationUUID  into DiskLocation
 2022-05-17 15:41:47 +08:00
Chris Lu
72e7dcde51
Merge pull request #3060 from natmaka/master 2022-05-16 07:42:58 -07:00
Nat Makarevitch
b72f0634d7
minor (typos...), done while reading around 2022-05-16 22:11:33 +08:00
guol-fnst
de6aa9cce8 avoid duplicated volume directory 2022-05-16 19:33:51 +08:00
Konstantin Lebedev
44f53ceda6 fix collectionIsMismatch charset 2022-05-16 13:23:23 +05:00
Konstantin Lebedev
10d435f2c2 fix skip loop 2022-05-16 13:16:27 +05:00
Konstantin Lebedev
279053572c avoid delete volume replica if collection mismatch 2022-05-16 13:07:05 +05:00
chrislu
9ff0d99002 3.04 2022-05-15 21:32:21 -07:00
chrislu
866fed1eab avoid grpc name conflict 
fix https://github.com/chrislusf/seaweedfs/issues/3055
 2022-05-15 21:31:29 -07:00
chrislu
cd2d10118c 3.03 2022-05-15 18:17:07 -07:00
chrislu
271b5aed96 shell: volume.fsck add a note for -reallyDeleteFromVolume option 2022-05-15 11:07:04 -07:00
chrislu
4076d74813 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-15 00:43:39 -07:00
chrislu
29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00
Chris Lu
ea4918394c
Merge pull request #3047 from shichanglin5/master 
fix the problem of metadata and tagging loss when files are copied
 2022-05-13 22:52:48 -07:00
shichanglin5
688d55488c test(s3api_object_copy_handlers_test.go): some unit tests have been added to the processMetadata & processMetadataBytes methods of s3api_object_copy_handlers.go 2022-05-14 10:40:29 +08:00
chrislu
735038b2c1 backup do not need to use preallocation 
fix https://github.com/chrislusf/seaweedfs/issues/3044
 2022-05-13 13:46:52 -07:00
shichanglin5
1166dead00 fix the problem of metadata and tagging loss when files are copied 
by adding processing of metadata and tagging in s3 api CopyObject (judging whether to copy or overwrite according to the directive header)
 2022-05-13 19:46:20 +08:00
Konstantin Lebedev
d12a423aa0 refactor 
add some tests

https://github.com/chrislusf/seaweedfs/pull/2996
 2022-05-12 12:23:02 +05:00
Konstantin Lebedev
f523aed3c9 fixes from comments 
https://github.com/chrislusf/seaweedfs/pull/2996
 2022-05-11 21:50:09 +05:00
chrislu
bff1ccc1de fix compilation 2022-05-11 00:52:15 -07:00
chrislu
76da688176 adjust log level 2022-05-11 00:45:19 -07:00
chrislu
3e20336bde fix atomic add int64 
fix https://github.com/chrislusf/seaweedfs/issues/3038#issuecomment-1123269831
 2022-05-11 00:44:35 -07:00
chrislu
a85720f0f6 remove deprecated function 2022-05-10 00:02:23 -07:00
chrislu
25559eeb92 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-10 00:01:46 -07:00
chrislu
791c29ae93 remove deprecated function 
WithQueryCachePolicyKeepInCache is deprecated and already default behavior
 2022-05-10 00:01:43 -07:00
qiang.zeng
06cb337b82 fix typo 2022-05-09 14:21:16 +08:00
chrislu
a4ca3ed1f0 3.02 2022-05-08 22:28:32 -07:00
chrislu
8bc16f1149 rename 2022-05-06 23:38:22 -07:00
chrislu
139e039c44 filer.sync: pass attributes for mount 
fix https://github.com/chrislusf/seaweedfs/issues/3012
 2022-05-06 03:54:12 -07:00
chrislu
10207a48a4 how to setup local test env 2022-05-06 02:25:56 -07:00
chrislu
db33b03735 add comments 2022-05-06 02:19:47 -07:00
chrislu
e350ba8896 larger test range 2022-05-06 02:17:28 -07:00
chrislu
71e0bcc5d5 a bit optimization 2022-05-06 02:11:58 -07:00
chrislu
b2a43dd044 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-06 01:51:28 -07:00
chrislu
d5fe5d12b7 filer: etcd fix pagination 
fix https://github.com/chrislusf/seaweedfs/issues/3016
 2022-05-06 01:51:19 -07:00
Chris Lu
a69bf53405
Merge pull request #2996 from kmlebedev/ydb 
ydb filer store
 2022-05-05 15:34:37 -07:00
Konstantin Lebedev
e6d2cb5911 ydb add empty doc file 2022-05-05 11:12:14 +05:00
chrislu
ef6c6c450e avoid fatal error if port is already in use 2022-05-04 14:55:14 -07:00
chrislu
f7366a9668 skip unix socket mode for windows 
fix https://github.com/chrislusf/seaweedfs/issues/3013
 2022-05-04 10:14:34 -07:00
Konstantin Lebedev
f58adaab25 build tag ydb since the growth of the binary is 5 mb 2022-05-03 23:20:09 +05:00
Konstantin Lebedev
f127b326bf add options to scaffold 2022-05-03 22:54:31 +05:00
Konstantin Lebedev
7b544576af refactor 2022-05-03 20:16:00 +05:00
chrislu
f17cd0d5cd return false if not found 
fix https://github.com/chrislusf/seaweedfs/issues/3011
 2022-05-03 07:18:34 -07:00
chrislu
b201edb9df fix wrong assignment 2022-05-03 07:12:33 -07:00
Konstantin Lebedev
04d5dff6bb add support native ttl 2022-05-03 17:52:23 +05:00