Patrick Schmidt
5df105b1f9
Fix a few data races when reading files in mount ( #3527 )
2022-08-26 16:41:37 -07:00
chrislu
f5156cf3a8
rename varaible
2022-08-26 10:23:42 -07:00
Konstantin Lebedev
e90ab4ac60
avoid race conditions for OnPeerUpdate ( #3525 )
...
https://github.com/seaweedfs/seaweedfs/issues/3524
2022-08-26 10:18:49 -07:00
Konstantin Lebedev
4f7a1f67cd
avoid race conditions for diskLocation.MaxVolumeCount ( #3526 )
2022-08-26 08:41:42 -07:00
chrislu
6f483a4d36
atomic add
...
fix https://github.com/seaweedfs/seaweedfs/issues/3514
2022-08-25 22:20:34 -07:00
chrislu
7394f7feee
minor: adjust ordering
2022-08-25 00:20:23 -07:00
chrislu
dbf0de4ce1
minor clean up
2022-08-25 00:19:08 -07:00
Patrick Schmidt
2930263dfd
Fix race conditions during in-flight size checks ( #3505 )
2022-08-24 20:03:34 -07:00
Patrick Schmidt
7b424a54dc
Add raft server access mutex to avoid races ( #3503 )
2022-08-24 09:49:05 -07:00
Konstantin Lebedev
f7aeb06544
s3: report metadata if the directory is explicitly created ( #3498 )
...
* replace mkdir to mkFile
* ContentLength must be zero
* revert mkDir
* Seaweedfs-Is-Directory-Key return metadata
2022-08-24 00:15:44 -07:00
Guo Lei
c57c79a0ab
optimiz commitig compact ( #3388 )
...
* optimiz vacuuming volume
* fix bugx
* rename parameters
* fix conflict
* change copyDataBasedOnIndexFile to an instance method
* close needlemap
* optimiz commiting Vacuum volume for leveldb index
* fix bugs
* fix leveldb loading bugs
* refactor
* fix leveldb loading bug
* add leveldb recovery
* add test case for levelDB
* modify test case to cover all the new branches
* use one tmpNm instead of two instances
* refactor
* refactor
* move setWatermark to the end
* add test for watermark and updating leveldb
* fix error logic
* refactor, add test
* check nil before close needlemapeer
add test case
fix metric bug
* add tests, fix bugs
* adjust log level
remove wrong test case
refactor
* avoid duplicate updating metric for leveldb index
2022-08-23 23:53:35 -07:00
chrislu
10414fd81c
ping timeout at 15 seconds
...
this 72 minute timeout setting seems unreasonably long
15 seconds is around the time when a new raft leader should be elected.
2022-08-23 23:28:16 -07:00
askeipx
2e78a522ab
remove old raft servers if they don't answer to pings for too long ( #3398 )
...
* remove old raft servers if they don't answer to pings for too long
add ping durations as options
rename ping fields
fix some todos
get masters through masterclient
raft remove server from leader
use raft servers to ping them
CheckMastersAlive for hashicorp raft only
* prepare blocking ping
* pass waitForReady as param
* pass waitForReady through all functions
* waitForReady works
* refactor
* remove unneeded params
* rollback unneeded changes
* fix
2022-08-23 23:18:21 -07:00
Guo Lei
762dc219a6
fix deletecount bug ( #3499 )
2022-08-23 08:53:24 -07:00
chrislu
21a933c932
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-23 01:52:32 -07:00
chrislu
f0b4a7659a
fix test
2022-08-23 01:52:29 -07:00
Patrick Schmidt
cda8cc22bc
Implement lseek syscall in FUSE ( #3491 )
...
See the man page of lseek:
https://man7.org/linux/man-pages/man2/lseek.2.html
2022-08-23 01:47:18 -07:00
chrislu
bf88006037
format
2022-08-23 01:20:45 -07:00
chrislu
9fce75607d
s3: report http.StatusOK if the directory is explicitly created
...
fix https://github.com/seaweedfs/seaweedfs/issues/3457
2022-08-23 01:16:46 -07:00
famosss
911475526c
fix: TestCommandEcBalanceSmall Unit test fails when CommandEnv is nil ( #3497 )
2022-08-22 23:54:51 -07:00
famosss
7eb15b1969
fix:Handle preflight cors requests ( #3496 )
2022-08-22 23:39:34 -07:00
chrislu
676e27c589
shell: stop long running jobs if lock is lost
2022-08-22 14:12:23 -07:00
chrislu
601ba5fb68
master: disable locking if not leader
2022-08-22 14:11:48 -07:00
chrislu
57e7582c36
refactoring
2022-08-22 14:11:13 -07:00
chrislu
7ea2c24c5d
metadata processing use retry instead of retryForEver
2022-08-22 11:25:40 -07:00
Patrick Schmidt
f875031f06
Reuse readDataByFileHandle in Read call ( #3482 )
2022-08-22 08:24:06 -07:00
famosss
bdba3da2e4
Handle preflight cors requests ( #3481 )
2022-08-22 08:21:38 -07:00
chrislu
f4b52d4c52
fix format
2022-08-22 00:20:21 -07:00
chrislu
b4131a3a72
filer.remote.sync: parallelize uploading to remove object storage
2022-08-22 00:01:48 -07:00
chrislu
f6b92b8561
refactor
2022-08-22 00:00:51 -07:00
chrislu
3c8e95eeb3
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-21 23:26:09 -07:00
chrislu
8bac903d29
mount: add locking to prevent nil pointer
...
fix https://github.com/seaweedfs/seaweedfs/issues/3479
2022-08-21 23:26:06 -07:00
qzh
74b53729e1
feat(weed.move): add a speed limit parameter of moving files ( #3478 )
...
* feat(weed.move): add a speed limit parameter of moving files
* fix(weed.move): set the default value of ioBytePerSecond to vs.compactionBytePerSecond
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-21 23:08:31 -07:00
chrislu
7807f6641a
minor
2022-08-21 22:54:56 -07:00
chrislu
c4e862e908
3.23
2022-08-21 19:13:26 -07:00
Guo Lei
81e7e6ceb3
the statistical methord for fileCount should stay same ( #3477 )
...
the statistical methord for fileCount between should stay same
They all equal to the entry count in .idx file.
relate to commit(c7892bc
2022-08-21 19:11:18 -07:00
chrislu
63fbf281c7
remove same file copying rage limitation
2022-08-21 17:16:21 -07:00
Patrick Schmidt
3a75d7f7aa
Implement copy_file_range syscall in FUSE ( #3475 )
...
See the man page of copy_file_range:
https://man7.org/linux/man-pages/man2/copy_file_range.2.html
2022-08-21 17:11:43 -07:00
chrislu
fef9c6a520
filer: do not always save files in "/etc" folder to filer store
...
fix https://github.com/seaweedfs/seaweedfs/issues/3476
2022-08-21 17:02:16 -07:00
chrislu
2762154130
fix compilation
2022-08-21 16:09:23 -07:00
chrislu
fb5808e0c3
EC: with multiple volume locations, the ec rebuilding may fail
2022-08-21 15:04:50 -07:00
chrislu
5790d01c6f
EC: after ec encoding, the source ec shards may fail to purge if the volume server has multiple disk locations
...
related to https://github.com/seaweedfs/seaweedfs/issues/3459
2022-08-21 14:52:37 -07:00
chrislu
8b3429858d
align memory for atomic read/write
2022-08-21 12:20:27 -07:00
Patrick Schmidt
2ef6ab998c
Avoid race conditions with current filer address ( #3474 )
...
When multiple filer requests are in-flight and the current filer
disappears and a new one is selected by the first goroutine, then
there can be a lot of race conditions while retrieving the current
filer.
Therefore, load/save the current filer index atomically.
2022-08-21 12:18:13 -07:00
Patrick Schmidt
f49a9297c2
Fix hanging reads in chunk cacher ( #3473 )
...
Sometimes when an unexpected error occurs the cacher would set an
error and return. However, it would not broadcast the condition
signal in that case, therefore leaving the goroutine that runs
readChunkAt stuck forever.
I figured that the condition is unnecessary because readChunkAt is
acquiring a lock that is still held by the cacher goroutine anyway.
Callees of startCaching have to wait for a WaitGroup which makes sure
that readChunkAt can't acquire the lock before startCaching.
This way readChunkAt can execute normally and check for the error.
2022-08-21 11:54:02 -07:00
chrislu
388f82f322
minor
2022-08-21 11:49:29 -07:00
chrislu
77e4b1376e
refactoring
2022-08-21 11:35:54 -07:00
Patrick Schmidt
3f758820c1
Fix FUSE server buffer leaks in file gaps ( #3472 )
...
* Fix FUSE server buffer leaks in file gaps
This change zeros read buffers when encountering file gaps during
file/chunk reads in FUSE mounts.
It prevents leaking internal buffers of the FUSE server which could
otherwise reveal metadata, directory listings, file contents and
other data related to FUSE API calls.
The issue was that buffers are reused, but when a file gap was found
the buffer was not zeroed accordingly and the existing data of the
buffer was kept and returned.
* Move zero logic into its own method
2022-08-21 11:33:58 -07:00
chrislu
c7892bc7c4
volume: file counter should be all files
...
address https://github.com/seaweedfs/seaweedfs/pull/3388#issuecomment-1220466228
2022-08-20 23:35:31 -07:00
chrislu
f7e0a65e75
retry for all errors
2022-08-20 23:34:45 -07:00
chrislu
649555b223
adjust log
2022-08-20 23:34:07 -07:00
chrislu
973f6dd162
refactoring
2022-08-20 22:38:15 -07:00
chrislu
65ff7198fe
adjust type
2022-08-20 22:37:24 -07:00
chrislu
28b862f45f
use util.Retry to retry
2022-08-20 22:03:27 -07:00
chrislu
3bf8e772f8
webdav: retryable data chunk upload
2022-08-20 19:18:12 -07:00
chrislu
f8fa430257
filer: retryable data chunk upload
2022-08-20 19:15:44 -07:00
chrislu
4081d50607
filer sink: retryable data chunk uploading
2022-08-20 19:09:15 -07:00
chrislu
409f39390d
mount: retryable data chunk uploading
2022-08-20 19:04:36 -07:00
chrislu
6c8822f269
filer.copy: retryable file part upload
2022-08-20 18:59:57 -07:00
chrislu
a3553da7f7
add fileId to UploadWithRetry return result
2022-08-20 18:54:59 -07:00
chrislu
d49d0a9fc2
filer.copy: retryable upload
2022-08-20 18:50:57 -07:00
chrislu
689b4ecdcc
remove unused collection and replication from upload result
2022-08-20 18:14:57 -07:00
chrislu
e3f40d538d
cleaner code
2022-08-20 17:51:30 -07:00
chrislu
9f50b310a0
remove unused code
2022-08-20 17:51:12 -07:00
chrislu
11f99836c3
filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0)
...
fix https://github.com/seaweedfs/seaweedfs/issues/3468
2022-08-19 23:00:56 -07:00
chrislu
fdd8c5d5e0
Filer if-modified-since check doesn't fire on "exact" #3467
...
fix https://github.com/seaweedfs/seaweedfs/issues/3467
2022-08-19 22:21:51 -07:00
chrislu
aac45f3e89
filer: retryable when error is not found
2022-08-19 03:36:15 -07:00
chrislu
58dd880322
minor
2022-08-18 23:51:08 -07:00
Eric Yang
b6f99c4d83
fix naming typo ( #3465 )
...
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-08-18 23:25:32 -07:00
chrislu
5702727db5
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-18 02:39:27 -07:00
chrislu
4d72a1ce6e
s3: remove deleted collection during metrics reporting
...
fix https://github.com/seaweedfs/seaweedfs/issues/3453
2022-08-18 02:39:07 -07:00
Guo Lei
3172c33fa6
fix redundant type from array ( #3462 )
2022-08-18 02:25:25 -07:00
chrislu
42c6e52513
s3: fix regression on HEAD directory operation
2022-08-18 02:13:58 -07:00
chrislu
4573c99ae3
fix tests
2022-08-18 00:15:46 -07:00
chrislu
2b580a7566
also migrate jsonpb
2022-08-17 12:42:03 -07:00
chrislu
eaeb141b09
move proto package
2022-08-17 12:05:07 -07:00
chrislu
e259052a70
volume: fix metrics for volume request time
2022-08-17 04:29:31 -07:00
qzh
7fcfaf7bc9
fix(filer.sync): offset may be set to 0 ( #3451 )
...
* fix(filer.sync): initializing the offset is related to the path
* fix(filer.sync): the offset maybe to be set to 0.
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 23:43:52 -07:00
qzh
400f0c3e5d
fix(filer.sync): initializing the offset is related to the path ( #3450 )
...
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 21:56:47 -07:00
chrislu
fa4d0093e1
3.22
2022-08-15 16:48:23 -07:00
chrislu
93261f5199
Revert "Refactor for Sync method ( #3426 )"
...
This reverts commit 670cb759f8
2022-08-15 15:35:31 -07:00
chrislu
9c588d4010
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-15 10:03:56 -07:00
chrislu
96caf21d09
less verbose log
2022-08-15 10:03:52 -07:00
Andrey Triumfov
31faa6d43d
Remove duplicate slashes in object path to prevent 500 errors ( #3442 )
2022-08-15 08:19:28 -07:00
chrislu
cb476a53ff
remove logs
2022-08-15 01:05:35 -07:00
chrislu
7c029b2183
3.21
2022-08-15 00:32:15 -07:00
Chris Lu
e88392b50f
Fix s3 pagination ( #3436 )
...
* Revert previous changes
* s3: use cursor to track tree traversal
fix https://github.com/seaweedfs/seaweedfs/issues/3166
* special cases for empty prefix and empty directory
* use constants
* address empty folder
* undo local changes
* fix IsTruncated
* adjust counting directories
* fix cases when prefix is a directory
* s3: handle directory object
works for
aws --endpoint-url http://127.0.0.1:8333/ s3api list-objects-v2 --bucket test --prefix "fakedir"
2022-08-15 00:30:19 -07:00
chrislu
7457c746f0
s3: fix aws s3api head-object
2022-08-14 23:52:35 -07:00
chrislu
789d9dbfe1
webdav: fix nil
...
fix https://github.com/seaweedfs/seaweedfs/issues/3440
2022-08-14 23:33:30 -07:00
Konstantin Lebedev
6199d9b923
rand start garbageThreshold and reset metric ReplicaPlacementMismatch ( #3396 )
...
* rand start garbageThreshold
reset metrics MasterReplicaPlacementMismatch
* vacuum through sleep
2022-08-12 13:11:09 -07:00
Konstantin Lebedev
4d4cd0948d
avoid infinite loop WaitUntilConnected() ( #3431 )
...
https://github.com/seaweedfs/seaweedfs/issues/3421
2022-08-11 15:03:26 -07:00
Eric Yang
986daecbfe
add upload response ( #3428 )
...
* add upload response
* fixing comments
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-08-10 22:43:09 -07:00
chrislu
ae93c966d9
ensure memory is aligned
...
fix https://github.com/seaweedfs/seaweedfs/issues/3427
2022-08-10 22:27:13 -07:00
Rain Li
670cb759f8
Refactor for Sync method ( #3426 )
2022-08-10 08:21:57 -07:00
chrislu
662ec97602
mount: adjust on du stats reporting
...
fix https://github.com/seaweedfs/seaweedfs/issues/3424
2022-08-09 20:21:01 -07:00
Abirdcfly
b0633716b7
delete minor unreachable code ( #3423 )
...
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-09 01:24:43 -07:00
chrislu
0854171d22
3.20
2022-08-07 14:42:28 -07:00
chrislu
d8e69df295
adjust comments
2022-08-07 11:35:33 -07:00
chrislu
0aeec04c31
quicker to adapt to pattern change
2022-08-07 10:14:01 -07:00
chrislu
928d29af9e
fix wrong logic about reader isRandomMode()
2022-08-07 10:13:04 -07:00
chrislu
d3d52b1818
shell: fs.meta.load add quieter mode
...
fix https://github.com/seaweedfs/seaweedfs/issues/3414
2022-08-07 01:42:15 -07:00
chrislu
358ccb5e0e
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-07 01:34:36 -07:00
chrislu
67814a5c79
refactor and fix strings.Split
2022-08-07 01:34:32 -07:00
John W Higgins
3afda0c89c
Allow postgresql to use standard environment variables for connection ( #3413 )
2022-08-07 00:58:53 -07:00
chrislu
1a4bf0dcb5
filer.sync: parallelize the filer.sync
2022-08-07 00:56:15 -07:00
chrislu
02b38f894b
debug
2022-08-07 00:55:49 -07:00
chrislu
0e9478488d
filer.sync: fix when excluded paths is empty
2022-08-07 00:55:34 -07:00
chrislu
8a880a139d
mount: fix truncate operation
...
fix https://github.com/seaweedfs/seaweedfs/issues/2609
2022-08-06 13:58:45 -07:00
LHHDZ
84ec68e11a
Add download speed limit support ( #3408 )
2022-08-05 01:16:42 -07:00
chrislu
b278bb24d3
mount: adjust df stats reporting when close to the limit
...
fix https://github.com/seaweedfs/seaweedfs/issues/3407
2022-08-05 00:31:41 -07:00
Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center ( #3405 )
...
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404
* GetDataCenter
* prefer same data center for ReplicationSource
* GetDataCenterId
* remove glog
2022-08-04 17:35:00 -07:00
Konstantin Lebedev
22181dd018
refactor FilerRequest metrics ( #3402 )
...
* refactor FilerRequest metrics
* avoid double count proxy
* defer to
2022-08-04 01:44:54 -07:00
chrislu
bd13a7968f
remove unused symlink resolving
2022-08-04 01:35:18 -07:00
chrislu
88945d9954
mount: do not follow soft link with xattr
2022-08-04 01:21:15 -07:00
chrislu
90db4b5a44
minor
2022-08-03 00:10:47 -07:00
Chris Lu
fc8035d672
Merge pull request #3394 from kmlebedev/metricsReplicatedWrite
...
Detailed metrics VolumeServerRequestHistogram for writing to disk and replication
2022-08-01 13:31:01 -07:00
Konstantin Lebedev
3ffa4ba91d
clean
2022-08-02 01:30:38 +05:00
Konstantin Lebedev
2a5629dd28
align
2022-08-02 01:29:44 +05:00
Konstantin Lebedev
df9cc31a0a
refactor
2022-08-02 01:18:26 +05:00
Konstantin Lebedev
0a5d3c3cf6
rename to localWrite and replicatedWrite
2022-08-01 23:09:09 +05:00
chrislu
bd5c9904be
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-08-01 09:40:05 -07:00
chrislu
e26d1cf4ef
fix test
2022-08-01 09:40:01 -07:00
Konstantin Lebedev
fc65122766
rename to LoadAvg_1M
2022-08-01 21:32:21 +05:00
Konstantin Lebedev
ab50f21ad9
Detailed metrics VolumeServerRequestHistogram for writing to disk and replication
2022-08-01 21:16:57 +05:00
Konstantin Lebedev
2f85e31f5c
Merge branch 'master' into gentle_vacuum
2022-08-01 20:51:58 +05:00
Konstantin Lebedev
5209ebbeef
remove percent
2022-08-01 20:40:38 +05:00
Chris Lu
b59bc607bf
Merge pull request #3338 from kmlebedev/issues/3083
...
rollback over onPeerUpdate implementation of automatic clean-up of failed servers in favor of synchronous ping
2022-08-01 08:23:10 -07:00
Konstantin Lebedev
78cbd8002f
revert Sleep
2022-08-01 20:21:23 +05:00
Konstantin Lebedev
cd5c7ad052
move to github.com/seaweedfs/seaweedfs
2022-08-01 16:36:32 +05:00
Konstantin Lebedev
634bc97840
fix conflicts
2022-08-01 14:47:00 +05:00
Konstantin Lebedev
3c75479e2b
Merge branch 'master' into gentle_vacuum
...
# Conflicts:
# weed/pb/messaging_pb/messaging.pb.go
# weed/pb/messaging_pb/messaging_grpc.pb.go
# weed/pb/s3_pb/s3.pb.go
# weed/pb/volume_server_pb/volume_server.pb.go
# weed/server/volume_grpc_vacuum.go
2022-08-01 14:45:22 +05:00
Konstantin Lebedev
c0d92f61a1
comment
2022-08-01 14:40:42 +05:00
Konstantin Lebedev
1d29f67c02
revert disk stats
2022-08-01 14:29:41 +05:00
Konstantin Lebedev
a98f6d66a3
rollback over onPeerupdate implementation of automatic clean-up of failed servers in favor of synchronous ping
2022-08-01 12:51:41 +05:00
chrislu
036566629a
filer.sync: fix synchronization logic in active-active mode
...
fix https://github.com/seaweedfs/seaweedfs/issues/3328
2022-08-01 00:06:18 -07:00
chrislu
303bd067b5
Revert "rename: delete source entry metadata only, skipping hard links"
...
This reverts commit 03466f955ehttps://github.com/seaweedfs/seaweedfs/issues/3386
2022-07-31 22:51:41 -07:00
chrislu
03c6f978f4
3.19
2022-07-31 17:16:51 -07:00
chrislu
afef014832
scaffold
2022-07-31 13:23:44 -07:00
Chris Lu
0bb3ba17ae
Merge pull request #3385 from Woellchen/feature/cluster-status-return-healthy
...
Add healthy indicator for raft status
2022-07-30 11:55:30 -07:00
Patrick Schmidt
1a4a36d510
Add healthy indicator for raft status
2022-07-30 19:34:26 +02:00
chrislu
cb096d1f53
minor
2022-07-30 09:52:52 -07:00
chrislu
0423d50106
wip note
2022-07-30 09:36:27 -07:00
chrislu
0f50fcb96a
moving
2022-07-29 23:39:11 -07:00
chrislu
13b9a52f80
fix deprecated functions
2022-07-29 01:34:39 -07:00
chrislu
be0ec7bc2f
fix test
2022-07-29 01:15:26 -07:00
chrislu
1f161f53dc
add back rack
2022-07-29 01:15:19 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
Konstantin Lebedev
2f0dda384d
vacuum show LA
2022-07-29 11:59:33 +05:00
chrislu
de3afd4199
fix rebasing
2022-07-28 23:25:46 -07:00
chrislu
06cd491abc
add message batch
2022-07-28 23:24:38 -07:00
chrislu
7576c244c4
add flatbuffer serde for message
2022-07-28 23:24:38 -07:00
chrislu
a3fae7cd16
add an empty placeholder for mq.topic.list
2022-07-28 23:24:38 -07:00
chrislu
aa4a22ad47
segment serde
2022-07-28 23:24:38 -07:00
chrislu
74f60f246f
dynamically connect to a filer
2022-07-28 23:24:38 -07:00
chrislu
9f479aab98
allocate brokers to serve segments
2022-07-28 23:24:38 -07:00
chrislu
bb01b68fa0
refactor
2022-07-28 23:24:38 -07:00
chrislu
94b8c42b2c
clean up
2022-07-28 23:24:38 -07:00
chrislu
8060fdcac5
remove old code
2022-07-28 23:24:38 -07:00
chrislu
f25e273e32
display data center and rack in cluster.ps
2022-07-28 23:22:52 -07:00
chrislu
68065128b8
add dc and rack
2022-07-28 23:22:51 -07:00
chrislu
8d31e73ffd
mq broker adds data center and rack
2022-07-28 23:22:07 -07:00
chrislu
87ccef72db
cluster.ps can list brokers
2022-07-28 23:22:07 -07:00
chrislu
ac66227798
renaming
2022-07-28 23:22:07 -07:00
chrislu
21b6b07dd8
renaming
2022-07-28 23:22:06 -07:00
chrislu
8ca7d1ef17
rename to SeaweedMQ
2022-07-28 23:22:06 -07:00
Chris Lu
3e7e922d16
Merge pull request #3378 from vanhtuan0409/change-put-policy
...
PutUserAction should completely replace identity actions
2022-07-28 19:43:25 -07:00
chrislu
89e276de64
fix loop
2022-07-28 18:06:49 -07:00
chrislu
a458bee9f2
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-07-28 16:32:02 -07:00
chrislu
e0eda52c54
mount: ensure symlink parent directory is tracked
...
fix https://github.com/chrislusf/seaweedfs/issues/3373
2022-07-28 16:32:00 -07:00
Tuan Vuong
1d9a888be9
PutUserAction should completely replace identity actions
2022-07-28 17:02:56 +07:00
zzq09494
01b7aa79fa
fix: Buckets are not created and deleted correctly on the filer with the same signature when they are created and deleted
2022-07-28 12:04:37 +08:00
chrislu
ca836568ac
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-07-27 12:12:43 -07:00
chrislu
3828b8ce87
"github.com/chrislusf/raft" => "github.com/seaweedfs/raft"
2022-07-27 12:12:40 -07:00
Chris Lu
b7e4e26778
Merge pull request #3371 from askeipx/issues/3341
...
delete disk_size metrics when collection deleted
2022-07-27 11:25:42 -07:00
Konstantin Lebedev
7e09a548a6
exclude directories to sync on filer
2022-07-27 19:22:57 +05:00
Evgeny Kuzhelev
ac5ce31278
leave notion to refactor after prometheus upgrade
2022-07-27 16:48:56 +05:00
Evgeny Kuzhelev
47c72e6f35
remove all (currently existing) collection volume metrics
2022-07-27 16:31:49 +05:00
Evgeny Kuzhelev
709c83716c
delete disk_size metrics when collection deleted
2022-07-27 15:02:41 +05:00
chrislu
f401b996eb
mount: fix symlink size reporting
2022-07-26 23:12:25 -07:00
Konstantin Lebedev
c88ea31f62
fix RUnlock of unlocked RWMutex
2022-07-26 12:57:07 +05:00
Konstantin Lebedev
da9d3e8f6c
refactor
2022-07-26 11:56:45 +05:00
Konstantin Lebedev
046c3d5ad4
fix logic else brake
2022-07-26 11:47:11 +05:00
chrislu
475185fb72
3.18
2022-07-25 22:41:38 -07:00
chrislu
ec0edb1ac4
filer: fix wrong logic during read
2022-07-25 22:40:00 -07:00
guosj
cc7a4b0a6e
correct comment
2022-07-25 11:46:41 +08:00
chrislu
33cff61992
fix glog test
2022-07-24 19:13:54 -07:00
chrislu
dbe4849ffd
3.17
2022-07-24 18:46:50 -07:00
chrislu
b98c0b4bf4
disable symlink translation for now
2022-07-24 18:46:32 -07:00
chrislu
2dc5d4adac
disable symlink translation for now
2022-07-24 18:24:34 -07:00
chrislu
3d2bbf96d2
adjust glog format
2022-07-24 17:29:08 -07:00
chrislu
2807fe08fc
adjust glog format
2022-07-24 16:44:58 -07:00
chrislu
2b64916d3e
set status to fuse.OK
2022-07-24 15:30:55 -07:00
chrislu
7386cf6a06
mount: resolve symlink
2022-07-24 15:11:24 -07:00
chrislu
40089b6753
maintain node lookup counter
2022-07-24 13:43:11 -07:00
chrislu
d24db396cc
fix removing path from inode2path
2022-07-23 18:20:29 -07:00
chrislu
41eeb4deef
do not add new inode during link
2022-07-23 13:37:07 -07:00
chrislu
02c2d81cde
fix removePathFromInode2Path
2022-07-23 13:21:42 -07:00
chrislu
4d5dc55766
mount: better support hard links
2022-07-23 12:57:00 -07:00
chrislu
ebda0689ab
use linux default max name length
2022-07-23 10:51:06 -07:00
chrislu
64f3d6fb6e
metadata subscription uses client epoch
2022-07-23 10:50:28 -07:00
chrislu
2c8818351f
mount: comment out stuck fuse inode notify during renaming
2022-07-22 18:17:34 -07:00
chrislu
a5c94acd4f
mount: check possible nil entry case
2022-07-22 15:33:37 -07:00
Chris Lu
0716092b39
Merge pull request #3350 from shichanglin5/optimize_masterclient_vidmap
...
Solve the problem that `LookupFileId` lookup urls is empty due to lea…
2022-07-22 02:57:50 -07:00
LHHDZ
994a2dec78
fix vid_map_test tests
2022-07-22 17:54:57 +08:00
LHHDZ
58c02d6429
Solve the problem that LookupFileId lookup urls is empty due to leader switching
...
The vidMap structure is modified to a linked list structure (the length is limited to 5). When the vidMap is reset, the current vidMap is added to the new vidMap as a cache node. When the query locations is empty, the cache node is searched to avoid problems when the master switches leaders.
2022-07-22 17:22:38 +08:00
ningfd
58f2dd6740
change topology mapping definition, improve spelling
2022-07-22 17:18:56 +08:00
ningfd
6f882eb354
optimiz: master ui will render data in order
2022-07-22 14:37:49 +08:00
chrislu
7a6c559ab4
fix Change replication via volume.configure.replication by collection
...
fix https://github.com/chrislusf/seaweedfs/issues/3346
2022-07-21 22:01:05 -07:00
chrislu
fc8241fb5e
leveldb3: add instant dropping bucket
2022-07-21 18:48:51 -07:00
chrislu
c93f7ffa44
explicit bucket aware declaration
2022-07-21 18:23:53 -07:00
Konstantin Lebedev
3c42814b58
avoid deadlock
2022-07-21 17:15:10 +05:00
Konstantin Lebedev
93ca87b7cb
use safe onPeerUpdateDoneCns
2022-07-21 15:51:14 +05:00
chrislu
6147b61b0a
mount: fix concurrent map read and map write
...
fix https://github.com/chrislusf/seaweedfs/issues/3344
2022-07-21 01:38:26 -07:00
Chris Lu
9ec0d1caaa
Merge pull request #3337 from guol-fnst/loading_volume
2022-07-20 09:06:06 -07:00
Konstantin Lebedev
7875470e74
onPeerUpdateGoroutineCount use int32
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
c5189c343b
remove ticker
...
update the topology before each file
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
e2d991d8d0
ticker.Stop
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
de4fcc0e2c
sync update topologyInfo
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
fa88dff7ce
update otherNodes
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
72dca31cfa
fix update topologyInfo
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
884ffbafee
clouse background update
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
39eaf426f8
fix TestVolumeServerEvacuate
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
b5e5f6f55a
update topologyInfo
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
867269cdcf
help rack
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
6f764e1014
volume server evacuate from rack
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
ba0e3ce5fa
volume server evacuate to target server
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
b6471ecd75
err msg with duplicated local subscription detected move to log level 1
...
https://github.com/chrislusf/seaweedfs/issues/3320
2022-07-20 18:40:50 +05:00
Konstantin Lebedev
6c390851e7
fix design
2022-07-20 18:08:12 +05:00
Tuan Vuong
312e13416b
filter duplicated action
2022-07-20 16:43:50 +07:00
guol-fnst
ac694f0c8f
rename parameter and reuse functions
...
rename milestone to watermark
2022-07-20 17:00:40 +08:00
Chris Lu
4820b594a7
Merge pull request #3298 from kmlebedev/evacuateTargetServer
...
volume server evacuate to target server
2022-07-19 13:56:49 -07:00
Konstantin Lebedev
d3f7c09c03
remove ticker
...
update the topology before each file
2022-07-20 00:54:23 +05:00
Konstantin Lebedev
11e393dbe7
err msg with duplicated local subscription detected move to log level 1
...
https://github.com/chrislusf/seaweedfs/issues/3320
2022-07-20 00:45:13 +05:00
Konstantin Lebedev
f6a966b4fc
add waiting log message
2022-07-20 00:31:57 +05:00
Konstantin Lebedev
6cfbfb0849
check for ping before deleting raft server
...
https://github.com/chrislusf/seaweedfs/issues/3083
2022-07-20 00:04:12 +05:00
chrislu
2ae3f812f8
minor
2022-07-19 11:43:31 -07:00
Chris Lu
9667588af0
Merge pull request #3318 from kmlebedev/issues/3310
...
Use fallback if urls are not found
2022-07-19 11:38:43 -07:00
Chris Lu
f0d560060f
Merge pull request #3336 from kmlebedev/raftSrvRemove
...
raftServerRemoval typo and logs
2022-07-19 11:32:17 -07:00
guol-fnst
91285bb51d
remove uncessary code
2022-07-19 20:00:44 +08:00
guol-fnst
b9256e0b34
optimiz
2022-07-19 16:58:53 +08:00
Konstantin Lebedev
f419d5643a
fix typo
...
add remove logs
2022-07-19 11:50:52 +05:00
guol-fnst
bec9f79659
use snapshot
2022-07-19 11:14:41 +08:00
Konstantin Lebedev
d422e7769c
ticker.Stop
2022-07-18 16:38:19 +05:00
Konstantin Lebedev
73a0dea16b
sync update topologyInfo
2022-07-18 16:27:02 +05:00
chrislu
56ec89625a
3.16
2022-07-18 00:53:05 -07:00
Konstantin Lebedev
2b4112e462
update otherNodes
2022-07-18 11:32:28 +05:00
Konstantin Lebedev
3c2774ec3d
fix update topologyInfo
2022-07-18 01:46:31 +05:00
Konstantin Lebedev
7b1497ee63
Use BackoffSchedule for getLookupFileId
2022-07-15 16:05:35 +05:00
Konstantin Lebedev
01996bccf8
Use fallback if urls are not found
2022-07-15 15:29:15 +05:00
Konstantin Lebedev
4d5144e50d
clouse background update
2022-07-15 13:51:08 +05:00
chrislu
1db012485f
mount: fix bug during busy writes
...
fix https://github.com/chrislusf/seaweedfs/issues/3315
2022-07-15 01:03:17 -07:00
chrislu
aca20cd9f4
more logging related to filer metadata subscription
2022-07-15 00:42:13 -07:00
Chris Lu
abeb7f65f8
Merge pull request #3314 from shichanglin5/fix_ETag
...
File upload should succeed should return `ETag` instead of `Etag`
2022-07-14 15:35:41 -07:00
chrislu
fa61074513
add clientId logging
2022-07-14 12:27:34 -07:00
chrislu
fbd8f868a1
filer may have trouble to re-connect
...
clientId is used twice: one for local metadata subscription, one for combined metadata subscription.
2022-07-14 12:15:31 -07:00
LHHDZ
c08f6d0ff7
File upload should succeed should return ETag instead of Etag
...
at `/go-1.18.1/src/net/textproto/header.go:39`, header is canonicalized by `textproto.CanonicalMIMEHeaderKey`
2022-07-14 19:45:01 +08:00
Chris Lu
957c9a2eb2
Merge pull request #3309 from guol-fnst/loading_volume
...
optimiz concurrency
2022-07-13 23:12:24 -07:00
guol-fnst
300b383cdf
use 10 or numCPU workers if env is not found
2022-07-14 14:06:46 +08:00
guol-fnst
308a48c0c2
optimiz concurrency
...
user can customize number of workers via env "GOMAXPROCS"
2022-07-14 09:57:25 +08:00
Chris Lu
1a130125de
Merge pull request #3308 from guo-sj/check_duplicate_ak
2022-07-13 09:23:37 -07:00
guosj
354b7bdff0
replace errors.New(fmt.Sprintf(...)) with fmt.Errorf()
2022-07-13 22:49:03 +08:00
chrislu
289402a741
writer pattern: similar changes to reader pattern
2022-07-13 02:30:44 -07:00
guosj
ab1b9697e6
supplement check duplicate accesskey
2022-07-13 17:28:20 +08:00
chrislu
6e90f7bdd0
detect sequential mode with a counter
2022-07-13 02:20:03 -07:00
chrislu
578c8aea0e
typo
2022-07-13 02:18:31 -07:00
chrislu
1451b389a4
Avoid fatal error: sync: Unlock of unlocked RWMutex
...
fix https://github.com/chrislusf/seaweedfs/issues/3306
2022-07-13 00:58:15 -07:00
石昌林
392da4e038
Make the prompt information clearer
2022-07-13 11:39:45 +08:00
石昌林
ad5e8f68ec
Check whether there is a duplicate accessKey when modifying iam
2022-07-13 11:36:11 +08:00
Chris Lu
fba5219dab
Merge pull request #3303 from guol-fnst/loading_volume
...
speeding up loading volumes
2022-07-12 03:01:24 -07:00
Konstantin Lebedev
6622240df7
fix TestVolumeServerEvacuate
2022-07-12 14:56:34 +05:00
chrislu
860c207c94
typo
2022-07-12 02:56:19 -07:00
Konstantin Lebedev
8372721a62
update topologyInfo
2022-07-12 13:47:21 +05:00
guol-fnst
26313060a3
speeding up loading volumes
2022-07-12 16:19:33 +08:00
Konstantin Lebedev
ee95d23a22
help rack
2022-07-12 11:56:58 +05:00
Konstantin Lebedev
087fa1347f
volume server evacuate from rack
2022-07-12 11:33:08 +05:00
Chris Lu
de97ba94c3
Merge pull request #3302 from guo-sj/fix_put_user_policy
2022-07-11 21:15:37 -07:00
guosj
b3076909d1
refactor code
2022-07-12 11:25:36 +08:00
Konstantin Lebedev
4236c36599
volume server evacuate to target server
2022-07-11 16:58:15 +05:00
Chris Lu
7929a50327
Merge pull request #3296 from guo-sj/fix_put_user_policy
...
refine PutUserPolicy
2022-07-11 01:49:25 -07:00
guosj
7dbf19d09a
refine PutUserPolicy
2022-07-11 16:38:41 +08:00
chrislu
37578929d4
fix compilation
2022-07-11 01:30:00 -07:00
chrislu
93ccc6e05f
3.15
2022-07-11 00:50:07 -07:00
chrislu
f87952a411
remove locks
2022-07-11 00:25:20 -07:00
guosj
5469efa336
close SkipCheckParentDirectory flag
2022-07-11 09:44:48 +08:00
chrislu
3d0defa663
fix compilation
2022-07-08 19:16:58 -07:00
chrislu
6f00c7f061
comments
2022-07-08 19:07:31 -07:00
chrislu
d685b94102
mount: rename also invalidate source inode
2022-07-08 19:07:03 -07:00
chrislu
48382676d2
fix filtering by volume id
2022-07-08 10:29:24 -07:00
liubaojiang
4e83e92156
add subscriber clientId if it is the first time
2022-07-08 20:27:25 +08:00
guosj
5b3192344d
fix iam CreatePolicy error
2022-07-08 17:42:21 +08:00
chrislu
e2ecf137a2
fix resetting fh.reader
2022-07-08 01:04:15 -07:00
chrislu
185332b91f
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-07-08 00:29:41 -07:00
chrislu
28add5a534
mount: fix racing conditions
...
prevent wrong reading when the SingleChunkCacher is started, but not finished yet
2022-07-08 00:29:39 -07:00
duanhongyi
1ceab96aba
filer tikv support tls
2022-07-08 14:23:06 +08:00
chrislu
a85ed3fe8f
minor
2022-07-07 22:27:24 -07:00
chrislu
2ea18cdcc8
remove dead code
2022-07-07 22:26:03 -07:00
chrislu
b7de5c6c43
shell: remove unused filer parameter from shell.toml
2022-07-07 15:01:23 -07:00
chrislu
f2f0482dd3
mount: random read also try to use the local cache first
2022-07-07 11:50:28 -07:00
chrislu
05c53820b9
mount: file handler release memory
2022-07-07 11:49:40 -07:00
chrislu
69ef6459c3
S3 ListObjectsV2 fix pagination under bucket root
...
fix https://github.com/chrislusf/seaweedfs/issues/3166
2022-07-06 02:28:34 -07:00
Konstantin Lebedev
785223e587
rabbitpubsub enable durable
2022-07-06 10:05:29 +05:00
chrislu
57975b93ff
adjust used size reporting
2022-07-04 13:47:42 -07:00
chrislu
a79f5bd0d5
adjust used size reporting
2022-07-04 13:46:32 -07:00
chrislu
3c79c77056
3.14
2022-07-04 13:15:49 -07:00
guol-fnst
5bc895673e
add getting bucket quota
2022-07-04 15:36:44 +08:00
guol-fnst
dd4664e610
fix usage
2022-07-04 11:31:39 +08:00
chrislu
141f662734
edge case checking when volume server does not have capacity to balance
...
fix https://github.com/chrislusf/seaweedfs/issues/3257
2022-07-01 12:16:18 -07:00
chrislu
bee3a7c798
fix tests
2022-07-01 12:09:42 -07:00
Chris Lu
ddb0f6bc74
Merge pull request #3258 from kmlebedev/issues_3255
...
volumeServer.evacuate fix select empty node
2022-07-01 02:36:30 -07:00
Konstantin Lebedev
3d5f1cbc5e
fix pagination
2022-07-01 14:17:49 +05:00
chrislu
06a8b174b5
also remove Sync() for idx file
2022-06-30 13:50:53 -07:00
chrislu
52580743b9
stuck with file.Sync()
...
starting to get weed/storage () - (master) > go test ./...
I0630 12:19:54 65819 needle_map_metric_test.go:26] FileCount expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:27] DeletedSize expected 1648 actual 1648
I0630 12:19:54 65819 needle_map_metric_test.go:28] ContentSize expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:29] DeletedCount expected 1648 actual 1959
I0630 12:19:54 65819 needle_map_metric_test.go:30] MaxFileKey expected 10000 actual 10000
I0630 12:19:54 65819 volume_loading.go:136] loading index /var/folders/jv/0rlhdck17jzgt7b3hcnq63mc0000gp/T/TestCompaction909350722/001/1.idx to memory
I0630 12:20:36 65819 volume_vacuum.go:98] Committing volume 1 vacuuming...
panic: test timed out after 10m0s
goroutine 61 [running]:
testing.(*M).startAlarm.func1()
/usr/local/go/src/testing/testing.go:2029 +0x8e
created by time.goFunc
/usr/local/go/src/time/sleep.go:176 +0x32
goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc0000cd520, {0x1c09bde?, 0x5ab798ea5c102?}, 0x1c67bd8)
/usr/local/go/src/testing/testing.go:1487 +0x37a
testing.runTests.func1(0xc0000cd520?)
/usr/local/go/src/testing/testing.go:1839 +0x6e
testing.tRunner(0xc0000cd520, 0xc00039fcd8)
/usr/local/go/src/testing/testing.go:1439 +0x102
testing.runTests(0xc0000dadc0?, {0x25b4460, 0x6, 0x6}, {0x2855108?, 0x40?, 0x25c4b80?})
/usr/local/go/src/testing/testing.go:1837 +0x457
testing.(*M).Run(0xc0000dadc0)
/usr/local/go/src/testing/testing.go:1719 +0x5d9
main.main()
_testmain.go:57 +0x1aa
goroutine 21 [chan receive]:
github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).flushDaemon(0x0?)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:883 +0x6a
created by github.com/chrislusf/seaweedfs/weed/glog.init.0
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:410 +0x1bf
goroutine 55 [syscall]:
syscall.syscall(0x44ac0f96?, 0xc00007ad80?, 0xc00098b480?, 0x10588df?)
/usr/local/go/src/runtime/sys_darwin.go:22 +0x4e
syscall.fcntl(0x100000001?, 0xc00098b4b0?, 0x1069a7c?)
/usr/local/go/src/syscall/zsyscall_darwin_amd64.go:319 +0x30
internal/poll.(*FD).Fsync.func1(...)
/usr/local/go/src/internal/poll/fd_fsync_darwin.go:18
internal/poll.ignoringEINTR(...)
/usr/local/go/src/internal/poll/fd_posix.go:74
internal/poll.(*FD).Fsync(0xc00038f1e0?)
/usr/local/go/src/internal/poll/fd_fsync_darwin.go:17 +0xfc
os.(*File).Sync(0xc00012a030)
/usr/local/go/src/os/file_posix.go:168 +0x4e
github.com/chrislusf/seaweedfs/weed/storage/backend.(*DiskFile).Sync(...)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/backend/disk_file.go:84
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).makeupDiff(0xc0000bb440, {0xc000633a40, 0x52}, {0xc000633aa0, 0x52}, {0xc000633b00, 0x52}, {0xc000633b60, 0x52})
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:295 +0x12fa
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).CommitCompact(0xc0000bb440)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:119 +0x3e8
github.com/chrislusf/seaweedfs/weed/storage.TestCompaction(0xc00025a000)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum_test.go:89 +0x305
testing.tRunner(0xc00025a000, 0x1c67bd8)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
goroutine 56 [chan receive, 10 minutes]:
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker.func1()
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:244 +0x41a
created by github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:234 +0x56
FAIL github.com/chrislusf/seaweedfs/weed/storage 600.194s
? github.com/chrislusf/seaweedfs/weed/storage/backend [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/backend/memory_map (cached)
? github.com/chrislusf/seaweedfs/weed/storage/backend/s3_backend [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/erasure_coding (cached)
? github.com/chrislusf/seaweedfs/weed/storage/idx [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/needle (cached)
ok github.com/chrislusf/seaweedfs/weed/storage/needle_map (cached)
ok github.com/chrislusf/seaweedfs/weed/storage/super_block (cached)
? github.com/chrislusf/seaweedfs/weed/storage/types [no test files]
? github.com/chrislusf/seaweedfs/weed/storage/volume_info [no test files]
FAIL
2022-06-30 12:32:55 -07:00
Konstantin Lebedev
5ed8165161
fix logic
...
add option targetServer
https://github.com/chrislusf/seaweedfs/issues/3255
2022-06-30 17:58:24 +05:00
石昌林
200a862701
fixed volume xx not found caused by missing VolumeLocation events
...
When the requested master node is the leader, `VolumeLocation` or `ClusterNodeUpdate` may be returned here. If it is `VolumeLocation`, the update will be performed while resetting the vidMap, otherwise the event will be lost
2022-06-30 13:41:56 +08:00
Chris Lu
4ae9884a6e
Merge pull request #3254 from shichanglin5/fix_subscribe_meta
...
[Critical] Avoid synchronizing from scratch due to err
2022-06-29 12:36:28 -07:00
Chris Lu
b98556cff5
Merge pull request #3253 from kmlebedev/constMultipartUploadsFolder
...
use const multipart uploads folder
2022-06-29 12:09:03 -07:00
石昌林
2b5b37231b
Make sure that onChangeEvent is called successfully before updating lastTsNs
2022-06-29 23:57:07 +08:00
石昌林
e17e6ed542
Avoid synchronizing from scratch due to err
2022-06-29 23:31:04 +08:00
Konstantin Lebedev
bcbdc4cb37
use const multipart uploads folder
...
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
chrislu
a8abab2412
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-29 02:44:14 -07:00
chrislu
449ecc8890
remove max connection age
...
related to https://github.com/chrislusf/seaweedfs/pull/3226
2022-06-29 02:44:12 -07:00
Chris Lu
c6e6e303db
Merge pull request #3200 from lapshin-vitaly/bufix/validate-tags-on-copy
...
validate tags on copy object and add regex for validating tags
2022-06-29 01:37:14 -07:00
Lapshin Vitaliy
7c3d9d0535
remove println
2022-06-29 10:43:02 +03:00
Lapshin Vitaliy
61b2e3f9a2
fix return
2022-06-28 18:31:46 +03:00
Lapshin Vitaliy
606252472c
fix return
2022-06-28 18:24:03 +03:00
Lapshin Vitaliy
e969370913
fix parseTagging function and add test for validate tags
2022-06-28 14:48:55 +03:00
Chris Lu
a79bf9d92a
Merge pull request #3252 from guol-fnst/update_tbk
...
remove hdfs, add tikv for full version update
2022-06-27 22:37:41 -07:00
guol-fnst
74c9f39d48
remove hdfs, add tikv for full version update
2022-06-28 12:35:07 +08:00
石昌林
e1b94eb6b9
fix error: concurrent map writes when add or remove cluster node
2022-06-28 10:34:59 +08:00
chrislu
4b1f48a399
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-27 12:40:50 -07:00
chrislu
4a65159250
fix reading time
2022-06-27 12:40:47 -07:00
xdadrm
3a42269da8
Fixes WebDAV 0-bytes files
...
Fixes the issue where files created via WebDAV show as 0-bytes size when read via fuse.
2022-06-27 20:19:21 +02:00
Chris Lu
4be406a095
Merge pull request #3249 from kmlebedev/issues_2589
...
avoid set currentMaster svc.local k8s discovery service domains
2022-06-27 11:05:07 -07:00
Konstantin Lebedev
9bff097b01
currentMaster redirected
2022-06-27 21:55:16 +05:00
Konstantin Lebedev
6c20a3b622
avoid set currentMaster k8s svc.local discoveruy service domains
...
https://github.com/chrislusf/seaweedfs/issues/2589
2022-06-27 21:47:05 +05:00
Lapshin Vitaliy
1a5981d583
fix test
2022-06-27 18:46:21 +03:00
garenchan
8aa19577f4
fix 3238: handle errors for GenerateDirUuid method
2022-06-27 22:04:50 +08:00
Lapshin Vitaliy
d7c3493d15
Merge branch 'master' into bufix/validate-tags-on-copy
2022-06-27 13:53:57 +03:00
chrislu
b9f7b6fb9a
adjust log message
2022-06-26 23:12:16 -07:00
Chris Lu
fdd880b111
Merge pull request #3237 from shichanglin5/remove_query_collection
...
Remove the collection query param of s3api and let the collection be …
2022-06-26 23:02:11 -07:00
石昌林
15bfc60a88
Remove the collection query param of s3api and let the collection be determined by the filer
2022-06-27 13:47:53 +08:00
yulai.li
af23e63e3f
Improve filer command help, add supported filer store list
2022-06-27 12:09:16 +08:00
chrislu
40a9634c3d
3.13
2022-06-26 20:13:52 -07:00
Chris Lu
cc01f71752
Merge pull request #3233 from blacktear23/filerstore-tikv
...
Add TiKV Filerstore back
2022-06-26 20:12:18 -07:00
yulai.li
824cbe32be
Make tikv filer enable/disable by build tags
2022-06-27 10:57:24 +08:00
chrislu
9dfea44aeb
fix unit
2022-06-26 12:21:38 -07:00
chrislu
3142e77eca
mount: stats report physical size instead of logical size
2022-06-26 12:14:34 -07:00
yulai.li
13159ad0d8
Merge branch 'master' into filerstore-tikv
2022-06-26 22:44:01 +08:00
yulai.li
46e0b629e5
Update tikv client version and add one PC support
2022-06-26 22:43:37 +08:00
chrislu
f77eda4b4f
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-25 09:22:52 -07:00
chrislu
3ce46c3566
fix entry full url lookup
2022-06-25 09:22:49 -07:00
Chris Lu
dc59ccd110
Merge pull request #3228 from shichanglin5/fix_volumeNotFound
...
When the connection with the leader is disconnected, the vidMap shoul…
2022-06-24 23:06:13 -07:00
石昌林
f1ea906c13
Code comment optimization
2022-06-25 11:56:09 +08:00
石昌林
0c8e033e68
When the connection with the leader is disconnected, the vidMap should not be reset immediately, but should be reset after connecting to a new leader
2022-06-24 17:41:46 +08:00
chrislu
9c517d2b35
masterclient: fallback to directly querying master in case of missing volume id location
2022-06-24 02:08:57 -07:00
Chris Lu
280e33092c
Merge pull request #3226 from kmlebedev/advancedtls
...
Advancedtls
2022-06-24 01:34:58 -07:00
chrislu
509a9047db
test compact map with snowflake sequencer
2022-06-23 21:45:51 -07:00
chrislu
217d7dc377
verify seq is always unique
2022-06-23 21:22:05 -07:00
Konstantin Lebedev
2e782dfdcb
rm defer comments
2022-06-24 01:50:13 +05:00
Konstantin Lebedev
7efaafb0ee
fix Authenticate
2022-06-24 01:42:04 +05:00
Konstantin Lebedev
b0aa51d7ef
enable require client cert
2022-06-24 00:29:23 +05:00
Konstantin Lebedev
ea7cdb8b0e
seperate option
2022-06-24 00:26:56 +05:00
Konstantin Lebedev
5b388ed6c1
initial advancedtls
2022-06-23 23:32:15 +05:00
chrislu
0c13fc873c
fix compilation
2022-06-23 00:45:11 -07:00
chrislu
96496d5286
master: broadcast new volume locations to clients to avoid possible racing condition
...
fix https://github.com/chrislusf/seaweedfs/issues/3220
2022-06-23 00:41:33 -07:00
chrislu
52c44d646e
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-22 00:34:27 -07:00
chrislu
530da0cc89
remove unused code
2022-06-22 00:34:25 -07:00
famosss
8a6e7a7df8
Merge branch 'chrislusf:master' into master
2022-06-22 09:58:56 +08:00
zzq09494
9df5ad5309
fix: vacuum create a lot of connections quickly
2022-06-22 09:57:22 +08:00
Chris Lu
d652ce1a36
Merge pull request #3216 from kmlebedev/mk_raft_basedir
...
avoid no such raft date directory
2022-06-21 12:21:30 -07:00
chrislu
6b3beb709d
fix format
2022-06-21 12:21:02 -07:00
Konstantin Lebedev
c1c8dad677
avoid no such raft date directory
...
https://github.com/chrislusf/seaweedfs/issues/3214
2022-06-21 13:47:51 +05:00
chrislu
4dc27e1ed5
3.12
2022-06-20 19:07:00 -07:00
chrislu
492da3dbce
master: put metadata under instance specific folder
2022-06-20 19:04:49 -07:00
chrislu
41dfe27102
Revert "remove max connection age"
...
This reverts commit b9b684194f
2022-06-20 14:49:39 -07:00
chrislu
2295766e2c
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-20 14:16:18 -07:00
chrislu
1d0c53ea56
remote storage: stop supporting hdfs as a remote storage
2022-06-20 14:15:59 -07:00
Stephan
1eb7826909
Fix link to osxfuse github page
2022-06-20 22:36:07 +02:00
chrislu
bf5a5abfb1
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-20 13:18:11 -07:00
chrislu
69a2e503ab
non-recursive directory deletion
...
fix a problem during git checkout a different version, where updated directories are deleted even though the directory is not empty.
2022-06-20 13:18:08 -07:00
chrislu
76f26c4b99
mount: delete target entry first during move
2022-06-20 13:15:02 -07:00
Chris Lu
460c7cf68e
Merge pull request #3203 from shichanglin5/volume_copy_preallocate
2022-06-20 07:41:45 -07:00
石昌林
81f7f08708
Determine whether to preallocate according to the master configuration before executing copy volume
2022-06-20 21:12:44 +08:00
石昌林
be5c901bea
Fix global counter key is incorrect & Add read lock when reading counter map to avoid memory problems
2022-06-20 19:16:53 +08:00
Lapshin Vitaliy
5f5fd0bc48
validate tags on copy object and add regex for validating tags
2022-06-20 11:32:58 +03:00
Chris Lu
9e2d6e897e
Merge pull request #3198 from guol-fnst/fix_statuscode
...
AbortMultipartUploadHandler should return 204 instead of 200
2022-06-19 22:57:13 -07:00
LHHDZ
8db9f13bc6
Merge branch 'master' into circuit_breaker
2022-06-20 13:42:18 +08:00
guol-fnst
1d77deccd0
AbortMultipartUploadHandler should return 204 instead of 200
...
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
2022-06-20 13:26:22 +08:00
石昌林
9e036df356
remove go.uber.org/atomic
2022-06-20 12:35:29 +08:00
geekboood
fdacd94af5
fix: invalid chunk data when failed to read manifests
2022-06-19 16:54:04 +08:00
石昌林
3dd60529c5
some code optimizations
2022-06-17 19:07:39 +08:00
石昌林
37df209195
add some unit tests and some code optimizes
2022-06-17 17:11:18 +08:00
Chris Lu
d28f8bb88f
Merge pull request #3192 from guol-fnst/weed_update
...
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-16 22:58:29 -07:00
guol-fnst
c7c20881f5
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-17 12:47:04 +08:00
chrislu
625fd16a2e
reduce upfront memory usage for low density volume
2022-06-16 15:39:29 -07:00
Chris Lu
5f8d6b5f13
Merge pull request #3184 from guol-fnst/weed_update
...
Weed update
2022-06-16 10:02:45 -07:00
guol-fnst
cd078471c4
fix path bug on windows
...
fix usage
2022-06-16 17:24:56 +08:00
chrislu
aaa9938353
Revert "adjust conditions"
...
This reverts commit e024586ff1
2022-06-16 01:26:36 -07:00
guol-fnst
995e221e97
remove -output, add -dir,-name for more flexibility
2022-06-16 16:07:32 +08:00
chrislu
e024586ff1
adjust conditions
2022-06-15 23:19:39 -07:00
