chrislu
7bc67399e4
listing for filer stores without prefixed query: break if no more progress
2022-02-19 10:12:51 -08:00
chrislu
248c0c8087
minor
2022-02-19 09:54:09 -08:00
chrislu
4ee0a6f47b
filer store: reduce one possible listing operation
2022-02-19 00:43:42 -08:00
chrislu
b3594278c9
optimize a bit
2022-02-18 23:36:10 -08:00
chrislu
df51e0c042
mongodb: remove ErrNoDocuments checking since it is only for creating single record, not for querying
2022-02-15 16:18:36 -08:00
chrislu
98cce6a150
mongodb: ensure closing cursor
2022-02-15 16:15:44 -08:00
banjiaojuhao
b5ec346700
FilerStore: add redis_lua
2022-02-15 20:54:57 +08:00
Eng Zer Jun
b92df1654c
test: use T.TempDir to create temporary test directory
...
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-02-14 10:38:13 +08:00
chrislu
85c1615b43
filer read empty file may cause OOM in some cases
...
fix https://github.com/chrislusf/seaweedfs/issues/2641
2022-02-07 23:08:54 -08:00
chrislu
9405eaefdb
filer.sync: fix replicating partially updated file
...
Run two servers with volumes and fillers:
server -dir=Server1alpha -master.port=11000 -filer -filer.port=11001 -volume.port=11002
server -dir=Server1sigma -master.port=11006 -filer -filer.port=11007 -volume.port=11008
Run Active-Passive filler.sync:
filer.sync -a localhost:11007 -b localhost:11001 -isActivePassive
Upload file to 11007 port:
curl -F file=@/Desktop/9.xml "http://localhost:11007/testFacebook/ "
If we request a file on two servers now, everything will be correct, even if we add data to the file and upload it again:
curl "http://localhost:11007/testFacebook/9.xml "
EQUALS
curl "http://localhost:11001/testFacebook/9.xml "
However, if we change the already existing data in the file (for example, we change the first line in the file, reducing its length), then this file on the second server will not be valid and will not be equivalent to the first file
Снимок экрана 2022-02-07 в 14 21 11
This problem occurs on line 202 in the filer_sink.go file. In particular, this is due to incorrect mapping of chunk names in the DoMinusChunks function. The names of deletedChunks do not match the chunks of existingEntry.Chunks, since the first chunks come from another server and have a different addressing (name) compared to the addressing on the server where the file is being overwritten.
Deleted chunks are not actually deleted on the server to which the file is replicated.
2022-02-07 03:46:28 -08:00
chrislu
4a311c7f5e
dedup local metadata subscribers
...
fix https://github.com/chrislusf/seaweedfs/discussions/2542
2022-01-23 16:14:22 -08:00
chrislu
606667f205
able to configure the quota for a bucket
2022-01-21 01:42:20 -08:00
guol-fnst
da9540e666
add gocql timeout setting
2022-01-18 15:21:13 +08:00
banjiaojuhao
45e9c83421
padding zero for sparse file
2022-01-13 22:21:22 +08:00
chrislu
1453263b63
remove dead code
2022-01-13 02:02:04 -08:00
chrislu
10ecf80ca1
add a debug capability to list all metadata keys
2022-01-11 23:25:04 -08:00
chrislu
34742be029
remove duplicated metadata subscription in filer
...
https://github.com/chrislusf/seaweedfs/issues/2545
2021-12-30 01:51:52 -08:00
chrislu
9a00c17555
reader: avoid wrong pattern detection due to lock waiting
2021-12-28 16:30:33 -08:00
chenkai
47c30e3add
filer list entries use context to break job
2021-12-28 15:03:41 +08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
banjiaojuhao
083bf3a137
filer server: add "datacenter, rack and datanode" for path specific configuration
2021-12-23 23:25:05 +08:00
chrislu
b541e39a2c
fix tests
2021-12-22 16:17:30 -08:00
chrislu
0ec7bc6710
detect non streaming mode on the first read
2021-12-22 16:05:38 -08:00
chrislu
4c1368d621
fix test
2021-12-22 16:05:08 -08:00
chrislu
0cb9036f66
mount: only cache the first chunk on stream read
2021-12-19 23:06:03 -08:00
chrislu
a152f17937
mount: improve read performance on random reads
2021-12-19 22:43:14 -08:00
Bl1tz23
83bffca7e6
fix MongodbStore.ListDirectoryEntries panic on find failure
2021-12-03 13:58:37 +03:00
Chris Lu
689f5513a9
redis3 supports sentinel
2021-11-29 01:09:51 -08:00
Chris Lu
2f72c24498
skip the rest logic
2021-11-29 00:32:21 -08:00
Chris Lu
96c66ca2aa
read deleted chunks when replicating data
2021-11-28 23:33:03 -08:00
limd
ec03f22cc3
Merge remote-tracking branch 'origin/master'
2021-11-25 16:07:14 +08:00
limd
220797bd71
support redis sentinel
2021-11-25 15:57:03 +08:00
Chris Lu
7bf891c00a
randomize same-dc servers and other-dc servers
2021-11-12 11:30:11 -08:00
Chris Lu
4729a57cc0
use constants
2021-11-08 17:47:56 -08:00
Chris Lu
e0fc2898e9
auto updated filer peer list
2021-11-06 14:23:35 -07:00
Chris Lu
5ea86ef1da
Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()"
...
This reverts commit af71ae11aa
2021-11-05 17:52:15 -07:00
Chris Lu
7348d2ae63
randomize traffic if replicated
2021-11-05 00:03:13 -07:00
Chris Lu
af71ae11aa
master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()
2021-11-03 01:09:48 -07:00
Chris Lu
0c8dea9de8
go fmt
2021-11-02 23:39:16 -07:00
Chris Lu
1b90d60765
filer: paginate large directories to delete file chunks
2021-11-02 01:04:50 -07:00
Chris Lu
24858507cc
rename API to avoid confusion
2021-10-30 19:27:25 -07:00
Chris Lu
f5824f369c
avoid reporting error when it is a user request error
2021-10-29 14:14:29 -07:00
Chris Lu
2e76834e4d
filer store: redis2 fix wrong pagination
2021-10-27 23:46:25 -07:00
Chris Lu
900e888695
filer: avoid possible prefixed pagination loop on unsupported filer stores
2021-10-27 23:45:48 -07:00
Chris Lu
8e2c9713a3
turn on new faster algorithm to translate into visible chunks
2021-10-16 23:35:20 -07:00
Chris Lu
e7d1f183cc
ensure file id is populated
2021-10-16 23:34:27 -07:00
Chris Lu
fc1694b321
add tests
2021-10-16 23:34:12 -07:00
Chris Lu
b9a2efd69b
temporarily reverting
2021-10-16 16:29:10 -07:00
Chris Lu
8a124ef9fc
Revert "remove deprecated code"
...
This reverts commit de7688c539
2021-10-16 16:09:33 -07:00
Chris Lu
93624635b4
Revert "go fmt"
...
This reverts commit 6c704eb2ba
2021-10-16 16:09:30 -07:00
