chrislu
86d4c52d6e
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-03-26 13:33:20 -07:00
chrislu
21e0898631
refactor: change masters from a slice to a map
2022-03-26 13:33:17 -07:00
Chris Lu
3153c836a7
Merge pull request #2744 from kmlebedev/issues_2545
...
avoid connect to the old filler address
2022-03-26 13:02:49 -07:00
chrislu
cae998eda1
Revert "Revert "remove duplicated metadata subscription in filer""
...
This reverts commit 34b743c481
2022-03-26 13:00:16 -07:00
chrislu
34b743c481
Revert "remove duplicated metadata subscription in filer"
...
This reverts commit 34742be029https://github.com/chrislusf/seaweedfs/issues/2545
2022-03-26 12:33:45 -07:00
chrislu
4e2388e1b5
mount: set file size if it is only on remote gateway
2022-03-21 02:09:54 -07:00
chrislu
6886167d5f
minor optimization
2022-03-17 00:03:28 -07:00
chrislu
4042fdf3bb
rename to skipCheckParentDir
...
related to https://github.com/chrislusf/seaweedfs/pull/2761
It's better to default to false.
2022-03-16 23:55:31 -07:00
zzq09494
40b0033fa7
go fmt
2022-03-17 14:19:48 +08:00
zzq09494
ee5417a5b3
fix: ut test error
2022-03-17 11:16:46 +08:00
zzq09494
81cce4b4c3
filer: support uploading file without needEnsureParentDir
2022-03-17 10:53:47 +08:00
zzq09494
a6a8892255
Revert "filer: support uploading file without needEnsureParentDir"
...
This reverts commit a93c4947ba
2022-03-17 10:27:17 +08:00
zzq09494
a93c4947ba
filer: support uploading file without needEnsureParentDir
2022-03-17 10:18:23 +08:00
chrislu
fbc9f0eb64
minor
2022-03-14 03:19:16 -07:00
chrislu
2f4cd20f90
tests skip reader pattern monitoring
2022-03-13 18:15:53 -07:00
chrislu
53513475bf
mount: add back random read support
...
avoid too much memory used also
2022-03-13 01:38:52 -08:00
Konstantin Lebedev
834210a9dc
avoid connect to the old filler address
...
https://github.com/chrislusf/seaweedfs/issues/2545
2022-03-10 15:24:45 +05:00
zzq09494
2ea18fca48
fix:support some databases which not support 'IF NOT EXISTS' keyword of 'create table command'
2022-03-10 13:37:34 +08:00
banjiaojuhao
f7f2a597dd
minor
2022-03-08 16:22:55 +08:00
banjiaojuhao
d61bea9038
[bugfix] filer: In file modification, old chunks will be mis-deleted when they are merged(Manifestized).
2022-03-08 16:22:55 +08:00
chrislu
6f3ec989a7
fix manifest batch size
2022-03-07 00:07:53 -08:00
banjiaojuhao
a07c93f4dd
filer: reset bytesBuffer before use
2022-03-06 21:07:36 +08:00
chrislu
da76af187f
mount: avoid possible index out of bounds error
2022-03-04 22:36:01 -08:00
chrislu
6fbbc78574
stream reading a whole chunk
2022-03-02 13:50:46 -08:00
chrislu
941ced60a4
download 2 chunks if at the beginning of a file
2022-02-27 03:57:24 -08:00
chrislu
551d00d51a
prefetch other chunks when stream reading
2022-02-26 23:20:45 -08:00
chrislu
7b1a713d2a
remove dead code
2022-02-26 03:23:15 -08:00
chrislu
3a58b7bac6
a little safer
2022-02-26 03:23:06 -08:00
chrislu
3345a50d9b
prefetch 2 chunks
2022-02-26 03:06:17 -08:00
chrislu
86ce69f709
remove logs
2022-02-26 03:00:20 -08:00
chrislu
28b395bef4
better control for reader caching
2022-02-26 02:16:47 -08:00
chrislu
72c0233938
less logs
2022-02-25 15:34:24 -08:00
chrislu
03466f955e
rename: delete source entry metadata only, skipping hard links
2022-02-25 02:57:54 -08:00
chrislu
8080fe4cc1
logs
2022-02-25 02:56:23 -08:00
chrislu
202a29d014
refactoring
2022-02-25 01:17:26 -08:00
chrislu
be3fc77391
mount2: use consistent inode
2022-02-25 00:53:27 -08:00
chrislu
ceaf993a27
mount2: add rdev
2022-02-24 14:51:25 -08:00
chrislu
b05962b90e
rename: handle hard links
2022-02-24 02:59:00 -08:00
chrislu
56da3494cb
leveldb3: adjust memory allocation for each bucket
2022-02-20 13:40:13 -08:00
chrislu
7bc67399e4
listing for filer stores without prefixed query: break if no more progress
2022-02-19 10:12:51 -08:00
chrislu
248c0c8087
minor
2022-02-19 09:54:09 -08:00
chrislu
4ee0a6f47b
filer store: reduce one possible listing operation
2022-02-19 00:43:42 -08:00
chrislu
b3594278c9
optimize a bit
2022-02-18 23:36:10 -08:00
chrislu
df51e0c042
mongodb: remove ErrNoDocuments checking since it is only for creating single record, not for querying
2022-02-15 16:18:36 -08:00
chrislu
98cce6a150
mongodb: ensure closing cursor
2022-02-15 16:15:44 -08:00
banjiaojuhao
b5ec346700
FilerStore: add redis_lua
2022-02-15 20:54:57 +08:00
Eng Zer Jun
b92df1654c
test: use T.TempDir to create temporary test directory
...
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-02-14 10:38:13 +08:00
chrislu
85c1615b43
filer read empty file may cause OOM in some cases
...
fix https://github.com/chrislusf/seaweedfs/issues/2641
2022-02-07 23:08:54 -08:00
chrislu
9405eaefdb
filer.sync: fix replicating partially updated file
...
Run two servers with volumes and fillers:
server -dir=Server1alpha -master.port=11000 -filer -filer.port=11001 -volume.port=11002
server -dir=Server1sigma -master.port=11006 -filer -filer.port=11007 -volume.port=11008
Run Active-Passive filler.sync:
filer.sync -a localhost:11007 -b localhost:11001 -isActivePassive
Upload file to 11007 port:
curl -F file=@/Desktop/9.xml "http://localhost:11007/testFacebook/ "
If we request a file on two servers now, everything will be correct, even if we add data to the file and upload it again:
curl "http://localhost:11007/testFacebook/9.xml "
EQUALS
curl "http://localhost:11001/testFacebook/9.xml "
However, if we change the already existing data in the file (for example, we change the first line in the file, reducing its length), then this file on the second server will not be valid and will not be equivalent to the first file
Снимок экрана 2022-02-07 в 14 21 11
This problem occurs on line 202 in the filer_sink.go file. In particular, this is due to incorrect mapping of chunk names in the DoMinusChunks function. The names of deletedChunks do not match the chunks of existingEntry.Chunks, since the first chunks come from another server and have a different addressing (name) compared to the addressing on the server where the file is being overwritten.
Deleted chunks are not actually deleted on the server to which the file is replicated.
2022-02-07 03:46:28 -08:00
chrislu
4a311c7f5e
dedup local metadata subscribers
...
fix https://github.com/chrislusf/seaweedfs/discussions/2542
2022-01-23 16:14:22 -08:00
