Konstantin Lebedev
4d08393b7c
filer prefer volume server in same data center ( #3405 )
...
* initial prefer same data center
https://github.com/seaweedfs/seaweedfs/issues/3404
* GetDataCenter
* prefer same data center for ReplicationSource
* GetDataCenterId
* remove glog
2022-08-04 17:35:00 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
chrislu
139e039c44
filer.sync: pass attributes for mount
...
fix https://github.com/chrislusf/seaweedfs/issues/3012
2022-05-06 03:54:12 -07:00
chrislu
9405eaefdb
filer.sync: fix replicating partially updated file
...
Run two servers with volumes and fillers:
server -dir=Server1alpha -master.port=11000 -filer -filer.port=11001 -volume.port=11002
server -dir=Server1sigma -master.port=11006 -filer -filer.port=11007 -volume.port=11008
Run Active-Passive filler.sync:
filer.sync -a localhost:11007 -b localhost:11001 -isActivePassive
Upload file to 11007 port:
curl -F file=@/Desktop/9.xml "http://localhost:11007/testFacebook/ "
If we request a file on two servers now, everything will be correct, even if we add data to the file and upload it again:
curl "http://localhost:11007/testFacebook/9.xml "
EQUALS
curl "http://localhost:11001/testFacebook/9.xml "
However, if we change the already existing data in the file (for example, we change the first line in the file, reducing its length), then this file on the second server will not be valid and will not be equivalent to the first file
Снимок экрана 2022-02-07 в 14 21 11
This problem occurs on line 202 in the filer_sink.go file. In particular, this is due to incorrect mapping of chunk names in the DoMinusChunks function. The names of deletedChunks do not match the chunks of existingEntry.Chunks, since the first chunks come from another server and have a different addressing (name) compared to the addressing on the server where the file is being overwritten.
Deleted chunks are not actually deleted on the server to which the file is replicated.
2022-02-07 03:46:28 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
99b599aa8a
remote.mount
2021-07-26 22:53:44 -07:00
Chris Lu
7359193e97
go fmt
2021-07-21 14:38:12 -07:00
Chris Lu
7ab389e7ec
optimization: improve random range query for large files
2021-07-19 23:07:22 -07:00
Chris Lu
450222dd64
add remote to filer.Entry and filer_pb entry, add RemoteConf
2021-07-19 02:47:27 -07:00
Chris Lu
540441fd38
go fmt
2021-02-28 20:34:14 -08:00
Chris Lu
678c54d705
data sink: add incremental mode
2021-02-28 16:19:03 -08:00
Chris Lu
a0e84c4fbc
go fmt
2021-02-10 23:41:05 -08:00
Chris Lu
821c46edf1
Merge branch 'master' into support_ssd_volume
2021-02-09 11:37:07 -08:00
Chris Lu
80b8692688
filer.sync: replicate outside of either cluster, only need to see filers
2021-01-24 00:01:44 -08:00
Chris Lu
2b76854641
add "weed filer.cat" to read files directly from volume servers
2021-01-06 04:22:00 -08:00
Chris Lu
1bf22c0b5b
go fmt
2020-12-16 09:14:05 -08:00
Chris Lu
51eadaf2b6
rename parameter name to "disk"
2020-12-13 12:05:31 -08:00
Chris Lu
0d2ec832e2
rename from volumeType to diskType
2020-12-13 11:59:32 -08:00
Chris Lu
e9cd798bd3
adding volume type
2020-12-13 00:58:58 -08:00
Chris Lu
f4abd01adf
filer: cache small file to filer store
2020-11-30 04:34:04 -08:00
Chris Lu
e219c57849
passing full path when assign volume locations
2020-10-25 15:46:29 -07:00
Chris Lu
387ab6796f
filer: cross cluster synchronization
2020-09-09 11:21:23 -07:00
Chris Lu
eb7929a971
rename filer2 to filer
2020-09-01 00:21:19 -07:00
Chris Lu
ca658a97c5
add signatures to messages to avoid double processing
2020-08-28 23:48:48 -07:00
Chris Lu
97d97f3528
go code can read and write chunk manifest
2020-07-19 17:59:43 -07:00
Chris Lu
31e23e9783
filer: support active<=>active filer replication
2020-06-30 22:53:57 -07:00
Chris Lu
ec2eb8bc48
add If-None-Match and If-Modified-Since
...
fix https://github.com/chrislusf/seaweedfs/issues/1269
2020-04-08 08:12:00 -07:00
Chris Lu
b97768c51c
refactoring
2020-03-23 01:30:22 -07:00
Chris Lu
c0f0fdb3ba
refactoring
2020-03-23 00:01:34 -07:00
Chris Lu
8645283a7b
fuse mount: avoid lookup nil entry
...
fix https://github.com/chrislusf/seaweedfs/issues/1221
2020-03-07 16:51:46 -08:00
Chris Lu
892e726eb9
avoid reusing context object
...
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu
7d10fdf737
fix directory lookup nil
2020-02-25 11:13:06 -08:00
Chris Lu
6ab7368ef2
filer: dynamically create bucket under /buckets folder
2020-02-24 22:28:45 -08:00
Chris Lu
d335f04de6
support env variables to overwrite toml file
2020-01-29 09:09:55 -08:00
Chris Lu
72a64a5cf8
use the same context object in order to retry
2020-01-26 14:42:11 -08:00
Chris Lu
c48fc8b4de
grpc send error via response instead of grpc error
2020-01-25 09:17:19 -08:00
Chris Lu
3fa1f150d9
refactoring
2019-06-22 20:04:56 -07:00
Chris Lu
b3b42bc947
replicate need to include new entry path
2019-04-16 00:44:31 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
Chris Lu
39ea8484c1
adjust proto API
2018-12-11 07:17:10 -08:00
Chris Lu
6219a9ad1f
filer ordered notifications
2018-10-31 21:48:05 -07:00
Chris Lu
08266b7256
go fmt
2018-10-11 00:08:13 -07:00
Chris Lu
e8ef501f02
add s3 replication sink
2018-10-03 23:36:52 -07:00
Chris Lu
31ed352ab6
replication handle cases when entry already exists
2018-09-25 09:27:03 -07:00
Chris Lu
9fe24991d5
refactoring
2018-09-23 00:40:36 -07:00
