Numblgw
aebf3952b7
filer sync: source path and exclude path support dir suffix ( #6268 )
...
filer sync: source path and exclude paht support dir suffix
Co-authored-by: liguowei <liguowei@xinye.com>
2024-11-21 08:25:12 -08:00
Max Denushev
a5fe6e21bc
feat(filer.backup): add ignore errors option ( #6235 )
...
* feat(filer.backup): add ignore errors option
* feat(filer.backup): fix 404 error wrap
* feat(filer.backup): fix wrapping function
* feat(filer.backup): fix wrapping errors in genProcessFunction
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
* Update weed/command/filer_backup.go
---------
Co-authored-by: Max Denushev <denushev@tochka.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-11-14 08:40:55 -08:00
chrislu
19d14b4c71
minor
2024-07-16 09:21:01 -07:00
vadimartynov
b796c21fa9
Added loadSecurityConfigOnce ( #5792 )
2024-07-16 09:15:55 -07:00
chrislu
3a82f5ffad
ensure metadata follow a specific folder
...
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12 11:17:30 -07:00
Konstantin Lebedev
d821cb3b18
fix: sync without dir /buckets/some/.uploads/hash_hash ( #5402 )
2024-03-20 12:54:29 -07:00
chrislu
83e4b02517
fix
2024-01-18 09:16:20 -08:00
chrislu
15b66a6633
refactor
2024-01-18 09:13:14 -08:00
XIAOYQ
be166b434f
fix: skip s3 .uploads
2024-01-18 22:13:46 +08:00
Konstantin Lebedev
8d23e36c45
fix: doDeleteFiles deletes files ( #5198 )
2024-01-12 11:04:29 -08:00
Konstantin Lebedev
1169f94310
Fix filer sync set offset ( #5197 )
...
* fix: compose 2mount with sync
* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
2024-01-12 10:57:18 -08:00
Konstantin Lebedev
b9d32d32e1
chore: filer sync add doDeleteFiles option for create only mode ( #5166 )
2024-01-06 10:02:16 -08:00
Konstantin Lebedev
2b229e98ce
fix: add doDeleteFile option for filer backup
2023-11-17 07:37:28 -08:00
Konstantin Lebedev
3c5295a1a6
filer backup add option for exclude file names that match the regexp to sync on filer
2023-11-13 06:23:46 -08:00
chrislu
5db9fcccd4
refactoring
2023-03-21 23:01:49 -07:00
chrislu
81fdf3651b
grpc connection to filer add sw-client-id header
2023-01-20 01:48:12 -08:00
chrislu
8a40fa8993
more detailed logs
2022-12-17 13:18:35 -08:00
Jiffs Maverick
4b0430e71d
[metrics] Add the ability to control bind ip ( #4012 )
2022-11-24 10:22:59 -08:00
chrislu
21c0587900
go fmt
2022-09-14 23:06:44 -07:00
Ryan Russell
2c92a9ff74
refactor: DefaultConcurrencyLimit var rename ( #3658 )
2022-09-14 06:30:32 -07:00
bernardx
228b133afa
new 'concurrency' parameter for filer.sync ( #3579 )
...
Co-authored-by: XIAOYQ <xiaoyq@eudic.net>
2022-09-02 23:03:23 -07:00
qzh
7fcfaf7bc9
fix(filer.sync): offset may be set to 0 ( #3451 )
...
* fix(filer.sync): initializing the offset is related to the path
* fix(filer.sync): the offset maybe to be set to 0.
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 23:43:52 -07:00
qzh
400f0c3e5d
fix(filer.sync): initializing the offset is related to the path ( #3450 )
...
Co-authored-by: zhihao.qu <zhihao.qu@ly.com>
2022-08-15 21:56:47 -07:00
chrislu
67814a5c79
refactor and fix strings.Split
2022-08-07 01:34:32 -07:00
chrislu
1a4bf0dcb5
filer.sync: parallelize the filer.sync
2022-08-07 00:56:15 -07:00
chrislu
0e9478488d
filer.sync: fix when excluded paths is empty
2022-08-07 00:55:34 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
Konstantin Lebedev
7e09a548a6
exclude directories to sync on filer
2022-07-27 19:22:57 +05:00
chrislu
64f3d6fb6e
metadata subscription uses client epoch
2022-07-23 10:50:28 -07:00
chrislu
4a65159250
fix reading time
2022-06-27 12:40:47 -07:00
zhihao.qu
4d0d1848c6
fix(filer.sync): modify clientName format : from -> to
2022-06-15 13:33:20 +08:00
zhihao.qu
42d04c581b
feat(filer.sync): add metricsServer in filer.sync.
...
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
2022-06-15 11:33:18 +08:00
zhihao.qu
14d82c3dea
feat(filer.sync): add offset to path.
2022-06-14 19:46:02 +08:00
zhihao.qu
cd5cca36a4
feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges
2022-06-09 10:53:19 +08:00
creeew
02ae102731
fix filer.sync missing source srv uploaded files to target when target down
2022-06-02 01:28:47 +08:00
chrislu
a2b101a737
subscribe metadata between a range
2022-05-30 15:04:19 -07:00
chrislu
1384529eb7
Fix filer.backup deletes files in backup folder in incremental mode
...
fix https://github.com/chrislusf/seaweedfs/issues/2919
2022-04-14 13:35:01 -07:00
chrislu
202a29d014
refactoring
2022-02-25 01:17:26 -08:00
chrislu
6a12520a96
fix logging
2022-01-10 01:00:11 -08:00
chrislu
5c87fcc6d2
add client id for all metadata listening clients
2021-12-30 00:23:57 -08:00
chrislu
9f9ef1340c
use streaming mode for long poll grpc calls
...
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu
e5fc35ed0c
change server address from string to a type
2021-09-12 22:47:52 -07:00
Chris Lu
8e125339d5
line wrap
2021-08-31 23:30:28 -07:00
Chris Lu
68ea99190f
add additionalPathPrefixes
2021-08-31 23:26:26 -07:00
Chris Lu
c5f38c365d
go fmt
2021-08-08 22:30:36 -07:00
Chris Lu
df85f7a1eb
adjust help message
2021-08-08 22:30:12 -07:00
Chris Lu
6b743dbbf9
refactor client subscribe metadata
2021-08-04 16:25:46 -07:00
Chris Lu
3afbf04007
add TLS grpc support for filer meta clients
2021-08-03 01:30:35 -07:00
Chris Lu
c2e0a75c1f
adjust logs
2021-06-24 12:46:00 -07:00
Chris Lu
2d7b4e5bb6
filer.backup: escape colon from path on windows
...
fix https://github.com/chrislusf/seaweedfs/issues/2084
2021-05-29 06:45:27 -07:00
