Commit Graph

114 Commits

Author SHA1 Message Date
chrislu
81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
James Hartig
4c85da7844
Include meta in ReadAllNeedles (#3991)
This is useful for doing backups on the data so we can accurately store the
last modified time, the compression state, and verify the crc.

Previously we were doing VolumeNeedleStatus and then an HTTP request which
needlessly read from the dat file twice.
2022-11-20 20:19:41 -08:00
chrislu
70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
870a30665b close responses 2022-08-31 00:24:17 -07:00
chrislu
4ffbda1c43 fix building for unmaintained 2022-08-17 14:45:38 -07:00
chrislu
eaeb141b09 move proto package 2022-08-17 12:05:07 -07:00
chrislu
13b9a52f80 fix deprecated functions 2022-07-29 01:34:39 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu
64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
guol-fnst
ac694f0c8f rename parameter and reuse functions
rename milestone to  watermark
2022-07-20 17:00:40 +08:00
creeew
02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
chrislu
a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
Nat Makarevitch
b72f0634d7
minor (typos...), done while reading around 2022-05-16 22:11:33 +08:00
chrislu
5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu
9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
225b019fe0 stream read multiple volumes in a volume server 2021-09-27 02:51:31 -07:00
Chris Lu
e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu
67949e25e9 add optional delay 2021-09-11 02:21:55 -07:00
Chris Lu
6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu
68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Chris Lu
6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
byunghwa.yun
285fdd2dd5 Add the additional meta log 2021-07-29 17:43:17 +09:00
Chris Lu
cc7714fdbe logging changes to debug 2021-06-27 05:54:16 -07:00
Chris Lu
a852f8a3f5 some code to stress test the meta data events 2021-05-22 18:03:13 -07:00
Chris Lu
2586621362 minor 2021-05-22 01:02:31 -07:00
bingoohuang
cf552417a7 minFreeSpace refactored 2021-04-27 10:37:24 +08:00
Chris Lu
7635f6b9fa disk file avoid file.Stat() 2021-02-20 20:06:06 -08:00
Chris Lu
6daa932f5c refactoring to get master function, instead of passing master values directly
this will enable retrying later
2021-02-17 20:55:55 -08:00
Chris Lu
487e435679 adjust http max idle connections per host
related to https://github.com/chrislusf/seaweedfs/issues/1802
2021-02-12 03:47:15 -08:00
bingoohuang
7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
Chris Lu
ec052f75d0 add example scripts 2020-11-17 13:14:03 -08:00
Chris Lu
283d584141 update to correct block size 2020-10-25 22:03:37 -07:00
Chris Lu
c693c98c15 write a lot of files 2020-10-25 19:28:11 -07:00
Chris Lu
3f7d1d1bf1 Only wait on retryable requests 2020-10-13 00:29:46 -07:00
Chris Lu
f2723c1bc8 do not idx file format
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
75de7002ff adjust size 2020-09-11 11:43:13 -07:00
Chris Lu
daf0a449f7 properly cancel context for streaming grpc 2020-09-09 12:07:15 -07:00
Chris Lu
eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu
c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu
332caf0cd7 maintain the unmaintained 2020-08-18 17:23:01 -07:00
Chris Lu
6a92f0bc7a refactoring to typed Size
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
4703a3daad add an example 2020-08-06 04:32:05 -07:00
James Hartig
ad12f77891 unmaintained: Added diff_volume_servers 2020-07-22 15:17:19 -04:00
Chris Lu
3080c197e3 rename UnCompressData to DecompressData 2020-06-20 08:16:16 -07:00
Chris Lu
e912fd15e3 renaming 2020-06-19 22:45:27 -07:00
ingardm
f711e8dde4
Rename see_dat_gzip to see_dat_gzip.go 2020-06-17 10:43:51 +02:00
ingardm
b7afa9fd57
Create see_dat_gzip 2020-06-17 10:43:28 +02:00
bingoohuang
b4e93b639d remove unnecessary comments 2020-05-29 11:28:08 +08:00
bingoohuang
accb4964b7 util.BytesToHumanReadable 2020-05-29 10:00:07 +08:00