Konstantin Lebedev
1f7e52c63e
vacuum metrics and force sync dst files ( #3832 )
2022-10-13 00:51:20 -07:00
Guo Lei
f95c25e113
types packages is imported more than onece ( #3838 )
2022-10-12 22:59:07 -07:00
Guo Lei
84c401e693
Optimiz leveldb metric ( #3830 )
...
* optimiz updating mapmetric for leveldb
* import loading leveldb
* add comments
2022-10-11 21:13:25 -07:00
Ryan Russell
277976bd76
refactor(storage): readability improvements ( #3703 )
...
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-16 02:43:17 -07:00
Guo Lei
c57c79a0ab
optimiz commitig compact ( #3388 )
...
* optimiz vacuuming volume
* fix bugx
* rename parameters
* fix conflict
* change copyDataBasedOnIndexFile to an instance method
* close needlemap
* optimiz commiting Vacuum volume for leveldb index
* fix bugs
* fix leveldb loading bugs
* refactor
* fix leveldb loading bug
* add leveldb recovery
* add test case for levelDB
* modify test case to cover all the new branches
* use one tmpNm instead of two instances
* refactor
* refactor
* move setWatermark to the end
* add test for watermark and updating leveldb
* fix error logic
* refactor, add test
* check nil before close needlemapeer
add test case
fix metric bug
* add tests, fix bugs
* adjust log level
remove wrong test case
refactor
* avoid duplicate updating metric for leveldb index
2022-08-23 23:53:35 -07:00
Konstantin Lebedev
3c75479e2b
Merge branch 'master' into gentle_vacuum
...
# Conflicts:
# weed/pb/messaging_pb/messaging.pb.go
# weed/pb/messaging_pb/messaging_grpc.pb.go
# weed/pb/s3_pb/s3.pb.go
# weed/pb/volume_server_pb/volume_server.pb.go
# weed/server/volume_grpc_vacuum.go
2022-08-01 14:45:22 +05:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
Konstantin Lebedev
2f0dda384d
vacuum show LA
2022-07-29 11:59:33 +05:00
guosj
cc7a4b0a6e
correct comment
2022-07-25 11:46:41 +08:00
chrislu
06a8b174b5
also remove Sync() for idx file
2022-06-30 13:50:53 -07:00
chrislu
81d6159290
volume: report error if a volume has nil data backend
...
fix https://github.com/chrislusf/seaweedfs/issues/3105
2022-05-29 16:59:30 -07:00
chrislu
37ab8909b0
use two flags: v.isCompacting and v.isCommitCompacting
2022-04-26 23:28:34 -07:00
Konstantin Lebedev
1e35b4929f
shell vacuum volume by collection and volume id
2022-04-18 18:40:58 +05:00
Chris Lu
97a4b66df7
Merge pull request #2704 from guo-sj/fix_bugs_in_return_value
...
fix return value in storage/volume_vacuum.go:444
2022-02-24 00:49:29 -08:00
guosj
3e7aa1caf5
fix return value in storage/volume_vacuum.go:444
2022-02-24 15:54:36 +08:00
chrislu
c29bc9a367
fix error handling
2022-02-23 15:34:25 -08:00
guosj
d68c27f82d
fix another return value bug
2022-02-23 16:21:25 +08:00
guosj
8f9aa0cddd
fix bugs in return value
2022-02-23 16:17:48 +08:00
Konstantin Lebedev
ef541972f8
updated needle with fsync
2022-02-08 00:10:53 +05:00
Chris Lu
3be3c17f59
volume vacuum: avoid timeout with streaming progress report
...
fix https://github.com/chrislusf/seaweedfs/issues/2396
2021-10-24 01:55:34 -07:00
Chris Lu
1aa7e99a89
skip file not found error when deleting
2021-05-15 09:37:39 -07:00
Chris Lu
400de380f4
volume server: support tcp direct put/get/delete
2021-03-05 02:29:38 -08:00
Chris Lu
7635f6b9fa
disk file avoid file.Stat()
2021-02-20 20:06:06 -08:00
bingoohuang
7256902fb0
fix typo offset.ToAcutalOffset to offset.ToActualOffset
2021-02-07 12:11:51 +08:00
Chris Lu
f3bb645018
file open error
2020-12-01 23:37:49 -08:00
Chris Lu
dc0bc48257
return file open error
2020-12-01 23:36:49 -08:00
Chris Lu
6d30b21b10
volume: add "-dir.idx" option for separate index storage
...
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
f2723c1bc8
do not idx file format
...
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
2020-09-10 14:42:52 -07:00
Chris Lu
6ccd7f0a4d
refactoring
2020-08-18 18:01:37 -07:00
Chris Lu
ee11d98650
refactoring
2020-08-18 17:35:19 -07:00
Chris Lu
6a92f0bc7a
refactoring to typed Size
...
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
6190fd665d
printout error
2020-04-27 12:41:31 -07:00
Chris Lu
c8ca234773
refactoring
2020-04-11 14:27:25 -07:00
Chris Lu
df9d538044
rename function
2020-04-11 14:19:44 -07:00
Chris Lu
cbfe31a9a8
idx file sync before compaction
2020-03-20 23:38:46 -07:00
Chris Lu
81797a059a
volume: sync volume file right before compaction
...
fix https://github.com/chrislusf/seaweedfs/issues/1237
2020-03-19 23:54:52 -07:00
Chris Lu
d439d83772
volume: follow compactionBytePerSecond
...
related to https://github.com/chrislusf/seaweedfs/issues/1108
2020-03-11 10:32:17 -07:00
Chris Lu
d3e1ef64b3
ensure error checking on windows
2020-03-10 13:33:26 -07:00
Chris Lu
88a110e67e
release file handle
2020-03-10 13:32:22 -07:00
Chris Lu
43e6261579
volume: fix memory leak
2020-03-10 13:31:23 -07:00
Chris Lu
f87f2045b3
delete old file first on windows
...
fix https://github.com/chrislusf/seaweedfs/issues/1210
2020-02-27 00:59:35 -08:00
Chris Lu
757c7d67ad
avoid nil DataBackend
2020-02-27 00:07:25 -08:00
Chris Lu
430b5a49e2
unused code
2020-02-26 19:35:00 -08:00
Chris Lu
d335f04de6
support env variables to overwrite toml file
2020-01-29 09:09:55 -08:00
Chris Lu
c8b2dac6c1
volume: avoid sharing volume dat file handle
...
possibly help on https://github.com/chrislusf/seaweedfs/issues/1184
2020-01-21 21:18:01 -08:00
Chris Lu
acf7ca7b93
volume: fix compaction
2020-01-08 09:45:03 -08:00
Chris Lu
efd2f50ede
compaction changed to .idx based deletion
2019-12-24 14:55:50 -08:00
Chris Lu
abffe857a1
change btree map to in memory level db
2019-12-24 10:18:56 -08:00
Chris Lu
72a561ab7c
refactoring
2019-12-24 08:13:01 -08:00
