zemul
6b4c033431
add mount log ( #4101 )
...
* filer.backup use replication.source.filer
* add mount log
* Revert "filer.backup use replication.source.filer"
This reverts commit 07bf6f956c
.
* fix
Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-01-03 00:00:45 -08:00
chrislu
48f2edc065
3.38
2023-01-02 23:23:02 -08:00
Chris Lu
d4566d4aaa
more solid weed mount ( #4089 )
...
* compare chunks by timestamp
* fix slab clearing error
* fix test compilation
* move oldest chunk to sealed, instead of by fullness
* lock on fh.entryViewCache
* remove verbose logs
* revert slat clearing
* less logs
* less logs
* track write and read by timestamp
* remove useless logic
* add entry lock on file handle release
* use mem chunk only, swap file chunk has problems
* comment out code that maybe used later
* add debug mode to compare data read and write
* more efficient readResolvedChunks with linked list
* small optimization
* fix test compilation
* minor fix on writer
* add SeparateGarbageChunks
* group chunks into sections
* turn off debug mode
* fix tests
* fix tests
* tmp enable swap file chunk
* Revert "tmp enable swap file chunk"
This reverts commit 985137ec47
.
* simple refactoring
* simple refactoring
* do not re-use swap file chunk. Sealed chunks should not be re-used.
* comment out debugging facilities
* either mem chunk or swap file chunk is fine now
* remove orderedMutex as *semaphore.Weighted
not found impactful
* optimize size calculation for changing large files
* optimize performance to avoid going through the long list of chunks
* still problems with swap file chunk
* rename
* tiny optimization
* swap file chunk save only successfully read data
* fix
* enable both mem and swap file chunk
* resolve chunks with range
* rename
* fix chunk interval list
* also change file handle chunk group when adding chunks
* pick in-active chunk with time-decayed counter
* fix compilation
* avoid nil with empty fh.entry
* refactoring
* rename
* rename
* refactor visible intervals to *list.List
* refactor chunkViews to *list.List
* add IntervalList for generic interval list
* change visible interval to use IntervalList in generics
* cahnge chunkViews to *IntervalList[*ChunkView]
* use NewFileChunkSection to create
* rename variables
* refactor
* fix renaming leftover
* renaming
* renaming
* add insert interval
* interval list adds lock
* incrementally add chunks to readers
Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order
* fix tests compilation
* fix tests compilation
2023-01-02 23:20:45 -08:00
dependabot[bot]
367353b936
build(deps): bump github.com/aws/aws-sdk-go from 1.44.167 to 1.44.171 ( #4100 )
...
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.167 to 1.44.171.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.167...v1.44.171 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 12:24:46 -08:00
dependabot[bot]
de787ce4ab
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.41.0 to 3.42.1 ( #4099 )
...
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.41.0 to 3.42.1.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.41.0...v3.42.1 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 12:24:28 -08:00
dependabot[bot]
04bba1a6b2
build(deps): bump modernc.org/sqlite from 1.20.0 to 1.20.1 ( #4098 )
...
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 12:24:15 -08:00
CommanderRoot
c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 ( #4094 )
2023-01-01 05:07:53 -08:00
CommanderRoot
c2280e94cf
filer.store.mysql: Replace deprecated upsert syntax ( #4096 )
2023-01-01 05:06:57 -08:00
CommanderRoot
265a56630b
filer.store.mysql: Escape table columns in SQL query ( #4095 )
2023-01-01 05:06:41 -08:00
Guo Lei
ed5f3f073b
add more help message, in case of misunderstanding ( #4092 )
2022-12-29 07:05:05 -08:00
Guo Lei
d8cfa1552b
support enable/disable vacuum ( #4087 )
...
* stop vacuum
* suspend/resume vacuum
* remove unused code
* rename
* rename param
2022-12-28 01:36:44 -08:00
dependabot[bot]
84492976c9
build(deps): bump github.com/aws/aws-sdk-go from 1.44.162 to 1.44.167 ( #4085 )
2022-12-26 11:48:35 -08:00
dependabot[bot]
bd176359cb
build(deps): bump github.com/prometheus/procfs from 0.8.0 to 0.9.0 ( #4084 )
2022-12-26 11:48:17 -08:00
dependabot[bot]
b2aa1a23ff
build(deps): bump github.com/arangodb/go-driver from 1.4.0 to 1.4.1 ( #4083 )
2022-12-26 11:47:52 -08:00
Guo Lei
5f993a5f20
Fix ide warning ( #4082 )
...
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2022-12-26 00:03:47 -08:00
fibbanachi
3901f2da4d
Add logtostderr option in helm chart ( #4080 )
2022-12-23 07:36:15 -08:00
Konstantin Lebedev
569bc7b54d
fs.verify needles status of file ( #4060 )
...
* fs.verify needles status of file
* fix parse args
* refactor logging
* set modifyTimeAgoAtSec
2022-12-21 15:54:17 -08:00
chrislu
77b7c9f7cc
lock and unlock chunks for read
2022-12-21 14:15:35 -08:00
chrislu
d1797deccb
remove duplicated logic of removing from writableChunks
2022-12-21 14:15:11 -08:00
chrislu
4878e0fd46
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-20 19:47:23 -08:00
chrislu
6ede19e825
add a simple file replication progress bar
2022-12-20 19:47:21 -08:00
Naden
27d187eed9
Fixes 4074: S3 Gateway fails to start when installed via Helm ( #4075 )
...
Fixes #4074 : S3 Gateway fails to start when installed via Helm
Co-authored-by: Naden <naden@harana.com>
2022-12-20 17:24:54 -08:00
Naden
3b90eb1911
Fixes: 4076: Host path prefix should be configurable ( #4077 )
...
Co-authored-by: Naden <naden@harana.com>
2022-12-20 17:24:12 -08:00
chrislu
f7beba8515
unlock before submitting the uploading jobs
2022-12-20 12:52:30 -08:00
chrislu
28d479e5c0
mount: adjust locking for upload pipeline
2022-12-19 15:07:22 -08:00
chrislu
f9fb3cd1fc
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-19 11:32:00 -08:00
chrislu
6c7fe40305
filer sink retries reading file chunks, skipping missing chunks
...
if the file chunk is not available during replication time, the file is skipped
2022-12-19 11:31:58 -08:00
chrislu
3fc3e7083c
metadata skip reading un-available logs
...
this means the volumes for metadata histories have been lost.
2022-12-19 11:30:50 -08:00
dependabot[bot]
a1ffa03d75
build(deps): bump google.golang.org/api from 0.104.0 to 0.105.0 ( #4067 )
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.104.0 to 0.105.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.104.0...v0.105.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 10:21:20 -08:00
dependabot[bot]
3886c31ed8
build(deps): bump github.com/aws/aws-sdk-go from 1.44.146 to 1.44.162 ( #4069 )
...
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go ) from 1.44.146 to 1.44.162.
- [Release notes](https://github.com/aws/aws-sdk-go/releases )
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.146...v1.44.162 )
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:22:36 -08:00
dependabot[bot]
9fdf98b826
build(deps): bump cloud.google.com/go/pubsub from 1.27.1 to 1.28.0 ( #4068 )
...
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go ) from 1.27.1 to 1.28.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.27.1...pubsub/v1.28.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:22:23 -08:00
dependabot[bot]
c7a28b20b5
build(deps): bump actions/checkout from 3.1.0 to 3.2.0 ( #4066 )
...
Bumps [actions/checkout](https://github.com/actions/checkout ) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/actions/checkout/releases )
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md )
- [Commits](93ea575cb5...755da8c3cf
)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:22:12 -08:00
dependabot[bot]
6c5d44a5c4
build(deps): bump actions/setup-go from 3.4.0 to 3.5.0 ( #4065 )
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](d0a58c1c4d...6edd4406fa
)
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:21:49 -08:00
dependabot[bot]
ae847c2780
build(deps): bump actions/dependency-review-action from 3.0.1 to 3.0.2 ( #4064 )
...
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action ) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/actions/dependency-review-action/releases )
- [Commits](11310527b4...0ff3da6f81
)
---
updated-dependencies:
- dependency-name: actions/dependency-review-action
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:21:41 -08:00
dependabot[bot]
3b85dc5fbb
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.40.1 to 3.41.0 ( #4070 )
...
build(deps): bump github.com/ydb-platform/ydb-go-sdk/v3
Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk ) from 3.40.1 to 3.41.0.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases )
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.40.1...v3.41.0 )
---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:21:30 -08:00
dependabot[bot]
b8a4ed455f
build(deps): bump golang.org/x/tools from 0.3.0 to 0.4.0 ( #4071 )
...
Bumps [golang.org/x/tools](https://github.com/golang/tools ) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/golang/tools/releases )
- [Commits](https://github.com/golang/tools/compare/v0.3.0...v0.4.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/tools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 09:21:20 -08:00
lfhy
038bab3726
add -filer.path to webdav command ( #4061 )
2022-12-17 20:07:02 -08:00
chrislu
8a40fa8993
more detailed logs
2022-12-17 13:18:35 -08:00
LHHDZ
f84c13dabd
fix response not allowed ( #4057 )
2022-12-13 05:48:50 -08:00
dependabot[bot]
693e07b509
build(deps): bump golang.org/x/net from 0.2.0 to 0.4.0 ( #4055 )
2022-12-12 08:57:22 -08:00
dependabot[bot]
1949944e18
build(deps): bump google.golang.org/api from 0.103.0 to 0.104.0 ( #4054 )
2022-12-12 08:57:11 -08:00
dependabot[bot]
6c8a3d57e4
build(deps): bump go.mongodb.org/mongo-driver from 1.11.0 to 1.11.1 ( #4053 )
2022-12-12 08:56:35 -08:00
dependabot[bot]
d88199c8b0
build(deps): bump github.com/tikv/client-go/v2 from 2.0.2 to 2.0.3 ( #4052 )
2022-12-12 08:56:23 -08:00
dependabot[bot]
21f77384f0
build(deps): bump github.com/go-redsync/redsync/v4 from 4.6.0 to 4.7.1 ( #4051 )
2022-12-12 08:56:00 -08:00
LHHDZ
bb289d7d69
sort tags by key ( #4047 )
...
Signed-off-by: changlin.shi <changlin.shi@ly.com>
Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-12-11 22:49:57 -08:00
chrislu
438146249f
3.37
2022-12-11 21:08:06 -08:00
chrislu
ac9dea0ad9
rotate log files
2022-12-11 21:05:58 -08:00
chrislu
fc6b9e6e0c
volume server logs add url info
...
debug https://github.com/seaweedfs/seaweedfs/issues/3964
2022-12-11 13:15:38 -08:00
chrislu
38c63c429b
Merge branch 'master' of https://github.com/seaweedfs/seaweedfs
2022-12-10 21:49:09 -08:00
chrislu
e1ca6308cb
add chunk etag when downloading from remote storage
...
fix https://github.com/seaweedfs/seaweedfs/issues/3987
2022-12-10 21:49:07 -08:00