mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-11-23 18:49:17 +08:00

Table of Contents

Async filer metadata backup to a different store

It is desirable to have a copy of the filer metadata. However, it is not always easy to setup replications for filer metadata store:

The store does support stream replication.
The store is embedded leveldb, RocksDB, etc.
It is just complicated.

Async filer metadata backup to a different store

We can continuously backup filer metadata to a different store, without running a filer instance.

         Metadata Changes
 Filer --------------------> `weed filer.meta.backup` ---> A backup store(LevelDB/RocksDB/
                                                                        Mysql/Postgres/
                                                                        Redis/Cassanra/...)

Just need to configure the backup store the same way as filer.toml.

weed filer.meta.backup can be stopped and resumed. The metadata backup progress is tracked in the backup store itself. So you can pause/resume any time.

It is worth noting that the backup store can be different from the source filer store. E.g., you can use a cheaper on disk LevelDB as a remote store to backup Redis.

         Metadata Changes
 Filer --------------------> `weed filer.meta.backup` ---> A backup store(LevelDB)
on Redis

You can even do this:

         Metadata Changes                                                           sqlite streaming  
 Filer --------------------> `weed filer.meta.backup` ---> A backup store(Sqlite)  ------------------> S3
on Redis

Introduction

API

Configuration

Filer

Filer Stores

Advanced Filer Configurations

FUSE Mount

WebDAV

Cloud Drive

AWS S3 API

AWS IAM

Machine Learning

TensorFlow with SeaweedFS

HDFS

Replication and Backup

Async Replication to another Filer [Deprecated]
Async Backup
Async Filer Metadata Backup
Async Replication to Cloud [Deprecated]
Kubernetes Backups and Recovery with K8up