1

I've been doing research on how to setup a self host backup high availability solution using raspberry pi's. The biggest requirement is master to master replication and cross platform client support. One thing that is a big focus is that the raspbery pi's might not be on the same lan network, maybe ever at a different house. Should be able to remove a hard drive and plug in a new one when there is a failure. Key thing to backup solutions is more than one copy. I want high availability and encryption.

After some research I've determined what may be the best opensourced solutions right now.

Client syncing

  • nextcloud - prefered option so far
    • has mobile and desktop sync clients
    • uses mysql or postgres db
  • resilio
    • Seems to be easiest of any of the options, supports peer to peer on all devices, may be option 2 if database master to master replication on nextcloud doesn't work out. Only option of all the technologies that is not opensourced. Does not charge a subscription for consumers, pay once for family license.

Server peer to peer file sync:

  • resilio
    • costs money if you use their client applications and want to do specific sub folders, doesn't cost for a full peer to peer replication, has a mobile client
  • dat
    • new peer to peer file protocol similar to websockets
  • syncthing

Database master to master replication:

  • biggest area of concern mainly for nextcloud file sharing service

Connection:

  • how to connect and setup a cluster would be interesting
    • vpn
    • telebit
    • aws vpc vpn connection
    • some other protocol

Databases:

  • mysql - supposedly supports clustering
  • gundb - nosql db don't think it would work with nextcloud
  • cockroach db - enterprise tested, has mysql and postgres api support
  • yugabyte - postgres api supports clustering claims to be faster than cockroach db
  • postgres - least likely to work has a master - slave replication support

Access over internet:

  • telebit - may support load balancing with same configuration haven't tested
  • openvpn - connect to vpn to access a raspberry pi hosted backup solution for a location

Load balancing:

  • pending research - several options out there, may have to be external from local network
  • telebit suppposedly supports it
  • could pay for aws application load balancer and connect local machines to aws vpc via aws vpn
  • apache load balancer - doesn't guarantee health checks if one is down
  • nginx load balancer

Automated installation/startup - want to make as plug in and play as possible

  • dietpi - works on rock64 and raspberry pi
    • has some already supported software and can setup an bash script to run on first install to install third party software
  • startup service
    • besides starting up all other technologies put together, will send out an email with network information and a vpn cert to connect and debug a cluster instance having issues. Super easy using a bash script, just though I'd mention this.

Honorable mentions:

  • syncloud combines several of these technologies, but out of the box doesn't have master to master replication recomendations.
  • owncloud - nextcloud is a opensourced branch off of this
  • backupbuddy - no ios solution
  • syncthing - no ios solution, has a android solution

Are there alternative technologies that people here would recommend and why? Main concern seems to be the database master to master replication.

closed as too broad by joan, goldilocks Apr 19 at 16:08

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.