2022-08-30: This needs a major update.
- Update from ClearOS to Virtualmin
- We should manage Virtualmin backups instead or in addition to Tiki Manager backups
- We should figure out a way to permit a Syncthing instance per Virtualmin Virtual Server.
Use case: I have some Tiki instances on ClearOS and I want automatic backups of different Tiki instances to various local computers for safekeeping. These are high level instructions as to how to combine Tiki Manager and Syncthing to do so. For a hosting company and its customer, this is the optimal solution: This permits a customer to delegate the hosting / management of an application while obtain automatic backups of their application and data.
At a high level:
- Tiki Manager (installed on the same server) will make incremental and automated backups of each site to a distinct folder
- Syncthing will securely copy each folder to one or many devices.
Benefits / Features
- Automated (unattended): put Tiki Manager backups on a cron job
- Incremental while being somewhat efficient for disk spaceThis could be improved quite a bit. Volunteers welcome! : Tiki Manager has a retention cycle: keep last 7 days, last 4 weeks and once per month forever
- Remote backup to any number of devices: Syncthing is designed for peer-to-backups
- Managed via a web interface (Syncthing)
- Web interface can be protected by 2-factor authentication (in addition to password)
- Data is encrypted in transport by Syncthing
- Cross-platform: Syncthing supports GNU/Linux, Windows, OSX, Android, *BSD, Solaris
- The Syncthing Lite Android app even lets you access files on-demand.
- Devices can be behind the firewall, and without a fixed IP address. Ex.: Syncthing on your laptop
- Throttling: Can set Incoming and Outgoing Rate Limit (KiB/s)
- Backup Tiki, the database and even files that are outside the web root: Tiki Manager checks the Tiki database to know which directories to fetch, if any.
- Designed to cope with a Ransomware attack (see recipe below)
- Flexible multi-site design: Each Tiki is backed up in own folder providing flexibility on what to send where.
- Easy to restore: backups are in a format that can be restored by Tiki Manager with the instance:restore command.
- Avoid disk full issues: Syncthing will halt before filling up 100% of your disk. (Tiki Manager still needs this added). TODO: Add alerting to both
- This recipe is not particularly efficient for bandwidth or disk-space. Ex.: there is no deduplication algorithm
ClearOS as web server
Install each Tiki as a distinct website using sub-domains: How to set up websites on ClearOS
Setup Tiki Manager
Install Tiki Manager in /opt/tiki-manager/app as per https://doc.tiki.org/How-to-install-Tiki-Manager-on-ClearOS#Install_Tiki_Manager_Step_By_Step
Use Tiki Manager to make instances, or adopt them
- Use local with these instructions: https://doc.tiki.org/Manager#instance:create
Configure Tiki Manager to backup your Tiki instances
The Tiki Manager archive folders have the following pattern:
Check that your backup is OK, and move on to the next step
Set up automatic backups
Tiki Manager will make the backups and also has a retention cycle (keep last 7 days, last 4 weeks and once per month forever)
Of all the instances
For just one of the instances
Syntax for doing a backup of only one instance with no human interaction (let's say instance #3)
php tiki-manager instance:backup --instances=3 --no-interaction
You may want to install this in a cronjob. To do it every day at midnight, do the following.
echo '0 0 * * * cd /opt/tiki-manager/app/ ; php tiki-manager instance:backup --instances=3 --no-interaction' > /etc/cron.d/tm-instance.wikisuite.org-backup
On the server
You can install many instances of Syncthing, but there is not yet a good way to segment permissions. So it's better to create one master Syncthing user. Ex.: tikisyncthingbackup and manage all the backups from that account. Do not give that passwords to customers.
then visit https://example.org:81/syncthing/ (it may ask login again in basic auth)
On your laptop
- Install Syncthing for your local computer: https://docs.syncthing.net/intro/getting-started.html#installing
- Connect the devices and setup shared folders as per https://docs.syncthing.net/intro/getting-started.html
Make sure Syncthing automatically restarts when you reboot so that you have unattended automatic backups. Just reboot to see if it's OK. If not, you need to look up the documentation for your Operating System.
Configuring Syncthing for your Tiki archive folder
Your archives will be in something like /opt/tiki-manager/app/backup/archive/1-example.org/: so sync that folder with your local computer.
Setting up Syncthing for backups
Syncthing by default is set to sync multiple folders so they get to the exact same set of files and folders. So if you delete a file in one folder, the deletion is propagated. In the contexts of backups, this could cause issues. Examples:
- Ransomware on any device: Then, all the devices would receive the corrupted data which replaces the valid data.
- One of the devices runs out of space, and its user decides to clear out old backups.
The solution is:
- On the device sending the backups: set Folder Type to "Send Only"
- On the device(s) receiving the backups: set File Versioning. Recommended setting is "Staggered"
Because the master server is set to "Send Only", there is no point in setting to "Staggered" (It doesn't receive any data, and thus doesn't need versioning).
- If sync is not working as expect, check the Syncthing folder interface for errors.