Loading...
 

History: TRIM and Syncthing for automated offsite backups

Preview of version: 23

Use case: I have a Tiki on ClearOS. and I want automatic backups of that Tiki to my local computer for safekeeping. These are high level instructions as to how to combine TRIM and Syncthing to do so.

ClearOS as a web host

An alternate scenario is a web host where several Tikis are hosted. These Tikis are for distinct projects/customers

The webhost should be able to backup all sites via TRIM. This is set up once per server (Syncthing fetches all TRIM backups)

Each project / customer should get a backup of their data via TRIM. This is a different setup for each site.

Setup TRIM

Please proceed to: How to install Tiki with TRIM

Configure TRIM to backup your Tiki site

In this example, we will add an instance named ii.wikisuite.org. TRIM and TIKI are in the same server. Pay attention where to input 127.0.0.1. todo change docs to use make instance with local option

[root@trimclient trim]# make instance
php -d memory_limit=256M scripts/addinstance.php
Default editor used (nano). You can change the EDITOR environment variable.
Default diff used (diff). You can change the DIFF environment variable.
Connection type [ssh|ftp] : ssh
Host name : 127.0.0.1
Port number : [22]
User : root
Instance name : [127.0.0.1] ii.wikisuite.org
Contact email : your@email.com
Instance information saved.
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: WARNING: All keys were skipped because they already exist on the remote system.

Testing connection...
After successfully entering your password, were you asked for a password again? [yes|no] no
Detecting remote configuration.
You are running on a ClearOS
Web root : [/var/www/virtual/127.0.0.1/] /var/www/virtual/ii.wikisuite.org
Web URL : [http://127.0.0.1] http://ii.wikisuite.org/
Working directory : [/tmp/trim_temp]
Instance information saved.
PHP Notice:  Undefined offset: 1 in /root/trim/src/rclib.php on line 70
Detected SVN : branches/15.x
[root@trimclient trim]#


You can backup as always.

[root@trimclient trim]# make backup
bash scripts/backup.sh php -d memory_limit=256M
php
which: no cpulimit in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
dont use cpulimit
Default editor used (nano). You can change the EDITOR environment variable.
Default diff used (diff). You can change the DIFF environment variable.
Note: Backups are only available on SSH instances.

Which instances do you want to backup?
[2] vv.wikisuite.org                                         m@example.org
[3] ii.wikisuite.org                                           your@example.org
>>> 3
Performing backup for ii.wikisuite.org
Downloading files locally.
RSYNC Return var: 0
Obtaining database dump.
Creating archive.
Return var: 0
[root@trimclient trim]#

Set up automatic backups

TRIM will make the backups and also has a retention cycle (keep last 7 days, last 4 weeks and once per month forever)

Of all the instances

https://doc.tiki.org/TRIM#To_setup_automated_backups

For just one of the instances

We will use the PIPE from the Linux bash in order to inject the desired number. In this case, it is the number 3.

[root@trimclient trim]# echo 3 | make backup
bash scripts/backup.sh php -d memory_limit=256M
php
which: no cpulimit in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
dont use cpulimit
Default editor used (nano). You can change the EDITOR environment variable.
Default diff used (diff). You can change the DIFF environment variable.
Note: Backups are only available on SSH instances.

Which instances do you want to backup?
[2] vv.wikisuite.org                                           m@example.org
[3] ii.wikisuite.org                                           your@example.org
Performing backup for ii.wikisuite.org
Downloading files locally.
RSYNC Return var: 0
Obtaining database dump.
Creating archive.
Return var: 0

As you see, no human interaction was needed.
You may want to install this in a cronjob, to do it every day at noon, do the following.

echo '0 0 * * * pushd /root/trim/ && echo 3 | make backup &&  popd' > /etc/cron.d/trim-ii.wikisuite.org-backup


Install Syncthing

On the server

On your laptop

Setup Syncthing


The TRIM archive folders have the following pattern:

  • /var/local/trim/backup/archive/1-example.org/
  • /var/local/trim/backup/archive/2-example.com/
  • etc.


Make sure Syncthing automatically restarts when you reboot so that you have unattended automatic backups. Just reboot to see if it's OK. If not, you need to look up the documentation for your Operating System.

Setting up Syncthing for backups

Syncthing by default is set two sync multiple folders so they get to the exact same set of files and folders. So if you delete a file in one folder, the deletion is propagated. In the contexts of backups, this could cause issues. Ex.: one the devices runs out of space, and its user decides to clear out old backups. Then, all devices would lose the old backups. The solution is the set Folder Type to "Send Only"

The other thing that could happen: The master server is compromised and the backups are corrupted. Then, all the devices would receive the corrupted data which replaces the valid data. The solution here is to set https://docs.syncthing.net/users/versioning.htmlFile Versioning. Recommended setting is "Staggered"

Because the master server is set to "Send Only", there is no point in setting to "Staggered" (It doesn't receive any data, and thus doesn't need versioning).

History

Advanced
Information Version
Marc Laporte 54
View
Marc Laporte 53
View
Marc Laporte 52
View
Marc Laporte 51
View
Marc Laporte 50
View
Marc Laporte 49
View
Marc Laporte 48
View
Marc Laporte 47
View
Marc Laporte 46
View
Marc Laporte 45
View
Marc Laporte 44
View
Marc Laporte Nicer with images :-) 43
View
Marc Laporte 42
View
Marc Laporte 41
View
Marc Laporte 40
View
Marc Laporte 39
View
Marc Laporte 38
View
Marc Laporte 37
View
Marc Laporte 36
View
Marc Laporte 35
View
Marc Laporte 34
View
Marc Laporte 33
View
Marc Laporte 32
View
Marc Laporte 31
View
Marc Laporte 30
View
Marc Laporte 29
View
Marc Laporte Benefits / Features 28
View
Marc Laporte 27
View
Marc Laporte 26
View
Marc Laporte 25
View
Marc Laporte cleaning up 24
View
Marc Laporte 23
View
Marc Laporte 22
View
Marc Laporte 21
View
Marc Laporte 20
View
Marc Laporte 19
View
Marc Laporte 18
View
Marc Laporte 17
View
Marc Laporte 16
View
Marc Laporte 15
View
Marc Laporte will be simpler 14
View
Marc Laporte Better domain name 13
View
Marc Laporte 12
View
Marc Laporte 11
View
Marc Laporte 10
View
Marc Laporte Code Plugin modified by editor. 9
View
Marc Laporte Removing to avoid confusion 8
View
Marc Laporte 7
View
luis.lucio 6
View
luis.lucio 5
View
  • «
  • 1 (current)
  • 2