TurnKey Linux Backup and Migration (TKLBAM)

New to TKLBAM? Take the tour first for a quick overview and screenshots.

Smart automated backup and restore

TKLBAM is GPL3 licensed free software built on top of Duplicity which provides an efficient system-level backup of changed files, users, databases and package management state. This system-level backup can be restored automatically on any installation of the same type of virtual appliance, regardless of the underlying hardware or location. The intended result is a functionally equivalent copy of the original system.

It is also designed to assist in migration of data and system configurations between different versions of the same type of virtual appliance though for some applications, additional manual steps, such as a database schema update, may be required to complete migration between versions.


Assuming the exact same server type (e.g. LAMP backup restored to LAMP server) and a pre-tested backup (to ensure that it includes all the required files):
A backup should always restore cleanly on the same server.
A backup should usually restore cleanly on a server of the exact same version (e.g. v16.1 -> v16.1).
A backup should most likely restore cleanly on a server of the same major version (e.g. v16.0 -> v16.1).
A backup might restore ok on a server of a single newer major version (e.g. v15.2 -> v16.1) although it may require some tweaking.
A backup will almost certainly need some manual intervention if you restore to a server of a (much) newer version (e.g. v14.2 -> v16.1).


It is generally recommended that when restoring to anything other than the same major version that the restore is staged. I.e. first download the backup, then restore the DB (where relevant) and specific files and directories from the download. E.g. (where BACKUP_ID is the actual ID of the backup to restore and only restoring the directories /root & /var/www):

mkdir -p /tklbam-dump
tklbam-restore BACKUP_ID --raw-download=/tklbam-dump
tklbam-restore /tklbam-dump --limits="/root /var/www"

Note that other backed up files can be found in /tklbam-dump, relative to root (/). E.g. The backup of /etc/my.conf will be found in /tklbam-dump/etc/my.conf. Note too, that manually moving files out of the backup directory won't restore permissions. If you wish to restore files from the download with the original permissions, add them to the --limits="" (contains a space separated list of paths; sub-paths to exclude can be prefixed with '-''). For more info, please see the tklbam-restore man page.

Comments

Jeremy Davis's picture

So all the stuff it does prior to using Duplicity is above and beyond!

TKLBAM works best with TurnKey Linux servers as it is preconfigured to back up the relevant areas of the filesystem (depending on which appliance you are using). I haven't used it much on other platforms as basically I don't use other platforms much any more but it now supports Debian and Ubuntu pretty easy (from what I have read) although you'll need to configure it all so it backs up the bits you want backed up...

Also by default TKLBAM uses AWS S3 as it's default backup location, so if you have it linked to a Hub account it gives you a nice UI to see what backups exist, when they ran, whether they were full or incremental, etc.

As for front-end, it has a Webmin module available but otherwise is commandline. As I mentioned, the Hub sort of also provides some 'front end' type functionality although that only applies when you use S3 as the backup target.

Jeremy Davis's picture

I just had a look at Duply. Sorry I was confused and just assumed that you meant Duplicity...! After having a look I think that you are right... It appears that TKLBAM and Duply are quite similar. I hadn't ever heard of it before so I can't offer any sort of comparison.

Having only ever used TKLBAM on TKL servers I don't think anything could compare in that context! But I don't know how hard it is to set up a fresh profile. I have always intended to get TKLBAM to back up my Debian desktop but never got around to it so far...

Perhaps have a look in the forums. I recall that there was at least one guy on there using it to backup a heaps of Debian/Ubuntu machines...

Liraz Siri's picture

Came across this thread quite a bit late. TKLBAM and Duply both wrap around Duplicity but Duply is a much thinner wrapper. It basically just improves the command line usability of Duplicity a bit.

TKLBAM on the other hand, was designed as a system-level backup/restore tool. For starters it doesn't just backup directories/files, it backs up and automatically restores databases, ownership, users/groups, etc. The goal is for restores to be as automatic as possible, so post-restore you get something much closer to a working system rather than just a collection of files you have to massage into place as with a normal backup.

With Duply you necessarily have to configure what files/directories you want backed up. TKLBAM downloads a system profile that contains a list of default directories to back up. You can tweak/override these defaults in /etc/tklbam/overrides but out of the box it already does what you want.

When you use it on a TurnKey system the profile also includes timestamps of the files in the image version so when it backs up a specific directory (e.g., /etc) it will only backup files within /etc that have changed since installation. This makes migrating your changes to a different/newer system easier.

And yes, there is a web interface for TKLBAM via a webmin module that's integrated into TurnKey by default but which you could install yourself on another system. Yadda yadda.

Jeremy Davis's picture

rm -rf /var/lib/tklbam

Then just tklbam-init to link your new key.

David Poole's picture

Worked for me.

[edit] looked at my history. Didn't actually try --force.

 

Phillip Moore's picture

It also supports snapshots and lock-free concurrent access.

Here's the github page: https://github.com/gilbertchen/duplicacy-beta

This is where they compare duplicacy to other backup tools such as duplicity: https://github.com/gilbertchen/duplicacy-beta#comparison-with-other-backup-tools

Not sure if TKLBAM can be setup with this or not, but I thought that it was something worth noting.

Jeremy Davis's picture

Duplicacy looks interesting. Liraz (the TKLBAM Daddy) hasn't been around on the forums for a while but I'll bring this to his attention! :)