Ryan J's picture
Ryan J - Wed, 2011/01/26 - 01:52

Hi,

I just discovered TKL a few days ago and I'm really interested in using it.  I have a few questions though.

1) I would really like to use LVM snapshots as described here:

https://blueprints.launchpad.net/turnkeylinux/+spec/use-lvm-snapshots-with-tklbam

Is it something I'll be able to do using the existing configuration options or would I end up having to hack around in the /usr/lib/tklbam libs?

2) I'm not familiar with how duplicity handles full backups / increments.  Is a full backup every month going to cause a re-upload of all my data and a big increase in the amount of storage I'm using?

For example, I have a file server with ~5GB of infrequently updated data.  Is my second full backup going to require transferring all 5GB?  Will it balloon my S3 bucket to 10GB?

3) I'm a tiny bit confused about tlkpatch.  Is it just a tool used to build images that can be shared with the community?

4) I could test this for myself, but it would take far longer than asking here, so I apologize in advance.  I have a TeamCity build server on a VM.  With the exception of the database, everything it needs to run is consolidated under one directory (/apps).  I can basically copy that directory to a new machine, import the most current database version, link to the init scripts and be running on the new machine.

Could I simply copy my whole implementation into a PostgreSQL (appliance) installation, import the DB  and have TKLBAM work auto-magically?

5) The ability to actually test my offsite backups is the thing that makes TKL so appealing.  Is it okay for me to start up an AMI (ex: the fileserver appliance), restore 10GB (or 20GB or 100GB) worth of backups, have a look around to make sure everything is as expected and shut it down within the hour?  It doesn't seem right to be able to do that for only $0.10.  Am I missing something?

Ryan

Forum: 
Support
Tags: 
tklbam
Jeremy Davis's picture

Jeremy Davis - Wed, 2011/01/26 - 02:32

Glad your loving TKL, I do too :) I'm certainly not a Linux poweruser (<2yrs) but I have been quite involved with TKL over the last few years so I will answer your questions to the best of my ability. Hopefully one of the devs (or someone who knows more about this stuff than me anyway) will be along to pad out my answers with more details.

  1. TKL now uses LVM by default and also by default leaves 10% of available HDD space (I assume for snapshots following on from the blueprint). AFAIK the rest of the suggestion has not been implemted yet (although I'd love to be proven wrong). I suspect you'll probably have to do a bit of fiddling around to get it all working nice. If you do, be great if you could document your findings as you go so others users can follow your lead, also it may help push this forward as a built-in TKL feature.
  2. By default yes (I think anyway) it will upload all your data again but AFAIK by default it will only keep one full backup, so your old one will be deleted. All this behaviour should be able to be modified. Have a look through the TKLBAM docs.
  3. TKLPatch is a development tool for community members to develop unofficial appliances. The idea is then that anyone can then use the community created patches to recreate the custom appliance. Assuming that the appliance created is worthy, the devs will consider any patch using open source software for release next time the appliances are updated.
  4. I reckon that'd work but easiest way to be sure is to test! :)
  5. That should be it pretty much! How cool is that!?
Liraz Siri's picture

Liraz Siri - Mon, 2011/02/07 - 08:46

LVM snapshots: currently TKLBAM is not integrated with LVM. We put that into the release so it would be possible to do that in the future. If you can't wait, you have two options: try implementing the desired behavior is a tklbam hook, or take advantage of the power of open source and add this feature to TKLBAM yourself.

Full backups: the very definition of a full backup is that a new independent backup chain is created. It's impossible to do that without re-uploading everything. You can configure the frequency of full backups as you wish though.

We don't overwrite full backups: TKLBAM will absolutely not "flush out" the last full backup when creating a new one. Deleting backed up data automatically is dangerous. Corrupted data can easily go unnoticed for longer than the time between full backups. If we "overwrite" the last full backup automatically, you lose your ability to restore corrupted data.

How many full backups to retain should left to the user. We're going to soon be adding a feature to the Hub that helps you manage data retention. See this previous forum post for details.

PostgreSQL: note that only MySQL is currently supported by TKLBAM. Hopefully that will also be fixed soon.

Skipping testing: just don't. I know nobody is used to testing their backups, but TKLBAM was designed to make this easy enough so you wouldn't have an excuse. Always test your backups. Otherwise you may be too late to discover the gap between what you think TKLBAM should be doing and what it actually did.

Don Sanderson's picture

Don Sanderson - Mon, 2011/02/07 - 13:46

Thanks Liraz, I don't know how I missed that previous post

That clears things up.

Add new comment