Mike Gifford's picture
Mike Gifford - Fri, 2011/11/25 - 18:37

I can see my backups here - https://hub.turnkeylinux.org/backups/

But not sure where they are going to in S3.  I think I should be able to see them here https://console.aws.amazon.com/s3/home but don't know what the bucket name would be.

Ultimately I want a way to pull down an instance from a backup and store it locally.  I might even want to burn it onto a DVD for old time sake.  

So is this possible within the Hub?

Forum: 
Support
Jeremy Davis's picture

Jeremy Davis - Sat, 2011/11/26 - 01:05

But you can do it with TKLBAM. If you wish to have a local backup and a remote (Hub) one you'll need to run 2 backups. A normal TKLBAM backup which will be seen and can be accessed by the Hub. And another to a local (or remote) location using the --address switch. For details see here: http://www.turnkeylinux.org/faq/backup-and-migration-tklbam#t601n2384

As for your questions regarding S3 bucket location, read here: http://www.turnkeylinux.org/faq/backup-and-migration-tklbam#t602n2374

Also if you plan to burn it to disk, then you may find the --volsize switch useful. As the name suggests it sets the volume size (in MB).

Mike Gifford's picture

Mike Gifford - Sat, 2011/11/26 - 13:57

I'd replied earlier, but think I didn't hit submit after preview.  By default most Drupal site's I'm involved with don't require that so I've gotten out of the habbit.  Very handy to know about the S3 buckets too..

These have both been really useful.  I took a bit of a simpler approach which seems to be working.

The instructions would have you mount a remote drive (possibly using something you've previously set up with something like sshfs).  It's much easier however to just do this which seems to work (at least when testing this as a newbie).

On the cloud server, create a directory, push the backup into it and then create a key:

root@drupal6 ~# mkdir /home/backup
root@drupal6 ~# tklbam-backup --address file:///home/backup/
root@drupal6 ~# tklbam-escrow /home/backup/key

Not eloquent, but then you can simply go to the command line in your VirtualBox instance and pull it down using old fashioned scp:
localhost ~# mkdir /home/backup
localhost ~# scp example-account.tklapp.com:/home/backup/* /home/backup
localhost ~# tklbam-restore --address=file:///home/backup --keyfile=/home/backup/key

This went through the process of replicating what I'd done on the server in a VirtualBox instance I'd set up from the OVF download in http://www.turnkeylinux.org/drupal6

From this I'm quite certain I should be able to simplify this process by just restoring Amazon from my VirtualBox instance.  Not exactly sure what I'd put into the options mind you.
 
 
 
Jeremy Davis's picture

Jeremy Davis - Sat, 2011/11/26 - 17:46

But you don't need to add a drive. TKLBAM uses Duplicity as a backend and as such you could use SFTP (SCP) direct to your VBox instance - although obviously you'd need to set up the keys first, but that is almost too easy IMO. You may not be interested but for completeness (for anyone else reading this) I'll include setting up the key too.

So set up a fresh key (I had a key already and I just overwrote it - I don't recall if the default appliance has keys set up?) :

root@core ~# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):<Enter>
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y <Enter>
Enter passphrase (empty for no passphrase): <Enter>
Enter same passphrase again: <Enter>
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
33:b3:fe:af:95:95:18:11:31:d5:de:96:2f:f2:35:f9 root@drupal6

You'll need to set up incoming internet connectivity to your VBox instance on port 22 (which may involve port forwarding). We'll assume that you've done that and are running it with an example.tklapp.com FQDN

root@core ~# ssh-copy-id root@example.tklapp.com
The authenticity of host 'example.tklapp.com (102.158.12.68)' can't be established.
RSA key fingerprint is 53:73:6b:0e:46:5e:1d:c0:16:98:dd:a0:3e:e6:f7:78.
Are you sure you want to continue connecting (yes/no)? yes <Enter>

Warning: Permanently added 'example.tklapp.com' (RSA) to the list of known hosts.
Now try logging into the machine, with "ssh 'root@example.tklapp.com'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

I double check that it works:

ssh root@example.tklapp.com

Which should log you straight into the machine no password etc required...

Now to run your backup direct to the VBox machine. Assuming you wish the backup to be stored in ~/backup, do this:

tklbam-backup --address scp://root@example.tklapp.com/backup

If the directory doesn't already exist it will be created. The beauty of this, is that if/when you want it to run again you only need to use the last line.

Obviously though you miss out on the advantage of being able to run a new cloud server with your backup already included. For that to work you need to use S3.

Mike Gifford's picture

Mike Gifford - Sat, 2011/11/26 - 22:03

Wanted to say I'm getting a lot out of your responses.  Really helping me get the environent set up.

One thing that could have been clearer is which server am I on & adding clarification around that in the commands you're providing.

I walked through your examples (and thanks for fleshing this out completely to beef up the docs).  And it's:

 

root@example.tklapp.com ~# ssh-keygen

and then on the virtual box you are backing up to the cloud server (and not S3 which would be the default):

root@Vbox ~# tklbam-backup --address scp://root@example.tklapp.com/backup
It's just easy to loose track of where one is and where files are being put. Is there supposed to be a : in the address above before the directory?  That would be more normal for scp.  
 
We're looking to improve the documentation too as I'm going to have to explain this to people when we adopt this approach.
Jeremy Davis's picture

Jeremy Davis - Mon, 2011/11/28 - 22:55

I was assuming that you have a TKL appliance running on AWS (via the Hub) and you wish to back that up to a local VBox instance. And it is the AWS instance in which you run all the commands. Although it doesn't need to be an AWS instance and the target doesn't need to be running on Vbox, it could be from any instance to any other one, as long as they have network connectivity (and you allow firewall exceptions and/or forward ports if/as need be - obviously you may need to substitue IP addresses for FQDNs - depending on your config).

If you plan to restore your backup to an AWS instance though you are much better off using the TKLBAM defaults as that will allow you to launch a new instance and have the backup auto restore on boot (from the Hub). These istructions work fine though if you want to also have a local copy of your backup. Also the S3 cloud storage is pretty cheap and is more reliable that a standard HDD.

Mike Gifford's picture

Mike Gifford - Mon, 2011/11/28 - 18:46

This is good feedback.  Like all of these online services however, it's only as good as it continues to be offered by Amazon. There's still no bill of rights for clients to see that they don't shut down the server tomorrow.

That being said, it's highly unlikely that this would happen or happen without sufficient time to move it elsewhere.  Just always a concern with any 3rd party service.

Jeremy Davis's picture

Jeremy Davis - Mon, 2011/11/28 - 23:06

But there's no reason why you couldn't have both. As long as your local VBox instance is available at all times, you could leave the standard S3 backup running but also setup a cron job to run your backup to local storage. In fact it doesn't even need to be a TKL instance that you backup to. You could set up a SFTP server on a Win box if you wanted to and that should also work fine (although I have no idea how you'd set up the keys and stuff).

Add new comment