TKLBAM: Confusion with multiple Passphrase and Escrow keys

Hi there,

I've been trying set up TLKBAM with S3 for a while now and have it working, but not without some confusion. I've read the documentation and followed the man pages but am having serious problems understanding the security measures, namely the difference between and purpose of the passphrase and escrow keys.

If I follow the man pages in order, it takes me through the following steps:

(Note: All of the following was done using the TLK Redmine 11.1 appliance)

# Step 1
tklbam-init [ API-KEY ]

No problems there

# Step 2
tklbam-passphrase [-options]

Here I used the '--random' flag to generate a random passphrase which it printed to the screen (as expected)

# Step 3
tklbam-escrow [ -options ] KEYFILE

The description here says: "Creates an escrow key you can pass directly to the restore command. Save this somewhere safe."

Arguments: "KEYFILE: File path to save the escrow key (- for stdout)"

As I wasn't sure where the keyfile should go, I decided to use '-' expecting it to print it to stdout. However, invoking this command as:

tklbam-escrow -

tklbam-escrow --random-passphrase -

just prints out the help text. I eventually decided to just use

tklbam-escrow --random-passphrase /path/to/some/dir/key

and it seemed as though it worked ok, printing out another passphrase to stdout.

So at this stage I now have 2 different passphrases and an escrow key. As I wanted to back up the keyfile to my local drive, I copied it using down scp. However, when I go into the Webmin interface it also gives me the option to download the escrow key. As a matter of curiosity I did so and compared it to the key I downloaded using scp and seen that they are not the same.

# Step 4
tklbam-backup

I proceeded to run a backup and all went ok

# Step 5
tklbam-list

A quick check and it's all there

# Step 6
tklbam-restore:

I wanted to test the backup so I installed a fresh TKL appliance VM using VirtualBox and this time decided to do the TLKBAM configuration and restore operation through the Webmin interface rather than the command line.

After entering the API key in the Webmin interface (on the new fresh VM) I was surprised to see that it also gave me the option to download the Escrow key (thinking perhaps it was able to retrieve it from the hub) but on downloading I could see that this was also different from the 2 I had previously downloaded from the original appliance.

Ignoring this I chose to restore my previous backup to this fresh VM. It asked me for the passphrase so I entered the first one generated from step 2 above.

The passphrase was accepted and the backup seemed to go ok. I rebooted the appliance but on reboot it asked me to enter the password for the root account again. It did the same for the MySql root account and Redmine admin account (as it would during a fresh appliance install)

It also asked me for the TLKBAM API Key again. When I entered this it gave a quick dialog saying something along the lines of "linking account", but quickly switched to another dialog saying

"error: already initialized".

Selecting ok, brought me back to the dialog to enter the API key so this time I chose to 'skip'.

Next was the dialog for installing security updates so I agreed to that but this completed very quickly (unlike the fresh install).

Going for a test I could see that phpmyadmin and another web application (both of which were installed separately from the original appliance default applications) were there and working as expected. Great! However, when I tried to test Redmine, (installed as part of the original appliance), using the URL 'http://[ip.address.of.vm]/railsapp/public' it failed with:

The requested URL /railsapp/public/dispatch.cgi was not found on this server.

Since I originally changed the apache config for Redmine from www.mydomain.com to redmine.mydomain.com, I thought this might be the issue so I changed my hosts file to point redmine.mydomain.com to the ip address of the new Virtualbox VM. This time it worked as expected so all is good. Very, very impressive. Turnkey and TKLBAM will definitely be part of my workflow from now on :)

However there are some things I am still unsure of.

Most importantly:

1. Why do I have 2 different passphrases and 3 different escrow keys?

2. What is the purpose of each?

3. What exactly is the Escrow Key used for?

4. Does is matter that I have stored my Escrow key on the same volume that is being backed up to the S3 storage?

Also:

5. Why did the appliance ask me for all of the root/admin passwords again on reboot?

6. What would have happened if I had chose different passwords on boot this time?

And if you have time:

7. Is setting the host file as I have done above the best method to test subdomains like this or is there a preferred method people use which allows them to subdomains is these cases no matter what the ip address is?

Apologies for the verboseness of the post and thank you for all the brilliant work!

Jack

Replies (5)

I'll look into that

Reply 1 2013-01-08

Thanks Liraz,

I'll have a read of that and give it a go.

It looks like the boot

Reply 2 2013-01-08

It looks like the boot process failed inside the initramfs. I don't think that can happen from a screwed up restore. More likely something to do with your external drive not being accessible to initramfs. Debugging this can be tricky. Your best bet is to try and mount your root filesystem by hand from within the initramfs rescue shell. That should give you a clue to what is going wrong.

This may or may not help you:

http://wiki.debian.org/InitramfsDebug

Updated behaviour

Reply 3 2013-01-08

Update:

Just after submitting the above post I found that I could no longer reach the VM with the browser. I rebooted the VM and was presented with a dialog saying that the network wasn't configured, giving me options to configure with DHCP or manually. Selecting the DHCP option gave and error saying that /etc/network/interfaces was read only. It also showed the login: text coming through the blue dialog but it didn't let me. I switched to a new terminal (Ctrl+Alt+F2) and logged in from there. A couple of seconds later it logged me out again and I couldn't log in. I restarted the VM and not it won't boot up properly. The last thing I see on the screen is:

    [ 5.662745] Console: switching to colour frame buffer device 80x30
No init found. Try passing init= bootarg.


BusyBox v1.13.3 (Ubuntu 1:1.13.3-1ubuntu11) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

All other terminals are blank except F7, whose messages end with:

Begin: Running /scripts/init-bottom ...
mount: mounting /dev on /root/dev failed: No such file or directory
Done.
mount: mounting /sys on /root/sys failed: No such file or directory
mount: mounting /proc on /root/proc failed: No such file or directory
Target filesystem doesn't have /sbin/init.

I'm not sure what's happening here but it should be noted that my VM is running from an external drive. Is it possible I messed up the TKLBAM restore somehow?

Any suggestions appreciated,

Jack

That's definitely clearer

Reply 4 2013-01-08

Hi Liraz,

Thanks very much for your response. That's definitely cleared up the passphrase confusion. I know a number of other people here have been similarly confused. That's pretty much along the lines of what I was thinking but it wasn't clear from the documentation.

Could I just trouble you a little more by asking if your could shed some light on why the escrow key downloaded from Webmin differs from the one I created and copied manually via SCP?

Thanks,

Jack

How there can be more than one passphrase

Reply 5 2013-01-08

If you use tklbam-passphrase to passphrase protect a backup you can't restore it from another machine without the passphrase because the encryption key is stored by the Hub encrypted with your passphrase. Again, tklbam-passphrase sets a passphrase on the key stored in the Hub.

So say you backup a TurnKey appliance just before someone drops a nuke on your datacenter. From the shock of it all you've forgetten your passphrase. You are now officially screwed. Unless... you have an escrow key! You can use the escrow key to restore your backup even if you've forgotten the passphrase to the key stored by the Hub.

But wait! What happens if an evil hacker breaks into your computer and steals your Hub API key and finds where you kept the escrow key?! Will they be able to restore your super secret backup? Yes they will. Unless the escrow key is itself protected with a passphrase. You don't have to protect your escrow key with a passphrase. It's just possible.

Bottom line, there are two possible passphrases:

the passphrase controlled by tklbam-passphrase which encrypts the key stored on the Hub. This is optional. The key to a backup is downloaded transparently from the Hub when you restore a backup.
the passphrase to your escrow key. Unlike the key stored in the Hub, the escrow key you have to manage yourself. That means you have to store it somewhere safe and copy the file to the computer where you'll use it to restore your encrypted backup.

Come to think of it I can see how this would be confusing to users so I'll have to think about maybe removing the option to have a passphrase on the escrow key in future versions of TKLBAM. Old passphrase protected escrow keys will still work of course.

If users really want to encrypt their escrow key maybe we should just do it with gpg. Hope this helps clear up some of the confusion...