TKLBAM: Backup of restored backup on another machine

Hello,

this is what I did:

1) I created a tkl-core (lucid RC) VM locally in VirtualBox. Recreated the content of a VM on my remote XenServer. Backed it up with TKLBAM.

2) I created a tkl-core (stable) on Amazon and restored the backup. Worked like a charm. Created a new backup from this instance.

3) Deleted the remote XenServer and installed a Proxmox server (Yay! Three cheers to the Proxmox team!).

4) Used the OpenVZ template of the TKL stable core (http://sourceforge.net/projects/turnkeylinuxovz) to create a TKL instance inside Proxmox. (BTW, it needs to be renamed to ubuntu-8.04.3-turnkey_2009.10_i386.tar.gz) in order for it to show up in the proxmox "create VM" menu. BTW2: It would be great to support OvenVZ templates in the regular release cycle of TKL).

5) Restored the Backup from the Amazon instance. Works without problems.

Now I want to get rid of the amazon instance and start backing up the proxmox instance. When I do

tklbam-backup

I get:

Traceback (most recent call last):
  File "/usr/bin/tklbam-backup", line 267, in
    main()
  File "/usr/bin/tklbam-backup", line 234, in main
    get_server_id())
  File "/usr/bin/tklbam-backup", line 110, in get_server_id
    return ServerConf()['serverid']
  File "/usr/lib/python2.5/site-packages/conffile.py", line 42, in __init__
    self.validate_required()
  File "/usr/lib/python2.5/site-packages/conffile.py", line 54, in validate_required
    raise ConfFileError(error)
conffile.ConfFileError: SERVERID not specified in /var/lib/hubclient/server.conf

I am not sure I fully understand the mechanism. When a restored backup is backed up again, is this supposed to go to the original backup or is the target a new backup? Which would make sense because otherwise there might be conflicting changes from the two instances that are derived from it. But that doesn't explain the exception.

Cheers,

Christian

Replies (10)

Yes they are very different

Reply 1 2010-11-08

That is why you can only run Linux OS in OVZ container, the core of an OS running in OVZ doesn't actually exist, it shares the kernel etc of the host OS. That is why VMs running under OpenVZ run so fast and use so few resources. :)

Thanks for that

Reply 2 2010-11-08

Thanks for that clarification. See, I had no idea. I thought they were just two different, but similar virtualization methods. So I'll try my luck with the KVM and let you know how it goes.

May be that TKLBAM & OpenVZ are not 100% compatible?

Reply 3 2010-11-08

I'm not sure and only guessing really. But I suspect that because there are significant differences between an OS running in a conventional VM environment and an OS running in a container virtualisation environment there may be some incompatabilities. Container virtualisation such as OpenVZ is basically an extension of, and elaboration on the idea of a chroot jail. Whereas a conventional virtualisation is a virtual environment with virtual hardware.

I don't know enough about much of this stuff to know how to troubleshoot this or where to even start looking. Although it may be useful to see what the TKLBAM backup actually contains, perhaps there's something obvious. I don't think the TKL devs have had much experience with OVZ (and don't yet officially support it) so may not be a lot of help but may be able to give you some ideas and head you in the right direction?

I suspect you'd have more luck if you restore to an instance of the appropriate appliance installed to KVM (you can still use Proxmox) - that should work flawlessly. Then if you want it as a OVZ container, convert the KVM install (using my instructions from the Dev wiki). Once you have that running succesfully, then it would be insteresting to try if you could backup and restore to/from an OVZ instance.

Good luck

Ok, seems like I was pleased

Reply 4 2010-11-08

Ok, seems like I was pleased to quickly.

I can restore the images from the backup and it works well if I just restart the servers running on the image. But once I reboot, the VM become unresponsive - does not answer to SSH / HTTP etc.

Darn! I thought I had it all figured out! I wonder if it is a problem with the OpenVZ template...

Any ideas? (I know I did not give enough details to be helpful).

There could theoretically be platform specific issues...

Reply 5 2010-11-10

As you can see from the follow up, this had nothing to do with OpenVZ. TKLBAM is designed to be hardware agnostic, but it's hard to know in advance if there isn't something we missed. The only way is to test exhaustively.

As I said I was only guessing :)

Reply 6 2010-11-09

I have nowherre near enough experience or knowledge to be sure about this (just enough to be dangerous!)

So this is possibly another issue? I understand that TKLBAM should be hardware agnostic (whether physical or virtual). I just thought that perhaps because OVZ is quite different then there may be something that doesn't work quite right (as apparently java doesn't, something to do with beancounters... over my head).

You know me Liraz, always willing to chuck my 2c in the ring. Sometimes turns out its 10 or 20c, other times its really only a button! :)

TKLBAM should be agnostic to virtualization

Reply 7 2010-11-09

It doesn't check what virtualization environment you are running in, and shouldn't really care one way or another if you are backing up OpenVZ or anything else. But... I haven't tested with OpenVZ yet, so we may have run into a new issue.

I have some TKLBAM maintenance overdue today. Will take a look at this with the other issues. Many thanks to Christian for reporting this.

A backup of a restored backup is a new backup

Reply 8 2010-11-09

Say you backup machine A, and then restore it to machines B and C. Then you backup B and C (perhaps after some customization).

B and C would both create new backups. This is the correct thing. Overwriting the backup for machine A would be dangerous because now you have three machines backing up to the same location and that can't be good.

I figured it out: nothing to do with OpenVZ

Reply 9 2010-11-10

I committed a fix to TKLBAM, but there's a very simple workaround you can use until we update the repository:

rm /var/lib/hubclient/server.conf

BTW, this has nothing to do with OpenVZ. It was a generic problem that would happen whenever you tried backing up an appliance restore from a backup of an Amazon EC2 instance.

The problem was that TKLBAM assumes if /var/lib/hubclient/server.conf exists, then it must have a SERVERID in it. This is always true for Amazon EC2 instances, because the Hub assigns them a SERVERID. But if you restore an Amazon EC2 backup to a non-Amazon EC2 instance, TKLBAM installs hubclient because it wasn't in the base installation profilem, and hubclient creates an empty server.conf in a misguided attempt to set permissions or something.

Using KVM this time and I am

Reply 10 2010-11-15

Using KVM this time and I am happy to report that all worked great! The migration from XenServer -> Amazon -> Proxmox/KVM is thus finished. It was a lot of work, but TKLBAM has really made it MUCH easier than it would have otherwise been, maybe without it I wouldn't have tried it in the first place. So thank you VERY much for your work on this!

Christian