TurnKey Linux Virtual Appliance Library

TKLBAM: Backup of restored backup on another machine

c.boulanger's picture

Hello,

this is what I did:

1) I created a tkl-core (lucid RC) VM locally in VirtualBox. Recreated the content of a VM on my remote XenServer. Backed it up with TKLBAM.

2) I created a tkl-core (stable) on Amazon and restored the backup. Worked like a charm. Created a new backup from this instance. 

3) Deleted the remote XenServer and installed a Proxmox server (Yay! Three cheers to the Proxmox team!). 

4) Used the OpenVZ template of the TKL stable core (http://sourceforge.net/projects/turnkeylinuxovz) to create a TKL instance inside Proxmox. (BTW, it needs to be renamed to  ubuntu-8.04.3-turnkey_2009.10_i386.tar.gz) in order for it to show up in the proxmox "create VM" menu. BTW2: It would be great to support OvenVZ templates in the regular release cycle of TKL).

5) Restored the Backup from the Amazon instance. Works without problems.

Now I want to get rid of the amazon instance and start backing up the proxmox instance. When I do 

tklbam-backup

I get:

Traceback (most recent call last):
  File "/usr/bin/tklbam-backup", line 267, in 
    main()
  File "/usr/bin/tklbam-backup", line 234, in main
    get_server_id())
  File "/usr/bin/tklbam-backup", line 110, in get_server_id
    return ServerConf()['serverid']
  File "/usr/lib/python2.5/site-packages/conffile.py", line 42, in __init__
    self.validate_required()
  File "/usr/lib/python2.5/site-packages/conffile.py", line 54, in validate_required
    raise ConfFileError(error)
conffile.ConfFileError: SERVERID not specified in /var/lib/hubclient/server.conf

I am not sure I fully understand the mechanism. When a restored backup is backed up again, is this supposed to go to the original backup or is the target a new backup? Which would make sense because otherwise there might be conflicting changes from the two instances that are derived from it. But that doesn't explain the exception.

Cheers,

Christian

Christian's picture

Ok, seems like I was pleased

Ok, seems like I was pleased to quickly.

I can restore the images from the backup and it works well if I just restart the servers running on the image. But once I reboot, the VM become unresponsive - does not answer to SSH / HTTP etc. 

Darn! I thought I had it all figured out! I wonder if it is a problem with the OpenVZ template...

Any ideas? (I know I did not give enough details to be helpful).

C. 

Jeremy Davis's picture

May be that TKLBAM & OpenVZ are not 100% compatible?

I'm not sure and only guessing really. But I suspect that because there are significant differences between an OS running in a conventional VM environment and an OS running in a container virtualisation environment there may be some incompatabilities. Container virtualisation such as OpenVZ is basically an extension of, and elaboration on the idea of a chroot jail. Whereas a conventional virtualisation is a virtual environment with virtual hardware.

I don't know enough about much of this stuff to know how to troubleshoot this or where to even start looking. Although it may be useful to see what the TKLBAM backup actually contains, perhaps there's something obvious. I don't think the TKL devs have had much experience with OVZ (and don't yet officially support it) so may not be a lot of help but may be able to give you some ideas and head you in the right direction?

I suspect you'd have more luck if you restore to an instance of the appropriate appliance installed to KVM (you can still use Proxmox) - that should work flawlessly. Then if you want it as a OVZ container, convert the KVM install (using my instructions from the Dev wiki). Once you have that running succesfully, then it would be insteresting to try if you could backup and restore to/from an OVZ instance.

Good luck

Christian's picture

Thanks for that

Thanks for that clarification. See, I had no idea. I thought they were just two different, but similar virtualization methods. So I'll try my luck with the KVM and let you know how it goes. 

Jeremy Davis's picture

Yes they are very different

That is why you can only run Linux OS in OVZ container, the core of an OS running in OVZ doesn't actually exist, it shares the kernel etc of the host OS. That is why VMs running under OpenVZ run so fast and use so few resources. :)

Liraz Siri's picture

TKLBAM should be agnostic to virtualization

It doesn't check what virtualization environment you are running in, and shouldn't really care one way or another if you are backing up OpenVZ or anything else. But... I haven't tested with OpenVZ yet, so we may have run into a new issue.

I have some TKLBAM maintenance overdue today. Will take a look at this with the other issues. Many thanks to Christian for reporting this.

Jeremy Davis's picture

As I said I was only guessing :)

I have nowherre near enough experience or knowledge to be sure about this (just enough to be dangerous!)

So this is possibly another issue? I understand that TKLBAM should be hardware agnostic (whether physical or virtual). I just thought that perhaps because OVZ is quite different then there may be something that doesn't work quite right (as apparently java doesn't, something to do with beancounters... over my head).

You know me Liraz, always willing to chuck my 2c in the ring. Sometimes turns out its 10 or 20c, other times its really only a button! :)

Liraz Siri's picture

There could theoretically be platform specific issues...

As you can see from the follow up, this had nothing to do with OpenVZ. TKLBAM is designed to be hardware agnostic, but it's hard to know in advance if there isn't something we missed. The only way is to test exhaustively.
Liraz Siri's picture

A backup of a restored backup is a new backup

Say you backup machine A, and then restore it to machines B and C. Then you backup B and C (perhaps after some customization).

B and C would both create new backups. This is the correct thing. Overwriting the backup for machine A would be dangerous because now you have three machines backing up to the same location and that can't be good.

Liraz Siri's picture

I figured it out: nothing to do with OpenVZ

I committed a fix to TKLBAM, but there's a very simple workaround you can use until we update the repository:
rm /var/lib/hubclient/server.conf
BTW, this has nothing to do with OpenVZ. It was a generic problem that would happen whenever you tried backing up an appliance restore from a backup of an Amazon EC2 instance.

The problem was that TKLBAM assumes if /var/lib/hubclient/server.conf exists, then it must have a SERVERID in it. This is always true for Amazon EC2 instances, because the Hub assigns them a SERVERID. But if you restore an Amazon EC2 backup to a non-Amazon EC2 instance, TKLBAM installs hubclient because it wasn't in the base installation profilem, and hubclient creates an empty server.conf in a misguided attempt to set permissions or something.

Christian's picture

Using KVM this time and I am

Using KVM this time and I am happy to report that all worked great! The migration from XenServer -> Amazon -> Proxmox/KVM is thus finished. It was a lot of work, but TKLBAM has really made it MUCH easier than it would have otherwise been, maybe without it I wouldn't have tried it in the first place. So thank you VERY much for your work on this!

Christian

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)