xtrac568's picture

Hey,

Some toys for the openvz users. These are .tar.gz templates for deployment of turnkey appliances on openvz platform.

Get them here sourceforge.com/projects/turnkeylinuxovz 

You'll find templates for all 2009.10 releases (uploading to sourceforge still in process..) except those based on tomcat/java (e.g. openbravo, zimbra), because of known memory issues with openvz and java. However, there are ways to tweak these appliances so they can work properly. I've made some possible configuration tweaks, and once it's confirmed to play nice, there is possibility to publish these appliances. I'm not sure about this yet, since then it's are not pure Turnkey Linux appliance but somewhat tweaked one.

Worth pointing out, "inithooks" are set to run on firstboot to regenerate secrets, and "confconsole" is removed, since there's no use for it on openvz.

Any questions, feel free.

Forum: 
Jeremy Davis's picture

Beat me to it! This is something I've had in mind for a while, but another thing I just haven't got around to yet... Actually I was considering producing a generic script that users could use on any TKL appliance to create their own OpenVZ template, but I abandoned that because I'm not a great scripter and with error handling etc, it was just getting bigger and more painful to use.

Sounds like you know your way around OpenVZ!? Are there any other tweaks you've made to the templates? How does your conversion from .iso to OpenVZ template compare to the instructions I posted on the Dev Wiki? I'd be really interested to hear as I'm a real Linux noob and bashed those instructons together through lots of reading and trial and error and (somehow) managed to get it all working very nicely.

Also for your interest, I have found that I can use confconsole fine if I allow my template to get assigned IPs via DHCP, whereas it is useless if I hardcode the IP with OpenVZ (confconsole tries to configure the wrong interface). For most appliances I use a hardcoded IP but I noticed some things don't play nice with a hardcoded IP when inside an OpenVZ container (eg BIND9) and will only work with a DHCP supplied address. I used the confconsole to get around that and set a static IP (although I know I could've just edited the config file).

xtrac568's picture

Hi

I've done it using more or less the same openvz recommended guidelines as you. My approach is that files are rsynced from physical server first, after you do necessary changes for container to boot, and then complete further openvz guidelines for standard template creation (e.g. service removal, first boot procedures) but the result should be the same. Parallels Virtuozzo feature "Migrate server to container" uses the same procedures.

Also i left the system state as much as possible to turnkey default, e.g. not updating packages, this is left for the user to decide whether to do updates himself or use turnkey auto-update.

What you can add to your procedures which is recommended, is to regenerate ssh keys on firstboot, or more useful run inithooks on boot, which takes care of ssh keys, ssl certs and other appliance specific actions.

On confconsole, I'm was more oriented to openvz provider environment where static ips are set outside of container (e.g. vzctl), so then confconsole becomes sufficient and can produce troubles in certain setups as you've said.


Liraz Siri's picture

I ask because a manual conversion process would add significant overhead to our release process. It's also error prone. We need to figure out how to automate this stuff. We could probably leverage tklpatch to do that. It's modular so we can write a simple script that uses only the parts we want (e.g., extracting the filesystem out of the ISO, applying OpenVZ related tweaks).
Liraz Siri's picture

First off, marvelous work! This is exactly the kind of independent community-driven initiative we need to help push TurnKey Linux forward. Alon and I have a todo list from here to the moon and back.

We don't have an OpenVZ testing environment set up at the moment so confirmation from the community that all is well with these appliance images will be very helpful. I know JedMeister is an OpenVZ fiend so I imagine he'll have some input to share about that.

Another thing that would be helpful is as much detail as you care share about the conversion process you used. If there is scripting involved maybe we can set up a repository for it on GitHub (or just edit your message and attach the current version). That way if any tweaks are required you won't have to republish all the images again, we can just tweak the script and re-convert. Also users can do their own conversions (e.g., when new appliance versions come out), and it will also make it easier for us to add the conversion to our release process, and offer OpenVZ builds as an official download option, once we've worked out all the issues...

xtrac568's picture

An shell script to automate creation is very doable, I've already wrote shell scripts to automate universal openvz template creation (which i also used in this process), however scripts are platform dependent, they rely on openvz host and rsync from physical as i explained.

What can be done that creation process is faster and more specific, e.g.

-have raw files all of appliances

-make necessary changes to files for ovz container to boot

-start ovz container (vzctl)

-remove services, firstboot procedures (vzctl exec)

-stop container (vzctl)

-create tar.gz

It's not necessary to rely on openvz host at all, but the example above is useful if you want to verify that openvz container is working correctly during creation.


xtrac568's picture

Another thing why it is useful to do all this within openvz host. If you want to make some updates to the existing template, and if these changes cannot be made directly editing template files, but must be done within running system instead.

you do the following with shell script aswell.

-create container with existing template (vzctl create)

-start container (vzctl start)

-make updates (vzctl exec)

-stop container (vzctl stop)

-create new tar.gz

And you have new updated template automatically created.


Liraz Siri's picture

It would be better to have just a really simple script that can run any where to convert ISOs to OpenVZ templates rather than an arrangement involving an OpenVZ server because adding a script to our build infrastructure is easier than adding another virtualization platform. So maybe we can verify that everything is working using OpenVZ but not depend on it for the actual conversion.
xtrac568's picture

I'm not sure what procedures your build platform does, but an option could be that you automatize openvz template creation using JedMeister's approach.

Certain openvz template requirements you can complete by customizing ISO files, but at some point you need to boot the appliance and remove services, such as udev, on an running appliance. If you know the way to remove these services on an ISO without booting appliance, then you can make pure ISO to Openvz template script.


Jeremy Davis's picture

 That's how I created my latest batch of templates. As I think I said somewhere else, I was considering releasing my script but it I was unhappy with it. I wanted to make it user friendly, but probably more importantly reliable (error handling etc). As I am a Linux noob that was taking more effort than I could muster so it died a natural death. As you say though it still requires a manual install (to edit services) and then copy out the tar.gz.

Surely there must be a way to edit services in a non running instance of TKL (/Debian/Ubuntu)?

 

Todd Forsberg's picture

Sorry for being lazy.  I didn't read the whole thread in detail.  Just skimmed thru it.

Have any of the v11 TKL appliance been successfully converted to OVZ?

-Todd Forsberg

Jeremy Davis's picture

But there is a thread here that discusses converting ISOs to OVZ templates. Its a fair bit of a read and a little mucking around but worth the effort IMO. If that all seems like too much, then let me know which one(s) you are after and I can convert them for you and upload them somewhere.

Todd Forsberg's picture

I'm really just in need of the Revision Control and MediaWiki at the moment.  If you don't mind trying to convert those 2 that would be awesome.  I have had a go at applying patches.  I was able to create a Gitorious ISO, but could not get it working.  I don't know if I messed up the patch process or if I just don't know what needs to be configured with Gitorious...  I'm still learning all this.

Where did you have in mind to upload?  I have an FTP server, I can give ya access if ya send me a private msg, I'll respond with credentials for you.

-Todd

-Todd Forsberg

Jeremy Davis's picture

I've uploaded the templates to SourceForge. Have a look here. Let me know how you go with them.

Also you'll need to rename them to get PVE to recognise them.

Todd Forsberg's picture

I dloaded both Revision Control and MediaWiki and installed to Proxmox.  Both boot up fine, but RevControl does not seem to be running some services.

When trying to clone the sample repo "helloworld" I get this error:

 fatal: '/srv/repos/git/helloworld.git' does not appear to be a git repository
 fatal: The remote end hung up unexpectedly

-Todd Forsberg

Jeremy Davis's picture

I think this may be associated with Ubuntu's move towards upstart in Lucid. I don't personally use Revision Control but I will try to have a look when I get a chance. Adrian is probably the man for the job though. His Linux know-how is miles ahead of mine and AFAIK he is currently using OVZ containers. I'm not sure if he actually uses the Revision Control appliance but knowing him I'm sure he has some git stuff running under OVZ so he may be able to shed some light on the subject.

I have posted over on the other thread (here) which will hopefuly get his attention. I guess really though I should probably start a whole new thread announcing the 'community' uploads to keep it all together. I'll try to do that later if I have time.

Todd Forsberg's picture

I noticed also that on the Revision Control appliance even the WebMin is not working.  Looks like the apache server may not be running.  I'm not sure how to check that or how to try to start it manually.

-Todd Forsberg

Jeremy Davis's picture

Also noticed that SourceForge is saying that it has been downloaded 0 times which seems strange. Perhaps the download was corrupted?

[update] I can confirm that this appliance is definately not working as it should. Initially I could get no http or https connection to it at all. I restarted Apache (service apache2 restart) and it seems ok, although it is again broken after a reboot. Coeection it is even more broken after a reboot. Apache is now refusing to start at all. Also Webmin doesn't want to run at all - although it says it starts (/etc/webmin/start). I will try a fresh conversion and let you know how I go.

As an aside, SF still says 0 downloads...

Jeremy Davis's picture

Not sure what went wrong with the other one but I've just recreated the template and this one seems to work ok (although I haven't tested extensively). The default website works as does Webmin, so that's a damn sight better than before :)

Download the fresh one and see if it works ok for you.

Todd Forsberg's picture

Ya, this one works fine.  

After initial testing, I restored a TKLBAM backup to this vm.  Then all hell broke loose.  No services again.

So for now, I will migrate my stuff manually.  The OVZ template does work.  Maybe there is some compatibility issue with restoring from an official TKL appliaance.  Not sure yet, so lets not draw conclusions.

Anyways, next thing I did was to try the TorrentServer.  Same issue as the first (v0) release of the Revsion Control (for OVZ).

My guess is that the whole first batch of appliances may have the same problem.  Not sure just guessing, since I only tried two.

-Todd Forsberg

Jeremy Davis's picture

Ok, not sure what's going on there. I'm pretty sure I'm using the torrent server at home ok - that's why I created the template in the first place (but perhaps it's just installed from ISO under KVM - can't be 100% sure until I check it out?)

There is obviously something really wrong going on. It deserves further investigation but I won't have a chance for at least a week.

Sorry about all this pain, but thanks for testing.

Todd Forsberg's picture

The TKLBAM restore worked fine (from an official TKL appliance to an OVZ vm).  I forgot to reboot the vm after.

SUGGESTION:  Maybe TKLBAM should prompt the user to reboot after a restore operation.

-Todd Forsberg

Liraz Siri's picture

I'm not sure about prompting for a reboot. I mean, if you're already on the command line and you want to reboot you just execute "reboot". The tricky part is remembering that you might need to do that which is why at the end of the restore TKLBAM prints this message:
We're done. You may want to reboot now to restart all services.
Todd Forsberg's picture

I guess I did not read the screen carefully (as I have found many user do not).  So, even developers are subject to the same issues.

-Todd Forsberg

Adrian Moya's picture

Hi guys, I'm sucessfully running a lot of TKLv11 images on proxmox, including Source Control & Torrent Server, and some custom appliances. I'll upload my templates to the sourceforge project, Jed already gave me access, and check while I'm home if there's any extra tweak I did to make them happen. We can collect the tips on a wiki page.

I think we can open a new thread with specific problems of each appliance, as this thread is not only old but it's not very easy for others looking info about the topic.

@Todd: I had those issues with services with my first templates, but latelly I manage to fix them all, which other templates are you planning to use?

Todd Forsberg's picture

Do ya have Gitorious as an OVZ template?  If not, can ya post your ISO for Gitorious on the SF TKL Community site.

I would also like to use Torrent Server in a OVZ format.

Many more too, but those can wait a bit.

Thanks.

-Todd Forsberg

Add new comment