TurnKey Linux Virtual Appliance Library

Precreated OpenVZ template caches

Hey,

Some toys for the openvz users. These are .tar.gz templates for deployment of turnkey appliances on openvz platform.

Get them here sourceforge.com/projects/turnkeylinuxovz 

You'll find templates for all 2009.10 releases (uploading to sourceforge still in process..) except those based on tomcat/java (e.g. openbravo, zimbra), because of known memory issues with openvz and java. However, there are ways to tweak these appliances so they can work properly. I've made some possible configuration tweaks, and once it's confirmed to play nice, there is possibility to publish these appliances. I'm not sure about this yet, since then it's are not pure Turnkey Linux appliance but somewhat tweaked one.

Worth pointing out, "inithooks" are set to run on firstboot to regenerate secrets, and "confconsole" is removed, since there's no use for it on openvz.

Any questions, feel free.

Jeremy's picture

Nice work!

Beat me to it! This is something I've had in mind for a while, but another thing I just haven't got around to yet... Actually I was considering producing a generic script that users could use on any TKL appliance to create their own OpenVZ template, but I abandoned that because I'm not a great scripter and with error handling etc, it was just getting bigger and more painful to use.

Sounds like you know your way around OpenVZ!? Are there any other tweaks you've made to the templates? How does your conversion from .iso to OpenVZ template compare to the instructions I posted on the Dev Wiki? I'd be really interested to hear as I'm a real Linux noob and bashed those instructons together through lots of reading and trial and error and (somehow) managed to get it all working very nicely.

Also for your interest, I have found that I can use confconsole fine if I allow my template to get assigned IPs via DHCP, whereas it is useless if I hardcode the IP with OpenVZ (confconsole tries to configure the wrong interface). For most appliances I use a hardcoded IP but I noticed some things don't play nice with a hardcoded IP when inside an OpenVZ container (eg BIND9) and will only work with a DHCP supplied address. I used the confconsole to get around that and set a static IP (although I know I could've just edited the config file).

RE: hard coded IP addresses.

I am trying to install some of the containers from here on my proxmox box. being new to it I need to know how to set it up using static IP addresses. I don't know confconsole, and don't know how to access it from the cli, so am unable edit it. how did you disable the DHCP from this?

 

thanks for your time

Peter.

Jeremy's picture

OpenVZ - Set static IP with Proxmox

You can set a static IP from the Proxmox WebUI when you create the machine. I don't recall exactly how its done now but I will try to check for you later this arvo (when I can get access). I don't think it works once the VM has been created though (although I could be wrong).

Also you can start the TKL conf console easily. Asuming you are root, at the CLI prompt type:

/usr/bin/confconsole

thanks you for responding so

thanks you for responding so fast.

 

yes I can set the IP in the webui, and it is venet. (and yes you can change the venet ip address when its running)  the problem I am having is I am unable to access the VE though the IP that I set. only through the proxmox console.

 

this is the etc/network/interfaces file from a fresh non booted ubuntu-8.04.3-i386-turnkey-rails-2009.10.tar.gz template

***********************

# This configuration file is auto-generated.
# WARNING: Do not edit this file, otherwise your changes will be lost.
# Please edit template /etc/network/interfaces.template instead.
# Auto generated interfaces
auto lo
iface lo inet loopback
auto venet0
iface venet0 inet static
    address 127.0.0.1
    netmask 255.255.255.255
    broadcast 0.0.0.0
    up route add -net 192.0.2.1 netmask 255.255.255.255 dev venet0
    up route add default gw 192.0.2.1
*****************************

 

this is the etc/network/interfaces file from a fresh non booted debian-5.0-standard_5.0-2_i386.tar.gz template

***********************

# This configuration file is auto-generated.
# WARNING: Do not edit this file, your changes will be lost.
# Please create/edit /etc/network/interfaces.head and /etc/network/interfaces.tail instead,
# their contents will be inserted at the beginning and at the end
# of this file, respectively.

# Auto generated lo interface
auto lo
iface lo inet loopback
**************************

this is the etc/network/interfaces file from a fresh booted ubuntu-8.04.3-i386-turnkey-rails-2009.10.tar.gz template

***********************

# This configuration file is auto-generated.
# WARNING: Do not edit this file, your changes will be lost.
# Please create/edit /etc/network/interfaces.head and /etc/network/interfaces.tail instead,
# their contents will be inserted at the beginning and at the end
# of this file, respectively.
#
# NOTE: it is NOT guaranteed that the contents of /etc/network/interfaces.tail
# will be at the very end of this file.

# Auto generated lo interface
auto lo
iface lo inet loopback

# Auto generated venet0 interface
auto venet0
iface venet0 inet static
    address 127.0.0.1
    netmask 255.255.255.255
    broadcast 0.0.0.0
    up route add -net 192.0.2.1 netmask 255.255.255.255 dev venet0
    up route add default gw 192.0.2.1
auto venet0:0
iface venet0:0 inet static
    address 192.168.1.112
    netmask 255.255.255.255
    broadcast 0.0.0.0
******************

I have edited/added/removed everything in the network folder and still am coming up with this problem.

also the TKL conf console is missing.

this is my home server so it doesn't have a public IP, my routers is set at 192.168.1.1  dhcp scope is 192.168.1.80-99 and I have static ip's above with the proxmox host at .100 and containers .101-125

as you can see the "up route add -net 192.0.2.1 netmask 255.255.255.255 dev venet0
    up route add default gw 192.0.2.1" dont match my network info. and it is auto generated somewhere and now I'm trying to track it down.

sorry if this info is fragmented, was a long day.

thanks for all your help!!!

Peter

Jeremy's picture

I can't help much more until I get home

I have just had a look if I can connect remotely from where I am at the moment. I can get the main interface but not the machines (all connections to remote ports other than http & http are blocked & PVE uses 5900+ for the remote VNC connection).

I'm starting to understand all this a little better than my earlier posts (from last year). Although TKL Conf console doesn't auto start you can still use it. But unless you are using veth it won't be any use to you (confconsole doesn't seem to see venet and instead tries to adjust your loopback interface, which isn't very useful). For interest sake, to get it running, at the commandline type:

/usr/bin/confconsole

I'm guessing from what you've posted that the IP you are trying to use/connect on is 192.168.1.112? I'll try to remember to check the interfaces file of one of my working machines tonight and post that here so you have something to compare to.

In the meantime, perhaps try restarting the machine after having set the venet IP in the PVE UI perhaps that will work. If you're urgent to get it going, try using veth instead (I think you'll need to recreate the machine to do that) and set the static IP with confconsole (start it using the above command).

RE: thanks you for responding so

These interfaces configs looks alright and dont worry about 192.0.2.1 gateway setting, because it does not have any effect when using Venet, since next hop is always server node.

Can you provide us with output of "ifconfig" command of running container once static ip is configured from proxmox?

 

Also, this is exact reason why i removed "confconsole" from these templates. Openvz uses Venet by default, and ppl find references to confconsole and try to use it to configure network on Venet container and the result is messed up networking.

If you really want to use confconsole, setup Veth networking instead of Venet and then install confconsole "sudo apt-get install confconsole".


sorry for the delay here 

sorry for the delay

here  is the ifconfig output

root@test2:/# ifconfig
lo      Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
        inet6 addr: ::1/128 Scope:Host
        UP LOOPBACK RUNNING  MTU:16436  Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:127.0.0.1  P-t-P:127.0.0.1  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)


venet0:0  Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:192.168.1.112  P-t-P:192.168.1.112  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
 root@test2:/#

 

also I do have net access because I can apt-get update. but I am still not able to access it va the IP I gave it 192.168.1.112

 

Peter

RE: sorry for the delay here

Hi,

As far as i can see, interfaces are configured correctly inside cointainer, and are working, since network is usable from within container as you say.

But i'm still not sure why you cannot access container from outside.

 

JedMeister, you are using Proxmox right? Any ideas whether this could be caused by some Proxmox settings?


Hey, What kind of network

Hey,

What kind of network interface does Proxmox use for Openvz by default, venet or veth?

When you install OpenVz on a clean server it will use venet by default, however it might be different on Proxmox.

Venet interface will not assign ips by DHCP, and you have to setup static ips from server node itself, using "vzctl set VEID --ipadd x.x.x.x --save"

Veth is a bridged interface and you can have ips assigned by DHCP and to setup static ip, you do it from container itself.


Jeremy's picture

It defaults to venet

I forgot to check yesterday. I'll try to remember to double check later.

I think this is how it is...

When you create a new machine (using the Proxmox WebUI) it defaults to venet with an IP of 127.0.0.1, you can change that to whatever static IP you desire (the Proxmox UI probably runs the command you suggest behind the scenes). I have found though that some applications (such as BIND and DHCP) do not like using venet and the only way I could get them to work properly is by using veth.

Hi I've done it using more or

Hi

I've done it using more or less the same openvz recommended guidelines as you. My approach is that files are rsynced from physical server first, after you do necessary changes for container to boot, and then complete further openvz guidelines for standard template creation (e.g. service removal, first boot procedures) but the result should be the same. Parallels Virtuozzo feature "Migrate server to container" uses the same procedures.

Also i left the system state as much as possible to turnkey default, e.g. not updating packages, this is left for the user to decide whether to do updates himself or use turnkey auto-update.

What you can add to your procedures which is recommended, is to regenerate ssh keys on firstboot, or more useful run inithooks on boot, which takes care of ssh keys, ssl certs and other appliance specific actions.

On confconsole, I'm was more oriented to openvz provider environment where static ips are set outside of container (e.g. vzctl), so then confconsole becomes sufficient and can produce troubles in certain setups as you've said.


Liraz Siri's picture

Ah so the conversion process is still manual?

I ask because a manual conversion process would add significant overhead to our release process. It's also error prone. We need to figure out how to automate this stuff. We could probably leverage tklpatch to do that. It's modular so we can write a simple script that uses only the parts we want (e.g., extracting the filesystem out of the ISO, applying OpenVZ related tweaks).
Liraz Siri's picture

A few ideas

First off, marvelous work! This is exactly the kind of independent community-driven initiative we need to help push TurnKey Linux forward. Alon and I have a todo list from here to the moon and back.

We don't have an OpenVZ testing environment set up at the moment so confirmation from the community that all is well with these appliance images will be very helpful. I know JedMeister is an OpenVZ fiend so I imagine he'll have some input to share about that.

Another thing that would be helpful is as much detail as you care share about the conversion process you used. If there is scripting involved maybe we can set up a repository for it on GitHub (or just edit your message and attach the current version). That way if any tweaks are required you won't have to republish all the images again, we can just tweak the script and re-convert. Also users can do their own conversions (e.g., when new appliance versions come out), and it will also make it easier for us to add the conversion to our release process, and offer OpenVZ builds as an official download option, once we've worked out all the issues...

An shell script to automate

An shell script to automate creation is very doable, I've already wrote shell scripts to automate universal openvz template creation (which i also used in this process), however scripts are platform dependent, they rely on openvz host and rsync from physical as i explained.

What can be done that creation process is faster and more specific, e.g.

-have raw files all of appliances

-make necessary changes to files for ovz container to boot

-start ovz container (vzctl)

-remove services, firstboot procedures (vzctl exec)

-stop container (vzctl)

-create tar.gz

It's not necessary to rely on openvz host at all, but the example above is useful if you want to verify that openvz container is working correctly during creation.


Another thing why it is

Another thing why it is useful to do all this within openvz host. If you want to make some updates to the existing template, and if these changes cannot be made directly editing template files, but must be done within running system instead.

you do the following with shell script aswell.

-create container with existing template (vzctl create)

-start container (vzctl start)

-make updates (vzctl exec)

-stop container (vzctl stop)

-create new tar.gz

And you have new updated template automatically created.


Liraz Siri's picture

OTOH, OpenVZ is hard to integrate with our build infrastructure

It would be better to have just a really simple script that can run any where to convert ISOs to OpenVZ templates rather than an arrangement involving an OpenVZ server because adding a script to our build infrastructure is easier than adding another virtualization platform. So maybe we can verify that everything is working using OpenVZ but not depend on it for the actual conversion.

I'm not sure what procedures

I'm not sure what procedures your build platform does, but an option could be that you automatize openvz template creation using JedMeister's approach.

Certain openvz template requirements you can complete by customizing ISO files, but at some point you need to boot the appliance and remove services, such as udev, on an running appliance. If you know the way to remove these services on an ISO without booting appliance, then you can make pure ISO to Openvz template script.


Jeremy's picture

Yeah my approach is easily scriptable.

 That's how I created my latest batch of templates. As I think I said somewhere else, I was considering releasing my script but it I was unhappy with it. I wanted to make it user friendly, but probably more importantly reliable (error handling etc). As I am a Linux noob that was taking more effort than I could muster so it died a natural death. As you say though it still requires a manual install (to edit services) and then copy out the tar.gz.

Surely there must be a way to edit services in a non running instance of TKL (/Debian/Ubuntu)?

 

DAB

Do you know our Appliance Builder - DAB?

http://pve.proxmox.com/wiki/Debian_Appliance_Builder

working great to automate building of Debian and Ubuntu OpenVZ appliances.

br, martin

error ssh generate

hello,

i use your template but i have small bug i creat vm but no start...

have you solution ?

(proxmox 1.6 )

tks ;)

read log

* Starting system log daemon... [ OK ]
* Starting Initialization hooks Generating a 1024 bit RSA private key
..........++++++
......++++++
writing new private key to '.tmpkey.pem'
-----
writing RSA key
[ OK ]
Generating public/private rsa key pair.
/etc/ssh/ssh_host_rsa_key already exists.
Overwrite (y/n)?

Jeremy's picture

Have you tried overwriting the key?

I would imagine that should work. Or perhaps you need to remove the exisiting key from the image? Give it a try.

I've just encountered the

I've just encountered the same problem, and it lies in the openvz template archive file.

Delete ./etc/rc2.d/S15ssh_gen_host_keys from the template as it conflicts with ./etc/rc2.d/S15inithooks

Seems to have got things working for me.

TKL v11 Appliances

Sorry for being lazy.  I didn't read the whole thread in detail.  Just skimmed thru it.

Have any of the v11 TKL appliance been successfully converted to OVZ?

Jeremy's picture

Not yet.

But there is a thread here that discusses converting ISOs to OVZ templates. Its a fair bit of a read and a little mucking around but worth the effort IMO. If that all seems like too much, then let me know which one(s) you are after and I can convert them for you and upload them somewhere.

I'm really just in need of

I'm really just in need of the Revision Control and MediaWiki at the moment.  If you don't mind trying to convert those 2 that would be awesome.  I have had a go at applying patches.  I was able to create a Gitorious ISO, but could not get it working.  I don't know if I messed up the patch process or if I just don't know what needs to be configured with Gitorious...  I'm still learning all this.

Where did you have in mind to upload?  I have an FTP server, I can give ya access if ya send me a private msg, I'll respond with credentials for you.

-Todd

Jeremy's picture

Sorry for delay posting

I've uploaded the templates to SourceForge. Have a look here. Let me know how you go with them.

Also you'll need to rename them to get PVE to recognise them.

Thanks

I dloaded both Revision Control and MediaWiki and installed to Proxmox.  Both boot up fine, but RevControl does not seem to be running some services.

When trying to clone the sample repo "helloworld" I get this error:

 fatal: '/srv/repos/git/helloworld.git' does not appear to be a git repository
 fatal: The remote end hung up unexpectedly
Jeremy's picture

There seem to be some issues with running services in OVZ

I think this may be associated with Ubuntu's move towards upstart in Lucid. I don't personally use Revision Control but I will try to have a look when I get a chance. Adrian is probably the man for the job though. His Linux know-how is miles ahead of mine and AFAIK he is currently using OVZ containers. I'm not sure if he actually uses the Revision Control appliance but knowing him I'm sure he has some git stuff running under OVZ so he may be able to shed some light on the subject.

I have posted over on the other thread (here) which will hopefuly get his attention. I guess really though I should probably start a whole new thread announcing the 'community' uploads to keep it all together. I'll try to do that later if I have time.

I noticed also that on the

I noticed also that on the Revision Control appliance even the WebMin is not working.  Looks like the apache server may not be running.  I'm not sure how to check that or how to try to start it manually.

Jeremy's picture

I'm just downloading it now and will have a quick look

Also noticed that SourceForge is saying that it has been downloaded 0 times which seems strange. Perhaps the download was corrupted?

[update] I can confirm that this appliance is definately not working as it should. Initially I could get no http or https connection to it at all. I restarted Apache (service apache2 restart) and it seems ok, although it is again broken after a reboot. Coeection it is even more broken after a reboot. Apache is now refusing to start at all. Also Webmin doesn't want to run at all - although it says it starts (/etc/webmin/start). I will try a fresh conversion and let you know how I go.

As an aside, SF still says 0 downloads...

Jeremy's picture

Fixed! - uploading new template to SF now

Not sure what went wrong with the other one but I've just recreated the template and this one seems to work ok (although I haven't tested extensively). The default website works as does Webmin, so that's a damn sight better than before :)

Download the fresh one and see if it works ok for you.

It works...

Ya, this one works fine.  

After initial testing, I restored a TKLBAM backup to this vm.  Then all hell broke loose.  No services again.

So for now, I will migrate my stuff manually.  The OVZ template does work.  Maybe there is some compatibility issue with restoring from an official TKL appliaance.  Not sure yet, so lets not draw conclusions.

Anyways, next thing I did was to try the TorrentServer.  Same issue as the first (v0) release of the Revsion Control (for OVZ).

My guess is that the whole first batch of appliances may have the same problem.  Not sure just guessing, since I only tried two.

Jeremy's picture

That sucks :(

Ok, not sure what's going on there. I'm pretty sure I'm using the torrent server at home ok - that's why I created the template in the first place (but perhaps it's just installed from ISO under KVM - can't be 100% sure until I check it out?)

There is obviously something really wrong going on. It deserves further investigation but I won't have a chance for at least a week.

Sorry about all this pain, but thanks for testing.

My Apologies...

The TKLBAM restore worked fine (from an official TKL appliance to an OVZ vm).  I forgot to reboot the vm after.

SUGGESTION:  Maybe TKLBAM should prompt the user to reboot after a restore operation.

Liraz Siri's picture

tklbam-restore prints a friendly warning about this

I'm not sure about prompting for a reboot. I mean, if you're already on the command line and you want to reboot you just execute "reboot". The tricky part is remembering that you might need to do that which is why at the end of the restore TKLBAM prints this message:
We're done. You may want to reboot now to restart all services.

I guess I did not read the

I guess I did not read the screen carefully (as I have found many user do not).  So, even developers are subject to the same issues.

Templates I'm running on proxmox

Hi guys, I'm sucessfully running a lot of TKLv11 images on proxmox, including Source Control & Torrent Server, and some custom appliances. I'll upload my templates to the sourceforge project, Jed already gave me access, and check while I'm home if there's any extra tweak I did to make them happen. We can collect the tips on a wiki page.

I think we can open a new thread with specific problems of each appliance, as this thread is not only old but it's not very easy for others looking info about the topic.

@Todd: I had those issues with services with my first templates, but latelly I manage to fix them all, which other templates are you planning to use?

Gitorious???

Do ya have Gitorious as an OVZ template?  If not, can ya post your ISO for Gitorious on the SF TKL Community site.

I would also like to use Torrent Server in a OVZ format.

Many more too, but those can wait a bit.

Thanks.

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)