Announcing TurnKey OpenVZ optimized builds (+ Proxmox VE channel)

OpenVZ and Proxmox VE has been a recurring topic of discussion on the forums, for which we have Jeremy to blame thank. He's done tons of research, testing, preaching, and then some.

What I love about Open Source is that if you have an itch, and the drive to scratch it yourself, you can.

That's exactly what Jeremy and Adrian did. They wanted OpenVZ optimized builds for their Proxmox VE deployments, so they developed a TKLPatch that would convert an ISO into an OpenVZ container. And if that wasn't enough, took the time to upload some of the builds to sourceforge so it would be easier for others to leverage their work.

Hats off to you guys, you rock!

TurnKey OpenVZ optimized builds

Based on Adrian's and Jeremy's work, we were able to add OpenVZ support to our build infrastructure in no time, and after some initial testing, triggered the whole appliance library to be built as optimized OpenVZ containers.

You can get them from the "Download -> More Builds" link on the appliance pages.

Pre-seeding / default passwords

Because OpenVZ builds are used in headless deployments (without a console), they include an inithook which preseeds default values and passwords (excluding the root password which is handled by the VZ CLI tools).

/usr/lib/inithooks/firstboot.d/29preseed

DB_PASS=turnkey
APP_PASS=turnkey
APP_EMAIL=admin@example.com
APP_DOMAIN=DEFAULT
HUB_APIKEY=SKIP
SEC_UPDATES=FORCE

Depending on your use case, you can preseed the values before the system is booted for the first time, or once the system has booted by executing turnkey-init.

It would be great if someone would add preseeding support to PVE... 

TurnKey Proxmox VE channel

A while back the Proxmox folks came up with the idea of adding a TurnKey channel to PVE, to allow users to download TKL appliances in the same way their custom built appliances are downloaded.

It was a great idea, but unfortunately it never got off the ground.

As I mentioned above, the great thing about Open Source is that you can scratch your own itch, and I was curious how the channel mechanism worked - so I dived in. When I came up for air I had added minimal third party channel support and a TurnKey Linux channel (github).

What this basically means is you can now download and deploy any TurnKey appliance on your PVE server in a couple of clicks without leaving your browser.

proxmox turnkey channel

I hope to see this integrated in the upcoming PVE 2.0 release [update: it's coming...]. If you're running PVE 1.9 then you can add the TurnKey channel as follows:

cd /usr/share/perl5/PVE
mv APLInfo.pm APLInfo.pm.bak
wget https://raw.github.com/turnkeylinux/pve-patches/master/PVE/APLInfo.pm

# update appliance list
pveam update

Comments

Jeremy Davis's picture

Because it works fine for me on my v1.9 host:

proxmox:~# cd /usr/share/perl5/PVE
proxmox:/usr/share/perl5/PVE# mv APLInfo.pm APLInfo.pm.bak
proxmox:/usr/share/perl5/PVE# wget https://raw.github.com/turnkeylinux/pve-patches/master/PVE/APLInfo.pm
--2012-01-17 10:24:14--  https://raw.github.com/turnkeylinux/pve-patches/master/PVE/APLInfo.pm
Resolving raw.github.com... 207.97.227.243
Connecting to raw.github.com|207.97.227.243|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6025 (5.9K) [text/plain]
Saving to: `APLInfo.pm'

100%[=====================================>] 6,025       --.-K/s   in 0s      

2012-01-17 10:24:18 (426 MB/s) - `APLInfo.pm' saved [6025/6025]

proxmox:/usr/share/perl5/PVE# pveam update
debug: channel updates: initiated
debug: proxmox: starting...
debug: proxmox: getting index signature
debug: url_get: http://download.proxmox.com/appliances/aplinfo.dat.asc
debug: url_get: 200 OK
debug: proxmox: getting index
debug: url_get: http://download.proxmox.com/appliances/aplinfo.dat.gz
debug: url_get: 200 OK
debug: proxmox: verifying index integrity
debug: proxmox: validating index syntax
debug: proxmox: update complete
debug: turnkeylinux: starting...
debug: turnkeylinux: getting index signature
debug: url_get: http://releases.turnkeylinux.org/pve/aplinfo.dat.asc
debug: url_get: 200 OK
debug: turnkeylinux: getting index
debug: url_get: http://releases.turnkeylinux.org/pve/aplinfo.dat.gz
debug: url_get: 200 OK
debug: turnkeylinux: importing A16EB94D from hkp://keyserver.ubuntu.com
  (RSA: 1)
debug: turnkeylinux: verifying index integrity
debug: turnkeylinux: validating index syntax
debug: turnkeylinux: update complete
debug: channel updates: finalizing
debug: channel updates: complete

And the TKL entries are now there under 'Appliance Templates' >> Download.

Jeremy Davis's picture

It's pretty much a case of copy/paste the commands (the beauty of CLI). So I would suspect that something is not quite right in your PVE install. Did you install from a 1.9 ISO, or did you update from an earlier ISO install, or did you install on top of Debian? Probably the first thing I'd try is to make sure that you have everything up to date. See here (it details upgrading to 1.9 from an earlier release, but should still apply in your instance).

Jeremy Davis's picture

I'm loving it Alon! :)

And I think your solution to the firstboot passwords is the best (and quite obvious in retrospect).

Nice work all round. Thanks heaps.

Chris Musty's picture

Absolutely love it! No more downloading the ISO, uploading it to PVE and then getting it going!

It all makes sence looking back on how PVE works but I never connected the dots.

Have you guys seen the beta for 2.0?

I barely got it running and started to play with it when I got called away to something else but I could not see immediatly any similarity.

Keep it going guys! Awesome work!

Chris Musty

Director

Specialised Technologies

Eric (tssgery)'s picture

Thanks for the cool work.

Maybe this is the kick in the pants I need to try out ProxMox VE. I have a server all set to install but haven't given it a whirl yet because I am so familiar with ESXi. Maybe today.

 

EDIT: Just a heads up for anyone else wanting to trying this and planning on using the ProxMox VE 2.0 beta. As of beta3... appliance templates are not visible in the UI yet.

Chris Musty's picture

Had a go at it and am a happy camper.

Now I can download ISO's directly to PVE!!!

w00t!!!

Chris Musty

Director

Specialised Technologies

Jeremy Davis's picture

And if he doesn't then he knows something I don't!

Chris Musty's picture

ok when i typed that i was creating an iso for a client. 

Chris Musty

Director

Specialised Technologies

Jason Adams's picture

I also had to ignore certification, but no big deal. I'm downloading a VZ now, and excited to give it a go.

This is so handy it makes me feel like I got away with something. Turnkey, you make me look like I know what I'm doing. :)

Ben Alexander's picture

Great added resource to another great resource,  Thanks for working this out.

Alon Swartz's picture

Martin just announced that the TurnKey channel will be integrated by default in PVE 2.0, and is already available in PVE 2.0rc1.

Jeremy Davis's picture

Yay! I'm excited!

Personally I think this is a huge win for both projects and their communities. TKL users wanting a solid base for deploying multiple TKL appliances to hardware have a fantastic OOTB option in PVE. And PVE users have an almost too easy way to access a huge range of ready-to-run TKL OVZ templates at their fingertips, without even leaving the PVE WebUI!

IMO this is the power of open source in action and great to see this sort of cross polination between projects. I think this will have flow on benefits for both PVE and TKL, and I would imagine increased exposure to both.

Thanks again for your efforts on this front Alon. And what great timing too. This will mean that from the first stable release, PVE will include access to TKL templates OOTB. Yay! Can you tell I'm excited? :D

I guess I'll have to have a proper test of PVE v2.0 now then! :)

Jeremy Davis's picture

But only one way to find out! I'm inclined to ask why you are still running v1.6...

Abkrim Mateos's picture

debug: turnkeylinux: getting index
debug: url_get: 200 OK
debug: turnkeylinux: importing A16EB94D from hkp://keyserver.ubuntu.com
debug: turnkeylinux: verifying index integrity

debug: turnkeylinux: unable to verify signature

tail -f /var/log/pveam.log
mar 22 12:12:18 turnkeylinux: importing A16EB94D from hkp://keyserver.ubuntu.com
?: keyserver.ubuntu.com: Connection timed out
gpgkeys: HTTP fetch error 7: couldn't connect: Connection timed out
gpg: solicitando clave A16EB94D de hkp servidor keyserver.ubuntu.com
gpg: no se han encontrados datos OpenPGP válidos
gpg: Cantidad total procesada: 0
mar 22 12:12:39 turnkeylinux: verifying index integrity
gpg: Firmado el vie 13 ene 2012 09:05:19 CET usando clave RSA ID A16EB94D
gpg: Imposible comprobar la firma: Clave pública no encontrada
mar 22 12:12:39 turnkeylinux: unable to verify signature
 
 
I'm lost.

Apreciate help



Yo solo se que no se nada...

Castris Hosting

Jeremy Davis's picture

Mine is working ok this morning:

/etc/cron.daily/pve:
debug: channel updates: initiated
debug: proxmox: starting...
debug: proxmox: getting index signature
debug: url_get: http://download.proxmox.com/appliances/aplinfo.dat.asc
debug: url_get: 200 OK
debug: proxmox: getting index
debug: url_get: http://download.proxmox.com/appliances/aplinfo.dat.gz
debug: url_get: 200 OK
debug: proxmox: verifying index integrity
debug: proxmox: validating index syntax
debug: proxmox: update complete
debug: turnkeylinux: starting...
debug: turnkeylinux: getting index signature
debug: url_get: http://releases.turnkeylinux.org/pve/aplinfo.dat.asc
debug: url_get: 200 OK
debug: turnkeylinux: getting index
debug: url_get: http://releases.turnkeylinux.org/pve/aplinfo.dat.gz
debug: url_get: 200 OK
debug: turnkeylinux: verifying index integrity
debug: turnkeylinux: validating index syntax
debug: turnkeylinux: update complete
debug: channel updates: finalizing
debug: channel updates: complete

Have you retried it?

Jeremy Davis's picture

Because it is readily available info (although in fairness not quite so obvious from OVZ builds because no confconsole auto running). Anyway it is port 12321 (https only).

Jeremy Davis's picture

Do you mean that you can't login as the Admin user?

I just downloaded the TKL Magento OVZ template from the PVE2 WebUI and it all seemed to be going ok, until I tried to log in. It wouldn't let me log in. I tried resetting the login info a few times (turnkey-init) and no dice. I even tried the 'forgot password' link and the email said it sent but I never got it.

Out of interest I launched a Small TKL Magento instance on AWS and same deal!? So I'm not sure what's going on...?!

I've had a bit of a dig through the firstboot scripts and I think this may be a bug. But I'm not sure. If what I think is going on is right, then others should have complained about this previously and it's not just limited to OVZ installs, all Magento appliances should be affected.

I'm tired and going to bed now so not going to do anymore hunting. Some links that may (or may not) be of use are here:
http://www.magentocommerce.com/boards/viewthread/272236/
http://www.magentocommerce.com/boards/viewthread/4957/
http://www.magentocommerce.com/wiki/recover/resetting-admin-password

Jeremy Davis's picture

Because OVZ appliances don't have a true console, firstboot scripts don't run properly (they cause Webmin and other services to hang because they run and wait for input but there is no way to access the dialog to provide the input).

As a workaround, firstboot scripts don't auto run in OVZ templates, they need to be run manually. Hence the 'turnkey-init' script. By running turnkey-init all the firstboot scripts that run automatically when appliances are installed in other mediums (such as ISO install, VM image, etc) are triggered (with the exception of setting the root account password as that is done prior to boot on the host OS).

But as I said, even after running the firstboot scripts I still couldn't login (to the Magento appliance). Not sure why? I'd be interested to hear your experience.

Alon Swartz's picture

Following the announcement of TurnKey Core 12.0rc (ISO , Amazon EC2), we've released an OpenVZ optimized build which is available in the TurnKey PVE channel. You can force the channels update as follows (or wait for the daily refresh):

/usr/bin/pveam update

If you come across any issues, or have ideas on improving the optimized build, please post a comment.

Changes from TurnKey 11.3 OpenVZ optimized builds:

  • Removed Ubuntu related hacks (not needed in Debian)
  • Removed NTP daemon (recommended by Martin)
  • Misc tweaks for Debian Squeeze.
Ric Moore's picture

Jeremy, if you'd do a video tutorial I'll nominate you to the Academy Award! A step by stepper would be perfect!! Let us know when you create it! I've installed ProxMox 2.1 and am having some basic problems wrapping my head around setting things up. I've been around for what seems like ages, but virtualization is a completely new concept for me. Thanks for your service!! Let me know where to send the pizza! Ric


Chris Musty's picture

I have toyed with the idea of doing a video for a while so I can get others to set up PVE without me being there.

I use mine predominately in my office for speed and cost then deploy to the cloud so the setup I do will probably not suit everyone.

If I can get a list of what should be included I will have a crack at a video!

  1. Requirements
  2. Setting up media
  3. Hardware
  4. Installing PVE
  5. PVE LAN settings
  6. Downloading templates
  7. Setting up containers
  8. Settings up ISO's
  9. turnkey-init
  10. confconsole (for static IP can also do manual)
  11. Hostname, hosts file etc

Anything else (this list is just top of my head)?

PS if I setup one in a data center it will take me some time to organise the server (and cost money) so may take alot longer!

Chris Musty

Director

Specialised Technologies

Jeremy Davis's picture

I think that'd be awesome. My personal opinion is that an officebox setup would be more generally useful, especially for newbs that just want to have a play. Also with TKLBAM transfering to the cloud (eg AWS) is pretty straightforward.

Chris Musty's picture

During geek hour tonight 9PM+ 12/7/12 AEST when the kids are in bed I will setup an old laptop.

Thinking more about the datacenter setup, its probably not going to be what everyone needs so I will just do a quick and dirty office (home) setup. This will exclude all the DNS, hostname and more advanced settings and concentrate on just provisioning a service on the home/office network.

I will also just concentrate on containers and skip ISO's for another day, purely for brevity.

So I will cover

 

  1. Requirements (brief)
  2. Installing PVE
  3. PVE LAN settings
  4. Downloading templates
  5. Setting up containers
  6. turnkey-init
  7. confconsole (for static IP)
  8. TKLBAM

Chris Musty

Director

Specialised Technologies

Jeremy Davis's picture

That sounds ideal I think and is probably what most TKL users would be looking for. I look forward to seeing how you go with it. 

Chris Musty's picture

Just did it with cam studio and have a 1Gb file!!!

I am converting it right now but I was surprised.

Does anyone use anything better for screen capture?

Will upload the tutorial as soon as its shrunk to a respectable net freindly size!

Chris Musty

Director

Specialised Technologies

Chris Musty's picture

I am sorry for the quality but after I shrunk the size things got fuzzy, audio is fine (especially considering I am using a cheap logitec webcam!).

There is a little rambling and I get a little toungue tied but the process is there.

Hope someone benefits from it.

http://dl.dropbox.com/u/20051174/ProxmoxContainers.zip

PS After converting it to mwv and zipping it is 77Mb - hope noone has dialup!

Chris Musty

Director

Specialised Technologies

Chris Musty's picture

so when I do a more professional one and edit out the boring bits makiing it less than 10 mins I will youtube it. (currently it is 1 second shy of 24 mins.)

This video was done completely unplanned and is full of babble but hopefully it will get someone going!

Chris Musty

Director

Specialised Technologies

Ric Moore's picture

I can't help with the hardware budget, but I can feed you! I think you have all the elements that needs to be covered there. This is NEW to me and I'm getting confused with the new jargon that comes with new territory. Thanks! I hope you do this soon. I know you all will do a great job. Ric


Chris Musty's picture

When you hear my accent you will know immediatly where I am from.

Virginia to New South Wales (Australia) would be a heck of a trip for a pizza.

I wonder if it will take longer than 30 mins (and hence be free?)...

Chris Musty

Director

Specialised Technologies

Ric Moore's picture

No worries Mate! I have friends in low places all over. Some of my rowdiest online friends are from AussieLand and New Zealand. We'll get a pizza to you somehow. If a kangaroo knocks at the door, let him in. Ric

:) Ric


Chris Musty's picture

Not critical but while you have putty open (or the console) you can initialise with you API key so you can do TKLBAM backups. Just type tklbam init XXXXXXXXXXX and you will be linked with your account.

I may even redo it later because the video quality is crap.

Chris Musty

Director

Specialised Technologies

Ric Moore's picture

Thanks! Yes, the video quality could have been better, but it would have been a huger dnload. I did get what I needed to know. I was letting DHCP assign the IP addresses which kept moving around. Thanks! Ric


Ric Moore's picture

I hope you keep on producing videos. That really helped me iron out several issues. Thanks again. Ric


Chris Musty's picture

Made a new topic for some ideas for tutorials - http://www.turnkeylinux.org/forum/general/20120719/video-tutorials

Chris Musty

Director

Specialised Technologies

Alon Swartz's picture

Following the announcement of 12.1, we've updated the TurnKey PVE channel with the new appliances, both 32-bit and 64-bit. You can force the channels update as follows (or wait for the daily refresh):

/usr/bin/pveam update

As noted in the announcement, 12.1 comes with the 'headless initialization fence' which leverages iptables to create a sort of virtual fence around an appliance

In PVE 1.9 (please leave a comment if this is also true in later versions), the default configuration doesn't support VM iptables, so the fence won't be created. The fix is simple though (source):

Edit /etc/vz/vz.conf, comment out the default IPTABLES and add the following:

IPTABLES="ipt_REJECT ipt_recent ipt_owner ipt_REDIRECT ipt_tos ipt_TOS ipt_LOG ip_conntrack ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length ipt_state iptable_nat ip_nat_ftp"

Then restart the service:

/etc/init.d/vz restart

Jeremy Davis's picture

Glad you got it sorted!
Jeremy Davis's picture

Glad to hear that you like our work!

If you have any questions/advice/etc please feel free to post in our forums. Bugs and feature requests go on our GitHub based Issue tracker. All of the appliance specific code can also be found on GitHub.

Thanks for using TurnKey!

BTW your English is fine, if you hadn't have added the footnote I never would have guessed that English wasn't your first language! :)

Jeremy Davis's picture

My guess is that you've long since moved on, but for the sake of completeness, yes, the LXC containers should work ok on OpenVZ. I didn't do extensive testing, but I did do some testing and they seemed ok...
maec's picture

From Proxmox, downloaded Turnkey - Core 14.2-1 LXC and get this at the end. Any ideas?


2018-01-05 09:35:23 (1.44 MB/s) - '/var/lib/vz/template/cache/debian-8-turnkey-core_14.2-1_amd64.tar.gz.tmp.31879' saved [147531649/147531649]

Use of uninitialized value in lc at /usr/share/perl5/PVE/API2/Nodes.pm line 1132.
Use of uninitialized value in concatenation (.) or string at /usr/share/perl5/PVE/API2/Nodes.pm line 1133.

TASK ERROR: wrong checksum: c92598c8f4d139a3a65b1e4a0c00953a !=
Jeremy Davis's picture

FWIW the additional page links on blog posts is a bug that started after our most recent website upgrade (wasn't initially noticed and we haven't been able to fix it yet).

Also sorry that I missed your post. FWIW new posts on old blog posts often slip through the cracks. Generally a new post in the forums is much better at getting attention.

Regarding your issue, TBH I'm unclear, and it's not one that I've hit. Although it is worth noting, that we only actively support the latest version of Proxmox (and the latest v15.x appliances). So I'm guessing that you must be running PVE v4.x (rather than the current v5.x). I'm almost certain that if you update to v5.x, then it should all "just work" (although you won't be able to get v14.x appliances, only v15.x).

Having said that, you can manually download our appliances for Proxmox, like this:

# Assuming you want to download LAMP v15.1
# from PVE commandline, running as root:
cd /var/lib/vz/template/cache
wget http://mirror.turnkeylinux.org/turnkeylinux/images/proxmox/debian-9-turnkey-lamp_15.1-1_amd64.tar.gz
wget http://mirror.turnkeylinux.org/turnkeylinux/images/proxmox/debian-9-turnkey-lamp_15.1-1_amd64.tar.gz.hash

Then read the instruction in the hash file to (optionally) check that you have the legitimate file (the hash file should be signed by us) and that the image is not corrupt (via the hashes). Although, it's also worth noting, that whilst I did do early testing of v15.0 on PVE v4.x, I updated to v5.x prior to the official v15.0 release (to ensure that they worked flawlessly with the lastest Proxmox).

I hope that helps.

Pages

Add new comment