TurnKey Meta Core aka TKL Master Server aka Virtual Environment

Sorry for the late reply I somehow missed this post!

First off as stated before I think this is a terrific idea and I'd love to learn more from those in the community who have experience with the various virtualization alternatives (e.g., KVM vs OpenVZ).

A few clarifications regarding our development plan for TurnKey Linux:

• Yes, future releases of TurnKey Linux will be based on Ubuntu 10.04 LTS
• Debian based builds of appliances will also be supported in the future.
• 64-bit support is planned.

In a nutshell our philosophy is to let users choose whenever possible so it's not a question of Ubuntu vs Debian, or 32-bit vs 64-bit.

We plan on supporting them all. Or rather, supporting the best options. Because really in many cases you can't say one is better than the other but rather that there are different pros and cons for each alternative and which is best depends on your circumstances.

On OpenVZ vs KVM

This is especially true with regards to virtualization technologies such as OpenVZ vs KVM. They're different approaches to the same problem. If I'm not mistaken, OpenVZ is a thin container-level virtualization solution, so you have one kernel and many userlands. OTOH, KVM is a hardware virtual machine based solution in which each virtual machine is running it's own emulated hardware stack and it's own kernel, etc. The level of isolation KVM provides you is much stronger, but you pay for it with greater overhead (e.g., RAM, CPU).

Because each is better suited for different usage scenarios if possible we want to support both.

Target the ISO build

Many virtualization solutions are designed to run on bare metal. You can theoretically nest some virtualization solutions but I don't think that's a popular usage scenario. I recommend you stick with hacking the ISO build.

Development process

What I suggest is trying to create a patch for TurnKey Core which implements a basic virtualization solution from a component you think would make a neat basis for this. Since it's a TurnKey appliance for running other appliances let's call it TurnKey Meta Core. Only once the basic, most important stuff is working well do you start to think about adding more bells and whistles, and always remembers that every additional component and feature has a cost (e.g., increased footprint, usage complexity, security risks, etc.) and that cost has to be weighed against the benefits. That's how we try and figure out what belongs in a TurnKey appliance BTW.

Starting goal: do the simplest thing that could possibly work

Simplest way to turn a bare metal machine into a platform for running other TurnKey appliances (though it doesn't have to be limited to TurnKey!).

Our target audience is technically savvy but not experts with this virtualization stuff. They like the concept and they want to run a whole collection of TurnKey appliances on that server in the closet and they want TurnKey to make it super easy for them to do that.

At the moment you shouldn't worry about the sort of extra things high-end users will want (e.g., support for hardware clusters). That's further down the road.

Experiment experiment experiment

Unless you already have extensive experience and know exactly what needs to be done there is really no point in planning too far ahead. The key here is to adopt an playful attitude and find out what works best via experimentation. That seems to work pretty well for us anyhow. Just immerse yourself in the technology and once you are really familiar with them then the right thing to do will be much easier to decide. Don't plan for the future. Let the future take care of itself.

Sure its a shame that OpenVZ requires a special kernel, but as you correctly point out, LXC should provide equivalent functionality within the mainline kernel and allow us to provide this sort of solution everywhere without asking hosting providers for special favors. That's an exciting prospect indeed!

Because it must be a very relevant consideration for you guys but its such an appealing technology for this proposed appliance!

Oh well, that may not be a problem from April next year. Whilst 10.04 won't support OpenVZ, it will support LXC which is a very similar container type virtualisation. So standard 10.04 kernel will support both KVM and LXC. As you would hope, they will both be in the repos too.

I think that is very exciting! 2010 may well be the year of the TKL Meta Core - with both LXC and KVM support!?! I can't wait!!

As a hosting customer I personally prefer hard virtualization technologies such as Xen because I know they are harder to oversell and I'm less likely to get a raw deal. But with a meta-TurnKey appliance which would typically be self-hosted or running inside a "hard" VPS overselling isn't really an issue because if you oversell you are overselling to yourself. So that's a nice way to squeeze more performance out of your given memory/CPU resources while still getting the benefits of security and functional isolation between appliances.

OpenVZ is also interesting because providing you can run your own kernel, it would be easier to deploy it nested inside a commodity VPS based on Xen / KVM. I know OpenVZ works under Xen at least.

Unfortunately the leading cloud provider Amazon EC2 doesn't let you run arbitrary kernels. I'm not sure they have allowed any kernels that support OpenVZ.

I just wrote a big long reply but I've lost it I think I accidentally browsed away from the page - Doh! I'll try again...

Anyway the point I was making is that I agree with pretty much everything you said Liraz! Whilst I still haven't done anything concrete I have done a little research.

I'm guessing you can see where this is headed! Proxmox have a Debian repo so you can install from there on Debian (and probably Ubuntu too?).

Whilst I like Xen (at least from what I've read) KVM is probably the easier technology to implement on Linux (as its already included in Debian & Ubuntu repos). It seems that whilst KVM will theoretically run on 32 bits, its not really supported (makes sense to some degree because of the RAM limitations - apparently guests on 32 bit KVM can only have max RAM of 2GB). Packages are only in 64 bit repo from what I can gather. KVM does not require a modded kernel (support is included in standard kernel).

OpenVZ is included in Debian repos but not Ubuntu. It uses on an older kernel (currently 2.26) so unless you wish to complie your own custom kernel, many of the new kernel features are unavailable (not such an issue with current TKL on 8.04 but not ideal for 10.04).

As Liraz also mentions, there is a trade off performance vs isolation. For smaller workplaces (esp non-profits like mine) its possibly a no brainer - less resource use = better VM performance on older/cheaper hardware, limited need for isolation. For larger corporate type locations or hosting services then it may be a no brainer in the other direction.

When I first started looking for a Virtualisation platform/Hypervisor I was initially leaning toward Xen, but then I found Proxmox and haven't looked back. The beauty of Proxmox is that 2 alternate (KVM & OpenVZ) technologies are available simultaneously. Even though it requires a custom kernel, Proxmox provide that. One downside of this is that they have to compromise and some of the newer KVM features are unavailable in Proxmox because of that. Hopefully this will change as it seems OpenVZ developement has picked up again (at least a little - it was stalled for some time).

Next on my agenda is to try to install the modded 32 bit deb onto TKL Core (Proxmox only provide a 64 bit deb - but someone has kindly modded it to work on 32 bit). Unfortunatley this only supports OpenVZ (no KVM) but may provide proof of concept.

In my opinion the only requirements for TKL purposes would be to hack the interface a little and add TKL VM images (KVM apparently supports VMware images so should be compatible with current TKL builds - I'll check on this)  and OpenVZ templates in the download section - that way users can directly download the images from within the web interface - pretty trick huh! Also probably want to add TKL to the page footer.

Going back to your comments Liraz - One thing I'd like to say about nested installs - for users on Desktops (esp Windows) who wish to use VirtualBox or VMware - the idea of nested OpenVZ VMs running on Meta Core would provide the funtionality that many are looking for (from reading the forums), the performance they want (using OpenVZ containers) without the resource penalty of running multiple VMs simultaneously.

but unfortunately I'm having some trouble getting it running on my old Ubuntu (9.10) laptop. (That's probably no great surprise as its a 5 yr old Dell with pretty sub-standard specs).

I can confirm that its in the 9.10 Ubuntu repos which could be a real bonus assuming that it will be in 10.04 too. As I think I already mentioned it supports KVM (also in repos) and Xen (requires custom kernel).

In my travels I found a handy site here re Eucalyptus setup/config amoungst the Ubuntu documentation that may be helpful if for anyone else who wants to have a look.

Considering that its not in the Ubuntu repos pre 9.04, if TKL goes the Eucalyptus route then it'll probably be best to either wait until post 10.04 release, or build some beta prototype on 9.10 or 10.04beta. It seems like there's probably too much work to bother building it on current TKL at this point.

I've messed around with Asterisk before by using Trixbox (formerly asterisk@home). They provide a GPL community edition iso installer at www.trixbox.org

I don't know if it would be worth creating a separate Asterisk appliance when Trixbox has been around for many years and is the original Asterisk appliance as far as I can remember.

Of course, if anyone has the expertise and desire to recreate this in TKL form, that would be awesome.

I think a separate Asterisk appliance would be a great idea but neither Alon or I have ever used that ourselves before so for us there would be a learning curve involved and that delays how soon we'll get to it. What would help is if members of the community with more experience come forward and contribute a tklpatch we can build on.

I just started looking at the VoIP solutions this past week.  As I wanted to shift from the win32/Hmail server for our church I thought this would be a great time to integrate the PBX so we could do some fun things like softphone/videophone.  I initially looked at this thread following the idea of multiple individual appliances.  The PBX idea seemed like a natural idea for an appliance... until you reminded me of the hardware issue for other people, although it should still work well for the softphone idea.  

Considering it further, the need for hardware (in my situation) on the voip is not truly necessary, therefore it might be worth looking into this not as stand alone appliance, but integrating it with another appliance such as the Domain server or Zimbra.

I have decided that using Zimbra Turnkey is my most likely candidate for the mail solution, and as I researched further it appears that Zimbra has a couple of zimlets to integrate with Asterix.  With that in mind, would it be too much to ask that it possibly be included in the next release?

If I get time I will try to install it on the existing Zimbra release.  I was hoping to use the PBX for a SIP based "softphone".  I must admit, I cheated and used a win32 version of Asterix just to see how it works.  As I am not too familiar with Linux, I am not sure I can do this, but I will try to install this weekend.

Cheers!

Feel free to move this if it is not suitable in this thread.   

I have no experience with either but I just had a look at both.

eyeOS looks very interesting but unless I'm missing something, this page seems to suggest that its more of a browser based desktop rather than something suitable for the basis of TKL Meta Core. Don't get me wrong though, I really like the look of it and think its a great candidate for a TKL appliance. Its almost along the lines of Google Apps (or ProjectPier, OpenGoo etc) but laid out like a normal PC desktop (even the file manager looks like nautilus!)

I didn't have quite as close a look at abiCloud but from the screenshots on the site it looks like it has a very polished interface, which is always nice. It definately looks like something along the lines of what we are looking for.

The only thing that I'm not 100% sure about is the fact it uses VirtualBox under the hood. I think VirtualBox is great but I tend to think of it as more of a Desktop VM environment than a Server one. I know it has progressed (since I last used it) and maybe I am being unfair as I know there are some advantages eg can be run on lots more hardware than some (such as Xen or KVM).

Also Oracle's recent aquisition of Sun means who knows what direction it's developement it will take. From what I've read they're keen to take on VMware which may be great, but its still unclear how much effort they will put towards supporting OpenSource development.

[edit] I have also edited the original post to include a link to abiCloud

OpenStack is a new project that has some major names behind it including RackSpace, NASA, CITRIX, and RightScale. I spoke about it the other day with Alon and he said he took a look at the architecture and was very impressed. It's implemented mostly in Python. Could be a winner!

I use Convirt on my server.
They also have a web interface. Though I haven't used it yet.

I was almost going to start a new one. But decided to revive this old one as there is still plenty of relevant info here.

I know Adrian has been playing with OpenStack so hopefully we will hear a bit from him when he has time?

I just wanted to post about something I came across in my travels which I really like the look of and is vaguely relevant to this thread. TBH it's probably more relevant to a TKL Core appliance (desktop/GUI). It's called Virtual Machine Manager (or virt-manager for short). It's a desktop GUI written in Python for controlling local or remote VMs and as it is uses libvirt as a backend can talk to a huge range of virtualisation technologies (pretty much all the major ones) including VirtualBox, Xen, KVM/QEMU, OpenVZ, LXC, VMware and even MS Hyper-V!

Besides libvirt another cool virtualsation tool I've come across is libguestfs It looks very cool set of tools for working with all sorts of virtual-HDDs. It's not in any current Ubuntu repos, but looks like it'll be in Precise/12.04 - see here.