TurnKey 12.1 64-bit maintenance release built with new tkldev build appliance

TurnKey 12.1 is out and it's the first 64-bit maintenance release to be built with tkldev - TurnKey's shiny new open appliance build system in a box.

With 64-bit support out the door, we've also pushed out a round of updates to the Hub so that users can finally deploy TurnKey on all instance sizes.

Full details on the changes to the Hub below, but first I'd like to talk a little bit about tkldev, TurnKey's new open build system. tkldev will soon be released as a standalone appliance along with the full source code to all appliances in the TurnKey Linux roster, which we will be maintaining on TurnKey's GitHub page.

tkldev: why a new build system?

As many of you know, in a bid to ease up TurnKey's dependency on the core development team, one our of strategic development goals has been to re-engineer the messy patchwork of scripts and build systems that used to be our build infrastructure with the goal of creating a self-contained "fabrication" appliance that could be used by appliance hackers to build any TurnKey appliance from source code.

We've actually had this in our sights for quite a while now. We made some bad technical decisions early on with regards to how we setup our "legacy" build infrastructure and realized a bit too late that we would have to redo everything if we wanted to get the open source community truly on board with TurnKey's development.

With everything on our plate (e.g., developing over a hundred appliances and the TurnKey Hub) it sometimes feel like we're running in place. So it's taken longer than we would have liked to make this happen. But... with a great sigh, no - heave - of relief I can proudly announce that work is finally done. Well almost. We just need to finish up the packaging and documentation.

We've been battle testing the new build system by using it to develop the upcoming TurnKey 13 release, based on Debian Wheezy, and also the TurnKey 12.1 maintenance release which I'm supposed to be announcing.

As usual, I seem to be getting a little bit ahead of myself, so let me try to get back on track.

What's new in TurnKey 12.1

64-bit (amd64) is the new default image type

Download links on the website have been replaced with 64-bit images. For users that prefer 32-bit images, they are still available for download from Sourceforge and our mirror network.

Core operating system upgrade

  • Upgraded base operating system from Debian Squeeze 6.0.5 to Debian Squeeze 6.0.7.

    Debian Squeeze is scheduled to be maintained with security updates until May 2014.

  • Upgraded all non-Debian Core components (e.g., Webmin 1.620)

  • Bugfixes and tweaks

For full details see the TurnKey Core changelog.

Fresh upstream application versions

This maintenance release includes the latest software versions for all components installed directly from upstream source code rather than the Debian package management system.

The exact details of the package version installed from upstream source code can usually be found in the appliance changelog, except where the version is determined at build time.

This includes the latest upstream versions of the main application in the following 71 appliances (from a to z):

appengine-go, appengine-java, appengine-python, appflower, b2evolution, bambooinvoice, cakephp, canvas, clipbucket, codeigniter, collabtive, concrete5, deki, django, drupal7, e107, elgg, etherpad, ezpublish, gallery, gitlab, icescrum, jenkins, joomla15, joomla25, limesurvey, magento, mambo, mibew, moodle, nodejs, omeka, openphoto, orangehrm, oscommerce, osqa, owncloud, phplist, phpnuke, phreedom, piwik, pligg, plone, prestashop, processmaker, projectpier, punbb, rails, redmine, sahana-eden, sencha, silverstripe, simpleinvoices, simplemachines, sitracker, statusnet, sugarcrm, symfony, tomatocart, tracks, twiki, typo3, ushahidi, vanilla, vtiger, web2py, wordpress, xoops, yiiframework, zencart, and zurmo.

In case you're wondering, the other 30 appliances in this release didn't get the very latest upstream version of their main application because all the major components are installed and maintained through the Debian package management system:

asp-net-apache, bugzilla, core, couchdb, dokuwiki, domain-controller, drupal6, ejabberd, fileserver, lamp, lapp, lighttpd-php-fastcgi, mahara, mantis, mediawiki, moinmoin, mongodb, movabletype, mysql, nginx-php-fastcgi, openldap, otrs, phpbb, postgresql, revision-control, roundup, tomcat, tomcat-apache, torrentserver, and trac.

The trade-off is that while the component versions may be less up-to-date, Debian provides back-ported security fixes which TurnKey automatically installs.

TurnKey headless initilization fence

This is a new feature that should make life easier for users of our headless OpenStack, Xen and OpenVZ builds.

A common problem with headless deployments of TurnKey is that turnkey-init, TurnKey's initialization wizard, can't be run on the first boot because we don't have a console that can interact with the user to properly configure the appliance (e.g., setup application and database passwords, domain name, etc.)

This typically results in frustrated users failing to understand why an uninitialized appliance doesn't seem to be working at all or complaining that they can't figure out how to log in.

The usual solution is to set a bunch of default passwords and hope users, who rarely bother (or want to bother) reading the documentation, will manage to change all of them before they get exploited.

But even when this approach sorta kinda works, having to figure out all the default passwords and rush to change them is inconvenient.

Worse, default passwords are dangerous, especially for anything connected to the Internet. They open up a window of vulnerability that can allow an attacker to compromise the system by racing to exploit your default passwords before you change them. Throw in botnets automatically scanning the network for low hanging fruit and you have a recipe for catastrophe.

TurnKey's solution to this conundrum is to leverage iptables to create a sort of virtual fence around an appliance. The fence intercepts attempts to access potentially vulnerable uninitialized applications, redirecting users instead to a mini-tutorial explaining how you need to log in as root first. On an uninitialized appliance logging in as root will automatically launch turnkey-init and help you finish setting everything up.

Introducing 64-bit images, phasing out 32-bit appliances

So why waste any time on a TurnKey 12 maintenance release (e.g., based on Debian Squeeze) when Debian Wheezy has already been released?

Basically, TurnKey 12.1 is a stepping stone for the upcoming TurnKey 13 release.

We'll be deprecating 32-bit support in TurnKey 13 and phasing it out completely by TurnKey 14. That means we'll be building all appliances in both 32-bit and 64-bit versions for TurnKey 13 but encouraging users to migrate to 64-bit because TurnKey 13 will most likely be the last major TurnKey release to come in both 32-bit and 64-bit image formats.

To make it easier for users to migrate from 32-bit to 64-bit we decided it would be a good idea to add 64-bit support to a maintenance release of TurnKey 12, the current major version of TurnKey.

That way existing users don't have to switch to a new major version of Debian while switching the operating system architecture at the same time.

Another reason we decided to bother with a maintenance release release now is that it made it easier to focus on testing the new infrastructure. Pinning down one thing that doesn't change (e.g., the base operating system + appliance roster) makes it easier to identify and work out the kinks in the pieces that have changed.

TurnKey Hub updates

  • Support for all instance sizes: with 64-bit support out the door, the Hub now supports all instance sizes, paving the way for using TurnKey for more heavy duty workloads.

  • Significant cost savings with new heavy / light reserved instances

    Up until now the Hub has only supported Medium Utilization type reservations. When we added support for reserved instances this was the only type of reservation available but Amazon have since added Light and Heavy Utilization type reservations.

    To help Hub users save significantly more money, especially when running larger instance sizes, we've added support for these types of reservations as well.

    Light Utilization reservations have a lower up-front purchase price than a Medium Utilization reservation but also a lower discount on usage fees. Heavy Utilization reservations have the highest up-front purchase price but provide greater discounts for continuously running servers.

    For example, you can save up to 76% (over $8000) of total costs when running an X-Large High-RAM instance continuously in the Amazon EC2 cloud if you purchase a Heavy Type reservation for three years.

    Purchasing a reserved instance is the Amazon EC2 equivalent of purchasing your own co-located server hardware, just with a bit more flexibility.

    The Hub provides an interactive cost savings calculator in the reserved instance dialog that shows the up-front fee, new hourly fee, break-even utilization level and expected cost savings for a continuously run server.

  • Deploy TurnKey in a cloud down under

    Some of our most important community members live in Australia so we're especially pleased to announce support for Amazon's new Sydney region.

  • New cloud servers plans & pricing

    • No hassle, free Micro support for everyone.

      You no longer have to invite anyone, sign up for a time-limited free trial, etc.. New Hub users can now immediately launch any TurnKey app as a Micro instance.

      New Amazon accounts automatically qualify for Amazon's free usage tier which provides up to 750 hours of Micro server usage each month for up to a year.

    • Budget => Bronze: The "Budget" plan has been renamed "Bronze".

      • New supported instance size: Medium (64-bit only)

        Previously only Micro, Small and Medium High-CPU instance types were supported on this plan.

      • Defaults to deploying 64-bit appliances on all instance types but allows you to choose 32-bit appliances if you want (e.g., backwards compatibility, increased memory efficiency). The upcoming TurnKey 13 release will also provide support for 32-bit appliances, but the next release after that, TurnKey 14, probably won't.

    • Hobby => Pay-per-use: The Hobby plan has been renamed Pay-per-use.

      • Supports deploying any S3-backed instance size, including all of the new large and x-large 64-bit only instance sizes.

      • Users that signed up to what we used to call the Hobby plan should receive an email from Amazon in the next few days detailing a price change which will come into effect two weeks later: we'll be dropping the markup on usage fees from 15% to 10% and adding a $10 monthly fee.

        We aren't too happy about having to change the pricing structure of what was formerly the Hobby plan. Unfortunately, in the wake of a series of steady price reductions in instance usage fees this plan has gone from bringing in a small amount of revenue, to barely covering costs, to actually costing us money. This happens because Amazon's billing system (AKA DevPay) charges us a small fixed fee per user that signs up. It used to be that the markup on usage fees covered this cost but now users with significant usage do the math and just sign up for a flat-rate plan.

        Instructions for canceling or switching to another plan are available on the Hub. See Account Details in your Hub account's EC2 account page.

    • Added three new flat-rate plans (e.g., Silver, Gold, and Platinum)

      Other than the support for larger instance sizes, the plans are similar to Bronze: a single flat-rate monthly fee allows you to deploy an unlimited number of instances with no markup on the standard Amazon EC2 usage fees.

      Current Bronze users that want to deploy larger instance sizes can upgrade seamlessly to one of the new plans by clicking "Switch Plan" on the new cloud plans and pricing page:

      https://hub.turnkeylinux.org/amazon/enable

Vote for TurnKey on SourceForge's project of the month ballot

TurnKey is a candidate for Sourceforge's June 2013 project of the month. If you like the work we're doing take a few seconds to vote for us:

http://twtpoll.com/xmrzlp

I'm no fan of demagogues and empty campaign promises, so rest assured I mean it when I promise that if you vote for us I vow, on my honor and the honor of my ancestors, to continue slaving away on TurnKey for as long as it takes to win our heroic, epic battle against the evil forces of entropy.

So vote! For TurnKey! For open source changelogs you can believe in! With your vote, together, we will create a party of superb open source software so powerful it will... WE will... repeal the oppressive laws of thermodynamics! Yes! For a better tomorrow!

Comments

Adrian Moya's picture

Liraz! Nice to have news from you! It seems that you guys are in good path of modernizing the project and be more open, as we discussed a lot in the past. 

The upgraded upstream versions are very welcome, and 64bit support is excelent news. I really like the way the project is evolving (deprecating 32 bit appliances).

Of course I'm still waiting for a chance to get my hands on the new tkldev. I'll be watching this space as usual, willing to see the results of your work on this field. 

Cheers! 

Liraz Siri's picture

Mulling over this, I think part of the reason Alon and I have been so quiet lately, besides being ridiculously busy, is that we've grown kind of ashamed at developing TurnKey "in the dark", instead of out of the open, like a proper open source project.

Once that sunk in I think Alon and I both felt we needed to work on fixing that before daring to metaphorically "show our faces".

Happily, tkldev will very soon be out, and not long after so should TurnKey 13. I think after that most of the shame will go away. I might even get back to sharing random thoughts and discoveries on the TurnKey Blog. It's been pitifully inactive of late.

tkldev won't be everything we envision in a community build infrastructure, but it will be a very important step forward. Further down the road, I'd like to create a cloud service that automatically builds and distributes community appliances. OTOH, I also have a few other important projects on my plate so when that will happen is anyone's guess.

If I could just figure out how to get fork() working in real life...

OnePressTech's picture

I'm with Adrian. Looking forward to tkldev.

As always...thanks to you (Liraz) and Alon for your hard work and dedication.

I know that you'll find you have a devout group of followers & supporters aching to become contributors as well. I'm certainly one of them.

Cheers boys.

Cheers,

Tim (Managing Director - OnePressTech)

Adrian Moya's picture

I'll cast my vote for Turnkeylinux on SourceForge right now!

Liraz Siri's picture

If I could, I would kiss your baby, look up and smile while waving at the crowd for a photo op.
Jeremy Davis's picture

Nice work! I know that you guys have been slaving away on this release (and the infrastructure that built it) and I'm so happy to see the fruits of your labour! IMO this is a major milestone release (which I guess is a bit funny in that it's a point release) and obvioulsy there is still plenty of work to be done but it's a huge step forward...

I think releasing the build infrastructure as an appliance is a fantastic idea and also providing the source for each appliance via GitHub is brilliant! I am assumimng that with these in place, individuals could choose to build an 'official' TKL appliance themselves?

And whilst I know it has been a lot of work for you guys, it will change the way and the degree to which the community can engage with the project. Hopefully (eventually) you may even see your workload decrease (and/or give room for new innovations and ideas to be fulfilled). Using GitHub will allow bug fixes and tweaks to be provided by the community and merged as appropriate. Also customised and new appliances will be able to be added to the fold relatively easily (community developers can use GitHub to host there custom appliance code and TKL could fork it to bring it in as a new official appliance).

I have a ton of questions about TKLDev and the implications for TKLPatch (like will it be depreciated or is it part of TKLDev) but I think I should wait for the announcement before I start rattling through them as I'm sure you will answer many before I even ask them!

Liraz Siri's picture

Yes, the new development infrastructure will put everyone on equal footing, at least in terms of development tools and resources. You'll be able to build an official appliance from scratch the same way we do it.

We're really hoping the new development infrastructure will encourage the community to forgive us for past mistakes that closed the project down and get some real collaboration going.

Having to maintain every bit of TurnKey ourselves has evolved into the archetypical Red Queen's race. With more development resources we could crank out releases more quickly, expand the library, and work on all the innovations we've been dreaming up but haven't had time to implement.

Heck, I might even have time to read my email and respond to community feedback within a more reasonable timeframe. Speaking of which, sorry for the late reply!

Alon Swartz's picture

Thanks for the initial feedback guys!

Just a quick note for those using the OpenVZ builds in Proxmox VE, the TurnKey channel has been updated for 12.1, so you'll be able to take advantage of inline downloading and deployment. Regarding the initialization fence and iptables, please see my comment.

Jeremy Davis's picture

And the detailed comment (you linked to on the OVZ announcement).

I'll endevour to have a play over the weekend and give some feedback on how that goes (I run PVE 3.0 at home and as you know use OVZ lots).

Jeremy Davis's picture

IIRC if you use a Linux desktop it's really easy... I'm pretty sure that you can just dd the ISO to the USB (you'll need to ensure that the USB is bootable first...). The instructions for PVE (which I have successfully used to create an install USB of PVE; PVE is also based on Debian) should work: http://pve.proxmox.com/wiki/Install_from_USB_Stick

If you are using Windows then another option that should work is: http://www.pendrivelinux.com/yumi-multiboot-usb-creator/

Seeing as it's a old PC though, are you sure that it will support booting from the USB you're using? Even if it technically does support booting from USB I have found some older motherboards can be a bit finiky on whether a USB will work or not... Some seem to only work with 4GB or smaller USBs and even then I have found that some brand USBs will work while others refuse to...

Although having said that, in my experience if you manage to get as far as the bootloader then that probably isn't the problem...

apn's picture

Hello

That all sounds good... but one query: I use the 'HubDNS' which integrates with Amazon's Route 53 service, which seems to require a 'pay per use' (previously 'hobby') plan.  But I have no need to run EC2 instances as I host a server on my own hardware.  

Does the new 'pay per use' plan pricing, with the new $10/month charge, have to apply in that situation?  I have no need for EC2 instances etc., so it seems a bit much if so, as it would mean you would have to pay $10/month just to access the dynamic DNS / HubDNS service.  If the $10/month plan is the only option for enabling HubDNS, do you have any other recommendations for achieving a dynamic DNS solution?

Many thanks.

Alon Swartz's picture

A plan isn't required for HubDNS, but your Hub account does need an Amazon account with Route53 enabled to manage custom domains (as always).

Jeremy Davis's picture

I was under the impression the only Linux AMIs that are not eligable for the Free Tier are licenced ones (like SUSE Enterprise Linux and Red Hat Enterprise Linux)... But perhaps that has changed. I found it difficult to confirm or deny this on the Amazon site. Some of my reading suggests that I am right, but it doesn't explicitly say and looking at the listing (that has the stars) it seems that there are only a few that are explicitly supported...

Are you using a micro instance? If so then it's probably worth contacting the TKL devs direct - best way IMO is via the Hub 'feedback' (blue button on left hand side whan logged in).

Liraz Siri's picture

If Amazon limit which types of AMIs are eligible for the free tier it's the first time I've heard about it. We certainly haven't done anything to exclude our images from the free tier. Perhaps something has changed? I'll try to ask our contacts at Amazon about that.
Hans Harder's picture

Good Job....

Just like the others, I can't wait to get involved in the new TKLDev environment.

I will start this weekend in taking one of the OpenVZ builds and see how easy it is to use this for LXC or what is needed to convert them.

 Thanks for the hard work.

 

Hans

QUOTE:  ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol

Liraz Siri's picture

Yes of course you can be signed up without a plan. That's exactly what the free evaluation is for. The pay-per-use plan might be more than you need at the moment.
John Carver's picture

environment.  One capability I hope is present is the ablity to create combo appliances.  I have an application that needs both the Drupal7 plus OpenLDAP for a directory application.  Client can't afford multiple hosted servers, so both must be combined into a single appliance.  I hope the new build process will allow us to build these combo appliances.

My next question (probably a dumb one) is, will TKLpatch work with the 12.1 iso's, or do we need to wait for more info on setting up the new development platform?

Information is free, knowledge is acquired, but wisdom is earned.

Liraz Siri's picture

You should be able to use tkldev to create combo appliances or pretty much any other TurnKey-based integration using exactly the same toolchain used to put together the "official" appliances.
Jeremy Davis's picture

And I believe that it is not being depreciated, but will live alongside TKLDev as an alternative pathway to develop appliances...

John Carver's picture

My first test of TKLPatch with a 12.1 iso failed :(

root@tklpatch ~/patches# tklpatch iso/turnkey-core-12.1-squeeze-amd64.iso example
# extracting root filesystem and isolinux from ISO
Parallel unsquashfs: Using 2 processors
24885 inodes (25366 blocks) to write

[===========================================================\] 25366/25366 100%
created 23257 files
created 2794 directories
created 1069 symlinks
created 38 devices
created 1 fifos
TKLPATCH_ISOLABEL: example
# applying patch example
# executing config script example/conf/pre-debs
chroot: failed to run command `dpkg-divert': Exec format error

This is probably because I'm running TKLPatch on a 12.0 core, i.e. 32 bit virtual host.  Guess I'll need to create a 64 bit host for testing.

Information is free, knowledge is acquired, but wisdom is earned.

Jeremy Davis's picture

I would expect that the architechture of your build environment would have to match the ISO.

Although I'm not sure about the upcoming TKLDev...

Alon Swartz's picture

Yep, the architecture of the host needs to match ISO you are patching. This holds true for TKLPatch as well as the soon-to-be-released TKLDev.

It is technically possible to support patching/building 32bit on 64bit with some hacks, but not the other way around IIRC.

As for TKLPatch, it will continue to be maintained as it's useful for patching ISO's. But, for developing and building appliances, TKLDev will be the tool you'll want to use.

John Carver's picture

I've got some perfectly good hosts that I'd like to keep using a few more years, but they're not 64-bit capable.  I'd rather not get locked into either upgrade hardware or live with 12.0.

Information is free, knowledge is acquired, but wisdom is earned.

Jeremy Davis's picture

So you will have OOTB 32 bit support (with security updates) until at least 12 months after Debian 8/Jessie is released (which should be a year or 2 away at least).

As Alon mentioned building 32 bit in 64 bit environment is possible so even beyond that hopefully TKLDev will be able to build 32 bit appliances...

L. Arnold's picture

I have 3 older HP Servers that will do 64 Bit bare metal, but only host 32 Bit Virtual Machines.  They will get upgraded, but only eventually.

It is critical, for me anyway, to have 32 Bit Builds available that can be  upgraded to 64 Bit when the pieces are in place.  Running into this just now with the PostgreSQL server  (at least I can't find the 32 bit images)

John Carver's picture

I'm not sure which type of image you're looking for, but the 32 bit iso is found at http://sourceforge.net/projects/turnkeylinux/files/iso/turnkey-postgresq...

If you can find the link to the 64 bit image, then edit the link and replace 'amd64' with 'i386'.  It works for iso images and I'm guessing it would work for other types as well.

Information is free, knowledge is acquired, but wisdom is earned.

Joann Lowis's picture

Just wanna say that after finaly getting my hands on it as well as "my grips".., 64bit TurnKey is da bomb! TurnKey's new "build system in a box" certainly made my life a lot easier. Many many thanks for the very hard dev-work! I'm pretty surprices as well at the combo appliances creation ability. Will have to spend some more time on it. Thanks again, Joann.

Jeremy Davis's picture

If you post in the support forum with a clear indication of what you are trying to do, what isn't working, etc. then perhaps someone can help!?

Having said that, perhaps the 'getting started with TurnKey' tutorial might be of some use?

If using ProcessMaker is your issue, then perhaps check out what resources they have on their website, like the tutorials or their wiki.

Pages

Add new comment