Introducing TKLDev - Turnkey's appliance development and build system in a box

Today we proudly announce the official release of TKLDev, the new mothership of all TurnKey Linux appliance development. With TKLDev, building TurnKey Core from scratch is as easy as running make. If you can handle that you're ready to roll:

root@tkldev ~$ cd core
/turnkey/fab/products/turnkey/core
root@tkldev turnkey/core$ make

As you might expect, we're pretty stoked to finally (finally!) be able to make this announcement. It's the culmination of a couple of years worth of unglorious work behind the scenes upgrading the development plumbing that makes TurnKey possible.

I personally believe it's also a critically important milestone for TurnKey, a project I've poured my heart and soul into and would like to see grow up to embrace it's full potential.

So what's the big deal with this new build system?

Here's the problem: as much as we'd like to be the omnipresent, omnipotent, and omniscient developers that can do everything ourselves there is way more awesome open source software out there than we (or any other closed group of developers!) could ever hope to bottle up into a working library of ready-to-use solutions. Not without a great deal of help anyhow!

We started this project about five years ago. It took us a couple of years to start realizing we had a problem and another year or so to realize we had a really big problem. TurnKey was all about open source but it wasn't truly open source itself because the relationship of the core development team with the community was pretty much one sided. We could look out, but nobody could look in.

We tried plugging the hole with TKLPatch, at least as a temporary stopgap, but that didn't work and we think part of the reason was that we hadn't gone nearly far enough. It wasn't enough to make it possible for the community to customize appliances. That just made the community into limited second class citizens and while it may have been better than nothing nobody really wants to be a second class citizen in a purported open source project. Give me full access to the source code and build chain or give me death! (or... a casual hrmpf of disinterest).

We realized that meant TurnKey was either doomed to stagnate or we figure out a way to re-engineer the development process in a way that truly democratized it so that everyone could join in on the same level.

So as many of you know (we've been waving our hands around about this for a while) a couple of years ago we started working on this in earnest and we've been gradually re-engineering all our legacy build infrastructure with the ultimate goal of creating the mother of all appliances - a self-contained appliance that can be used to build any appliance (including itself) from source code.

Like Liraz said in a previous blog post:

We've actually had this in our sights for quite a while now. We made some bad technical decisions early on with regards to how we setup our legacy build infrastructure and realized a bit too late that we would have to redo everything if we wanted to get the open source community truly on board with TurnKey's development.

With everything on our plate (e.g., developing over a hundred appliances and the TurnKey Hub) it sometimes feels like we're running in place. So it's taken longer than we would have liked to make this happen.

TKLDev and appliance sources released

Long story short we've re-engineered our entire build infrastructure, bottled it up into a self-contained appliance and battle tested it by building the entire TurnKey 12.1 maintenance release. We also moved to GitHub to streamline collaboration with the community.

TKLDev is ready for immediate download in all the standard build formats in both 32-bit / i386 and 64-bit / amd64 architectures. You can also deploy it straight in the cloud via the TurnKey Hub.

Although TKLDev is relatively easy to use, it'll probably save you some time if you read through the documentation first to get an overview of how things works, development best practices, etc.

TKLDev lets you build any TurnKey appliance from scratch, directly from source. You can find the source code for all appliances in the TurnKey library at github.com/turnkeylinux-apps. All our other repositories live at github.com/turnkeylinux. We put appliances in a separate GitHub account because with so many of them they need their own namespace.

The way forward - TKL 13.0 Wheezy, a community effort?

A while back we announced release candidates for TurnKey Core 13.0, but then back tracked a little with the 12.1 maintenance release, and then side tracked again with TKLDev and the move to GitHub.

Well, it was all for a very good cause but I think we're done playing hop-scotch. With all the pieces now in place, we're moving full force on the TKL 13.0 release. The first things we'll be doing is updating all the TurnKey specific components, Core and TKLDev for 13.0. With that, the infrastructure is set to start upgrading the appliances.

We're hoping with the release of TKLDev and full appliance source code that the upcoming release will be more of a community effort. Thanks to the new infrastructure there is no special sauce, no wizard behind the curtain driving TurnKey development. We're giving Linux geeks everywhere the (super?) power to build any appliance, including Core, from scratch. What you do with it is limited only by your imagination but we're hoping at least a small part of that imagination will go towards helping us improving TurnKey for everyone's benefit.

Hopefully, we've made this stuff easy enough to use so nearly anyone with the inclination can jump in and make cool stuff happen. All the hard, really technical bits are automated so you don't need a lot of Linux experience to work with TKLDev. The same amount of experience you'd need to install and configure a regular Debian installation will take you a long way. The difference being that you can mass reproduce the results and collaborate with others on the development in ways you couldn't do otherwise.

If we've done our job well, we won't be able to anticipate all the different ways the community is going to put TKLDev to use. Some of the basic uses should be pretty obvious though. Let's say you want to improve an existing appliance. Maybe upgrade an appliance to use the latest upstream versions of your favorite open source application. Or maybe you've come up with a cool new feature that will make TurnKey Core (and by extension all other appliances) more useful? Fork, update, test, send a pull request, and your commit becomes an official part of TurnKey. It's that easy.

I can't wait to see how this evolves. Lets get hacking!