Introducing TKLDev - Turnkey's appliance development and build system in a box

Today we proudly announce the official release of TKLDev, the new mothership of all TurnKey Linux appliance development. With TKLDev, building TurnKey Core from scratch is as easy as running make. If you can handle that you're ready to roll:

root@tkldev ~$ cd core
/turnkey/fab/products/turnkey/core
root@tkldev turnkey/core$ make

As you might expect, we're pretty stoked to finally (finally!) be able to make this announcement. It's the culmination of a couple of years worth of unglorious work behind the scenes upgrading the development plumbing that makes TurnKey possible.

I personally believe it's also a critically important milestone for TurnKey, a project I've poured my heart and soul into and would like to see grow up to embrace it's full potential.

So what's the big deal with this new build system?

Here's the problem: as much as we'd like to be the omnipresent, omnipotent, and omniscient developers that can do everything ourselves there is way more awesome open source software out there than we (or any other closed group of developers!) could ever hope to bottle up into a working library of ready-to-use solutions. Not without a great deal of help anyhow!

We started this project about five years ago. It took us a couple of years to start realizing we had a problem and another year or so to realize we had a really big problem. TurnKey was all about open source but it wasn't truly open source itself because the relationship of the core development team with the community was pretty much one sided. We could look out, but nobody could look in.

We tried plugging the hole with TKLPatch, at least as a temporary stopgap, but that didn't work and we think part of the reason was that we hadn't gone nearly far enough. It wasn't enough to make it possible for the community to customize appliances. That just made the community into limited second class citizens and while it may have been better than nothing nobody really wants to be a second class citizen in a purported open source project. Give me full access to the source code and build chain or give me death! (or... a casual hrmpf of disinterest).

We realized that meant TurnKey was either doomed to stagnate or we figure out a way to re-engineer the development process in a way that truly democratized it so that everyone could join in on the same level.

So as many of you know (we've been waving our hands around about this for a while) a couple of years ago we started working on this in earnest and we've been gradually re-engineering all our legacy build infrastructure with the ultimate goal of creating the mother of all appliances - a self-contained appliance that can be used to build any appliance (including itself) from source code.

Like Liraz said in a previous blog post:

We've actually had this in our sights for quite a while now. We made some bad technical decisions early on with regards to how we setup our legacy build infrastructure and realized a bit too late that we would have to redo everything if we wanted to get the open source community truly on board with TurnKey's development.
With everything on our plate (e.g., developing over a hundred appliances and the TurnKey Hub) it sometimes feels like we're running in place. So it's taken longer than we would have liked to make this happen.

TKLDev and appliance sources released

Long story short we've re-engineered our entire build infrastructure, bottled it up into a self-contained appliance and battle tested it by building the entire TurnKey 12.1 maintenance release. We also moved to GitHub to streamline collaboration with the community.

TKLDev is ready for immediate download in all the standard build formats in both 32-bit / i386 and 64-bit / amd64 architectures. You can also deploy it straight in the cloud via the TurnKey Hub.

Although TKLDev is relatively easy to use, it'll probably save you some time if you read through the documentation first to get an overview of how things works, development best practices, etc.

TKLDev lets you build any TurnKey appliance from scratch, directly from source. You can find the source code for all appliances in the TurnKey library at github.com/turnkeylinux-apps. All our other repositories live at github.com/turnkeylinux. We put appliances in a separate GitHub account because with so many of them they need their own namespace.

The way forward - TKL 13.0 Wheezy, a community effort?

A while back we announced release candidates for TurnKey Core 13.0, but then back tracked a little with the 12.1 maintenance release, and then side tracked again with TKLDev and the move to GitHub.

Well, it was all for a very good cause but I think we're done playing hop-scotch. With all the pieces now in place, we're moving full force on the TKL 13.0 release. The first things we'll be doing is updating all the TurnKey specific components, Core and TKLDev for 13.0. With that, the infrastructure is set to start upgrading the appliances.

We're hoping with the release of TKLDev and full appliance source code that the upcoming release will be more of a community effort. Thanks to the new infrastructure there is no special sauce, no wizard behind the curtain driving TurnKey development. We're giving Linux geeks everywhere the (super?) power to build any appliance, including Core, from scratch. What you do with it is limited only by your imagination but we're hoping at least a small part of that imagination will go towards helping us improving TurnKey for everyone's benefit.

Hopefully, we've made this stuff easy enough to use so nearly anyone with the inclination can jump in and make cool stuff happen. All the hard, really technical bits are automated so you don't need a lot of Linux experience to work with TKLDev. The same amount of experience you'd need to install and configure a regular Debian installation will take you a long way. The difference being that you can mass reproduce the results and collaborate with others on the development in ways you couldn't do otherwise.

If we've done our job well, we won't be able to anticipate all the different ways the community is going to put TKLDev to use. Some of the basic uses should be pretty obvious though. Let's say you want to improve an existing appliance. Maybe upgrade an appliance to use the latest upstream versions of your favorite open source application. Or maybe you've come up with a cool new feature that will make TurnKey Core (and by extension all other appliances) more useful? Fork, update, test, send a pull request, and your commit becomes an official part of TurnKey. It's that easy.

I can't wait to see how this evolves. Lets get hacking!

Comments (31)

This was a bug in deck

Reply 1 2013-07-23

Hi Eric,

Awesome that you've already ported your patches to TKLDev. I'll ask Alon about forking your repos into turnkeylinux-apps so we can include them in the next release.

Anyhow, I traced the problem with make clean to a bug in deck which I committed a fix to earlier today.

Cheers!

[update]: Alon updated the archive with the bugfixed deck package.

Added these to list of TKLDev community projects

Reply 2 2013-08-01

Figured it would probably be a really good idea to create a page on the wiki tracking community TKLDev projects like the three you listed above.

I also added a couple more TKLDev projects (Minecraft, Observium) I found on your GitHub profile that you didn't list above. You've been holding out on us!

So far, so good!Deployed and

Reply 3 2013-07-20

So far, so good!

Deployed and setup easy enough; took me about 30 minutes to convert a tklpatch to the new fab system. I took my three most popular applainces (subsonic, nzbapp and newznabapp) and migrated them without difficulty. My github repos are updated so others can try them out if they'd like:

https://github.com/tssgery/subsonicapp

https://github.com/tssgery/nzbapp

https://github.com/tssgery/newznabapp

I did find a little issue with running "make clean" on an appliance but if you do what the error message says, your'e good to go. Details are at: /forum/general/20130719/tkldev-make-clean-not-working-properly-theres-easy-fix and I may find the time to submit a pull request for a fix. Gotta love opensource stuff.

Here comes the first pioneer

Reply 4 2013-07-19

Hi Eric! Looking forward to your feedback as soon as you give TKLDev a test ride.

YES!!!!

Reply 5 2013-07-18

This is great news! I have hobbled together a similar appliance that builds my patches, which I can now obsolete and throw away! Which is fantastic! I will be giving this a try this weekend!

First the moon, then the outer planets

Reply 6 2013-07-19

Adrian, I like your vision. There are a few features missing for TKLDev to become a fully fledged autonomous, planet colonizing von Neumann probe, but eventually, why not! I promise to try and integrate it with first viable open source AI, then maybe it will improve itself recursively. That would be a big win.

Cool!

Reply 7 2013-07-18

Great news again! I was about to ask via twitter as I already have a couple of projects to do with TKL! I hope to get some time to download, test and give some feedback.

For what I understood from the post, if you send this TKLDev to the space on it's own, it could reproduce herself and colonize a planet? :O

I'll check the docs and keep in touch!

Yes there is! :)

Reply 8 2013-07-19

The 64 bit version is the default (from the links on the site) but 32 bit version is available from the TKL SourceForge repo. If you have a look in the files section under your preferred medium (e.g. ISO) you'll find it (e.g. turnkey-tkldev-12.1-squeeze-i386.iso)

Happy building! :)

Added a link on the blog post and tkldev app page

Reply 9 2013-07-19

Thanks for pointing out that it's not so obvious where the 32-bit image is. Other people might be wondering about that. I've added a couple of links to the blog post and tkldev app page so that they you don't have to read through to JedMeister's comment (helpful as always) to find it.

I'm excited that you're excited!

Reply 10 2013-07-19

Thanks for the encouragement Dan, and for showing up for all our important announcements. I know you've been following TurnKey's progress for a few years now...

Great News

Reply 11 2013-07-18

This is excellent! I am excited to see TKL continue to grow and become even better.

Yes, LXC will run inside a VM. It's an OS-level feature.

Reply 12 2013-07-19

Unlike other virtualization techniques, LXC is implemented inside the Linux kernel as an operating system level feature. Like chroot, but with more features. All containers share the same kernel so for security's sake you have to limit certain permissions inside the container even from root (e.g., the ability to insert/remove kernel modules, see processes outside the container, etc.) but you get much higher efficiency in return.

LXC works fine with TKL

Reply 13 2013-07-19

LXC works fine with TKL

The only problem I ran into, is that sometimes a shutdown inside a lxc container brings the host down.... But that was only if I had a redhat as host.

You can easily take the tkldev or a core and add some lxc and cgroup package to it, and then you can start up the other TKL appliances inside that host, GREAT FOR TESTING...

I am running ( 2 years now) the TKL lamp container and TKL svn container on the same host , no problems

startup time is a few seconds for a container and the container only runs the applications, no kernel.

The only overhead you have is the diskspace needed for a container.

And LXC runs just fine inside vmware or virtualbox

LXC would be cool!!

Reply 14 2013-07-19

Thanks Liraz. Had a quick look at LXC and it looks pretty interesting.

FYI - my motivation for multi-app appliances is to minimise the cost for my SMB clients. There is a minimum cost for a VM since any VM is always doing something even if doing nothing (obviously). Add on a network monitor and an external heartbeat poll and the cost of a VM doing "nothing" goes up. Where an SMB I.T. infrastructure includes multiple apps that play well together (repository, network monitor hub, document management system), the overall service cost is reduced considerably by grouping apps in fewer VMs. LXC seems like it would help to simplify the lifecycle management of a multi-app VM (assuming that LXC can be run within a VMWare, AWS, etc VM of course).

Multi-app appliances are a very interesting area for development

Reply 15 2013-07-19

Thanks Tim, I'm looking forward to seeing what you come up with so keep us in the loop.

BTW, we've been thinking about multi-app appliances for about as long as we've been thinking about TKLDev and friends. Unfortunately we haven't had the time to tackle this one ourselves yet but we do have a few ideas. To sketch the outline, I envision a meta TurnKey appliance that supports running multiple TurnLey inside thin containers that share the same kernel. LXC would be a good candidate component for this. I gather it's improved a lot in recent Linux versions. Last I checked there were issues with the quality of isolation between containers but you don't necessarily need good isolation to get this to work, that would just give you a big security bonus.

Spectacular achievement guys. Leaves Bitnami in the dust :-)

Reply 16 2013-07-18

Well done guys. With this TKLDev release you have turbo-charged what was already a key backbone of the Cloud revolution...Turnkey Linux.

I'm out of the country for the next month but when I return I will be starting to build some multi-app appliances which I will be contributing back to the community.

Rock & roll!!

Thanks! Hopefully this will be just the start

Reply 17 2013-07-19

Thanks Jeremy! Alon and I have tried pushing the envelope in terms of how far we could stretch out our resources, but we also ran many times head first into our limitations and I'm hoping with TKLDev in the hands of the community our limitations will no longer be TurnKey's limitations.

Fingers crossed, we'll see the open source development model finally unleashed in this neck of the woods. That would be exciting!

Great news Guys!

Reply 18 2013-07-19

Congratulations. I know that this has been a big dream of yours (and mine too) so it's great to see it finally realised...

I'm still not sure when I'll have time to have a proper play with it but I'm super excited! :)

Finally!

Reply 19 2013-07-19

This looks promising...

No coincidence!

Reply 20 2013-07-19

We've been keeping close tabs on your activity in the forum and have identified a pattern of vacation time around July. TKLDev's release was carefully timed...

Great !!

Reply 21 2013-07-19

Great !!

Since my vacation is just starting, I can't think of a better timing... will be a interesting learnfull vacation !

Have a look at this thread

Reply 22 2013-07-23

/forum/general/20130722/tkldev-build

Crashed with USB 3.0 only system

Reply 23 2013-07-25

It run into initramfs when system has only USB 3.0. I used 'noacpi nomodeset' at grub, did I missing something?

Agreed.

Reply 24 2013-11-25

Hey Adrian! Glad to see you pitching in...

Docker is basically an advanced wrapper around LXC. I've played around with it and was sold. Unless we run into any major obstacles, TurnKey integration is in the (near-term) future.

+1 Docker integration

Reply 25 2013-11-05

+1 Docker integration

I made some experiments in the past with LXC and TKL, but never get to something useful. Now the idea of Docker for TKL sounds very good, considering that the open PAAS environment may be moving towards Docker also as a way to enable portability between platforms...

My 2 cents,

TurnKey Docker optimized builds available

Reply 26 2013-12-29

We've just announced TurnKey Docker optimized builds, see the announcement for full details.

Docker integration will happen soon

Reply 27 2013-11-25

Hi Chux,

Thanks for the feedback and sorry for the late reply. We're aware of docker and are just as excited about it. I think it has the potential to solve two important problems:

As a distribution channel, e.g., get TurnKey into more places
Support lightweight multi-app configurations

It's on the shortlist of things we're looking at following the release.

A few thoughts...

Reply 28 2013-11-25

The source code for all the TKLDev builds are on GitHub so there are plenty of examples (the whole TKL library of them!). Note that it is a separate repo to the TKL one (IIRC it is called TKL-apps or something similar).

There is no issue basing your appliance on the PostgreSQL appliance (rather than the LAPP appliance). Just use the PostgreSQL appliance as your base (similar to what you may have done with TKLPatch).

As for your source code on LaunchPad, is that code you have created yourself or the code for the OpenERP software? If it's code you created then you may be able to import that into GitHub. If it's the source for OpenERP itself then as Liraz said you should be able to install it, same as you would have done with TKLPatch.

Point m to a Surce w/ Git and BZR sources

Reply 29 2013-11-25

It would be great if I could look at a source which is calling Git, Launchpad and Python sources (I expect Python is in them all).

If you knew one of the Git/Source/Appliances I could look at that would be great.

OpenERP, btw, should be forked from PostgreSQL Appliance as it has its own Web Server, so Apache is not needed (and is somewhat in the way). PostGresSQL uses the "lightspeed" (or similar named) server for the Database interface.

Ideally Ports should be contollable in Webmin so that perhaps OpenERP took 80/443 and PostGre took a off highway port.

Room for improvement in the docs...

Reply 30 2013-11-25

Hi Arnold,

Thanks for weighing in. There's definitely room for improvement in the docs. We should look into that. The process for using TKLDev is as follows:

Deploy thel TKLDev app and follow the setup instructions. I propose building TurnKey core as a test case. If that works you know you're ready to start building your own apps.
Fork the TurnKey app sources for the app that's closest to your use case. If you want to develop OpenERP you should probably look into the LAPP or PostgreSQL apps.

I propose taking a look at the sources for a few apps to understand what is going on. You can install software from any source, including Launchpad. You just have to provide the installation instructions (e.g., like you would have to do in a TKLPatch) and test that they work.

Cheers

Going through the documentation... 12 pages at least..

Reply 31 2013-11-25

I have been reading through the documentation and It took about 11 pages to get to the "answer" of my first question. How do I run TKLDEV on a New Appliance Build?

I had been thinking that, like TKLPatch I could install TKLDEV into the system and do a build. That appears not to be the case.

As I understand it the new Appliance needs to be built, then Forked back up to Git, then pulled from Git into TKLDEV for the ISO to be built.

Question though, if part of the appliance I am building is hosted on LaunchPad am I "out of luck" or can I somehow specify the bzr sources in the new System Build?

I am wanting to do an OpenERP appliance on top of the PostgreSQL appliance. Perhaps there are some Git Sources for OpenERP (a confusing number but none that I can find is CLEARLY the source).

All guidance is appreciated.

(small feedback, docs should perhaps have an outline structure... and as an aside not sure why I would want to make an ISO of Core, other than going through the steps.. Perhaps illustrating early on How to convert Core into Lamp, and make an ISO of that would be more telling of the process involved) My thoughts anyway. Thanks for all the hard work here!