v14.0 stable release - Massive Community Effort!

Not sure how long it will take. Actually TBH I'm not even sure how much work they will need. I anticipate that we may need to make some adjustments for systemd but we'll see...

Hello, this looks great and comes just in time for a new project.

Will the docker images be pushed to 

https://hub.docker.com/search/?q=turnkeylinux

soon? That would be most welcome!

Version 14 was just in time for two Debian Jessie systems I needed to install....      Turnkey makes install,  backup and recovery easy , safe and reliable.     For us  backup and recovery are a very to have set up for any system. B&R has worked perfect for us.  We run all our systems on a local network.  With Turnkey backups I know that most of our systems are 5-10 minutes away from running on amazon using turnkey hub. 

Thank you to all who produce Turnkey .

As for the note re lxc-create; that's awesome to know. And will help when we get that far...

FYI I'm currently battling with the headless builds. I'm having issues with the fencing under LXC. It worked great under OVZ and from what I can see the LXC appliance works around it by preseeding the inithooks (although TBH I never actually used LXC before)...

Note, I have called the package "lxc-dev-tkl" because the Debian Jessie lxc package "conflicts" with "lxc-dev" (so if I called it "lxc-dev" then it would uninstall lxc!).

My testing so far confirms that the Jessie versions of LXC and Python meet the requirements for Ansible's lxc_container management.  However when I try to install the lxc bindings for Python2 using pip, I get a failure because lxc-dev is missing.  I can't install that package because it was withheld from Jessie due to some issues. 

LXC has embraced Python3 and has built those bindings into LXC.  Ansible on the other hand is holding back on adopting Python3 and it's scripts will fail if you try to use it, hence the need for the Python2 bindings.  I can't figure out how to build and install them on Jessie without having the lxc-dev package.  In Ubuntu, the desired bindings are packaged as 'python-lxc'.  See https://launchpad.net/ubuntu/+source/python-lxc

I also discovered that lxc-create now uses xz compression for downloading images, but there appears to be a missing dependancy on the package 'xz-utils'.  That will need to be added to plan/main.

I anticipate that the new templates we will provide for v14.0 will be labelled as LXC/OVZ templates but will essentially be the current v14.0 appliances built with the existing OpenVZ build code. Preliminary testing shows that only a bit of a tweak (remove systemd and replace with sysvinit) is required. The tweak is because the version of SystemD in Jessie does not play nice in a container. Apparently upstream SystemD has already resolved this so when we get to v15.0 (Debian Stretch - currently 'testing') we'll be all good...

Re your required versions; other than lxc-python2 (which you note we can get with pip) it appears that you're in luck:

# apt-cache policy lxc python lxc-python2
lxc:
  Installed: (none)
  Candidate: 1:1.0.6-6+deb8u1
  Version table:
     1:1.0.6-6+deb8u1 0
        500 http://ftp.au.debian.org/debian/ jessie/main amd64 Packages
        500 http://security.debian.org/ jessie/updates/main amd64 Packages
python:
  Installed: 2.7.9-1
  Candidate: 2.7.9-1
  Version table:
 *** 2.7.9-1 0
        500 http://ftp.au.debian.org/debian/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
N: Unable to locate package lxc-python2

Both of the above would be included in the LXC appliance anyway; and it sounds like the python library for interacting with LXC would be useful to include OOTB too. So unless there is any other significant requirements it sounds like that would be a goer...

Re your comments about building LXC in TKLDev, I have a few comments to make:

Firstly we have intention to improve on TKLDev by moving from building in a chroot to building in a container. We may use LXC although we are not yet committed. Personally I think that systemd-nspawn may be a better alternative? I'm not sure whether that will happen in time for v14.1 or not though. Ideally IMO we would also include QEMU too so we can do non-native architecture builds (I'm thinking ARM).

So concurrently to that; we would also be aiming to allow the building of all appliance build types in TKLDev. You can already leverage buildatsks to build any of the appliances in any of the build types. But I think that it would be neater if it were baked into TKLDev and we could specify the desired buildtype at build time... In the short term that would probably be a case of including buildtasks pre-packaged into TKLDev; but longer term it would probably require more than that so it is an integral component...

Beyond some of the subtleties I think that your plan is a great one. Actually beyond some of your specifics it is very similar to discussions that Alon, Liraz and I have had...

A framework to provide for scripted automated testing of appliances is already in the works (currently using the TKLDev build chroot; but eventually the build container). However it is not a current priority so there is no ETA. I would hope that the basics would be in place for v14.1 though. I anticipate that once it is to a usable point it will be public and posted on GitHub...

@Jeremy, could you use some help with the LXC appliance?  I'd like to see some changes that would allow the new Ansible appliance to manage containers on the LXC appliance using Ansible's lxc_container module.

I'd also like to break the dependance on openvz images for the lxc-turnkey template. Are there plans to provide pre-built LXC images on the TurnKey mirrors?

Also for 14.1, I'd like to be able to build LXC images in the TKLdev appliance.  The usecase I have in mind is for developing a custom appliance, i.e. one which is not in the official TurnKey Suite.

1. Release to GitLab or GitHub triggers Jenkins causing it to launch an Ansible playbook
2. Ansible playbook causes TKLdev to download latest appliance via git and launches build
3. Ansible playbook launches lxc_container using LXC image from TKLdev
4. Ansible playbook runs blackbox tests for functionality, security, etc against running LXC container
5. If all tests pass, Ansible playbook pushes appliance to production

I can tell you the specific appliances that have been depreciated for v14.0 (or more correctly since the release of v12.1):

It was a nice surprise to see the release of 14.0 on the very day I returned from a month long trip to Europe. Tip of the hat to Norway and Austria.

Thanks, Jeremy, for your kind words. I know that the changes I suggested were cause of some of the delay and they added greatly to your workload.  Thanks for having faith and being willing to take on such large changes so late in the release cycle.  That's part of what makes it great to be a member of the TurnKey GNU/Linux community. I was pleased to see how smoothly the install of the 14.0 Ansible server proceeded. The entire TurnKey community owes you a round of applause for taking on a difficult task and seeing it to a successful conclusion.

Huge thanks to everyone involved in bringing 14.0 and the appliances to release.

I have the ownCloud appliance up in a VM and it's all working very well (after I got my head around its "Trusted Domain" thing). 

Phil

Currently you could use TKLDev and use git to maintain your own tweaked branch. When there are updates you pull them into your tweaked branch and then (re)build ISO (or whatever build you like with Buildtasks).

Having said that I understand that having a simple TKLPatch possibly makes more sense in some scenarios. Also some may prefer to work with modified "official" builds (rather than building their own)...

I would like to see TKLPatch or something like it continue to be supported. I use it extensively as a way to setup defaults for the different packages from the base turnkey appliances which are unique to my environment. For instance, I have several patches that configure different settings for SAMBA for my own environment which I combine into one master patch to create my own flavor of the fileserver appliance. I then leverage most of those same patches to customize the torrentserver appliance since I want most of the same settings applied to the SAMBA installation on that appliance. Using the TKLPatch mechanism allows me to easily use one patch for multiple appliances, and the nesting capability allows me to ensure that a collection of patches gets applied in the correct order if there are any dependencies between patches.

Since these are customizations that I'm making to the official releases for my own environment, I don't think they belong in the official app repository, and if I try to simply maintain separate branches of the app repos, any updates to get a SAMBA patch working on the fileserver appliance would have to be manually copied over to the torrentserver appliance rather than just running the TKLPatch against the torrentserver appliance. This would reduce efficiency since I would essentially be duplicating work and makes it harder to maintain bug fixes.

Now, maybe there is a mechanism in the TKLDev to handle this scenario already which I haven't discovered. If so I would be willing to use it in place of the TKLPatch. If not and you don't want to support both TKLDev and TKLPatch, I would like to see some sort of patching capability added to TKLDev. Something like make-patch which would take an argument of the patch location (folder or archive) and apply it to the root.patched or root.sandbox.

BTW, I love what you all have done with Turnkey Linux and especially the TKLPatch/TKLDev products which greatly aid a part-time sysadmin like me in maintaining Linux servers.

You could try doing this:

fab-install /path/to/turnkey-core-14.0-jessie-amd64.rootfs/ isolinux

Although ultimately perhaps it would be better to just convert your patch to TKLDev build code and build from scratch? We don't intend to put a lot of effort into maintaining TKLPatch in the longer term whereas TKLDev will get lots of loving! :)

The fix hasn't yet been merged into the main TurnKey repo. But once that's done then hopefully very soon after Alon will rebuild TKLPatch and update the v14.x repo with the fixed copy.

If anyone wants to test this out then the changes that I made can be inspected on GitHub here. You can replicate those changes by manually editing your local /usr/bin/tklpatch-prepare-cdroot to match the commit (i.e. remove the one line in red in the left pane; and add the 6 lines in green on the right).

Also I probably should have opened an issue regarding this on the tracker ages ago. But better late than never... :)

For the record an OVA is essentially a VMDK and an OVF bundled together in a single (OVA) file which most application which can read an OVF will support (I have tested in VirtualBox and VMware Player and both work fine OOTB). The beauty of the OVA format is that you can use it as is and there is no need to unzip!

What I mean by that is that under the hood, TurnKey is Debian. So you can use the Debian upgrade path if you desire.

However TKLBAM is the recommended method. Please note that there are some significant changes between v13.0 and v14.0 so it is likely that some post migration tweaks will be required. Ideally I'd like to automate some of the common tweaks to make it easier but unfortunately I haven't got there.

The beauty of TKLBAM is that you can restore your backup to a new server and treat that as a test migration. If you document anything you need to do then once you have everything working as you want; destroy your test server; do a fresh backup (to get all the latest data); restore to a new server (following your documentation). Then (assuming it's an EBS baked server) you can stop your old server and do finals config to make your new server live. Once you are satisfied that everything is 100% good then you can destroy your old server and your old backups. Personally I'd wait a few days at least...

You might be interested in knowing that we made all of the appliances for 14.0 available on our VPS hosting platform, and each image can be installed with just one click. This sets up a base installation with sensible default.

We wrote a little about it at https://www.gigatux.com/news/index.php/2015/11/09/turnkey-linux-14-now-a... and we make these images available to all via our Debian repository.

This is a fairly old blog post which relates to an old release. As such, it isn't super relevant anymore. As it seems to be attracting a lot of spam posts, I'm going to lock it. Please feel free to comment elsewhere or better still sign up and start a new thread.