You are here
New release candidates for TurnKey Linux 11.0 (part 1)
We've pushed out new RC (Release Candidates) builds for part 1 of the upcoming TurnKey Linux 11.0 release and we need your help testing them! See the appliance pages for download links.
The current crop of release candidates only include Ubuntu Lucid based ISO images for now. Debian Lenny based images will follow, as will builds specially optimized for the the full range of supported virtualization and hosting platforms (e.g., VM build, EC2 AMIs, ESX4, Xen, Eucalyptus, etc.).
In part 1 we've updated all the existing appliances except for Zimbra and Openbravo which we haven't got to yet. They'll be in part 2 along with all the fantastic new appliances developed during the community development contest.
We broke up work on the next release into two parts because with all the new appliances the size of the appliance libary is doubling and that was a bit too much to handle in one go.
Also, Alon is flying off to Orlando today for the Ubuntu Developer Summit and we wanted to have something out the door to show off to our friends in the Ubuntu community.
New versioning scheme
Instead of date based versions the new versioning scheme tracks TurnKey Core, regardless of the date or the underlying distribution.
In other words a TurnKey Linux 11.0 appliance based on Ubuntu Lucid and a TurnKey Linux 11.0 appliance based on Debian Lenny are the same version because they have the same Core features.
We're also hoping to get rid of some of the confusion with date based releases. For example, we pushed out the maintenance release 2009.10-2 in April 2010 but the version confused many people into thinking the appliances were obsolete, even though Hardy will be supported for another 3 years and security updates were installed on first boot and daily afterwards.
Now built-in: TKLBAM (TurnKey Linux Backup and Migration)
TKLBAM, which we announced a few weeks back as a separate component, is now tightly integrated with TurnKey out of the box. This includes a spiffy new Webmin module so users who dislike the command line can fully enjoy TKLBAM from the comfort of their web browser. Now you can backup and migrate fully working systems anywhere with literally just a few mouse clicks.
As an old school Unix guy who loves the command line, what I like the most about the new Webmin module is how it facilitates easy discovery of TKLBAM's functionality and ties everything together seamlessly. No need to memorize funny command line flags or decipher manual pages for configuration options. At a glance you can see what can be done. If you don't understand something you just click on the question marks for embedded help pop-ups.
Screenshot #1: backup menu
Screenshot #2: restore menu
New Core features
Besides TKLBAM integration, we've made a broad range of improvements to the Core features shared by all appliances. In fact one could argue more has changed between the last beta and the release candidate than between the previous release and the beta.
-
Basic configuration dialogs on first boot: Previously only ISO builds would prompt you, during installation, to configure the appliance (e.g., set passwords). Users of the pre-installed builds (e.g,. the virtual machine build for VMWare/VirtualBox) had to do basic configuration by hand.
With the new mechanism, the user experience is consistent for all build types. Also the first boot configuration mechanism is modular so appliance customizers can easily add their own hooks.
-
LVM (Logical Volume Management) support: instead of installing the appliance filesystem to a naked partition, we setup LVM by default. This makes it much easier to adjust storage capacity later. Also it adds basic support for filesystem snapshots, which future versions of TKLBAM may leverage to support new kinds of databases.
-
auto-apt-archive: uses GeoIP to automatically configure closest APT package archive for maximum network performance.
-
Command line convenience: a range of small improvements that make working with the command line a bit more comfortable:
-
smart, programmable bash shell completion: helps you get more done with fewer keystrokes.
-
support for $HOME/.bashrc.d hooks (see my blog post about shell hooks)
-
persistent environment variables (see $HOME/.bashrc.d/penv):
penv-set foo=bar exit # later... echo $foo
-
-
Inverted webshell color scheme: real men use white on black for their command line shells. And they like it!
- Display system info in the login motd: seen in the above screenshot.
Appliance specific changes
- The latest and greatest: all appliances are now built using the newest software packages available with Long Term Support from Ubuntu. For software not available in the package management system (e.g., redmine), the latest stable software versions from upstream are used instead.
-
A range of performance optimizations: for example...
- All LAMP-based appliances now includes xcache for PHP opcode caching which improves PHP performance, as code only has to be recompiled by PHP into opcode if it changes, and not on every page load.
- All Rails based appliances now come with Ruby Enterprise which improves memory performance, traditionally one of Ruby's weak points.
- Many small improvements and bug fixes
- Exciting potential for new bugs: as yet unidentified.
Distribution level changes
-
Upgraded base distribution: from Ubuntu 8.04.3 to Ubuntu 10.04.1
-
The good: newer packages based on a new Ubuntu LTS release which will be supported for 5 years.
-
The bad: roughly 50MB in additional bloat for all appliances. This is mostly a consequence of bloated dependencies within the new Ubuntu version. Some of the dependencies are unnecessary for our use case (e.g., plymouth) but the only way to get rid of them is to fork the packages and we'd rather avoid that.
Fortunately, the upcoming Debian Lenny based builds do not have this extra bloat so anyone who cares will have an alternative.
-
-
No more pinning: As discussed earlier we've discontinued use of complex package management configurations to achieve mixed hybrids of Ubuntu and Lucid.
In the works...
- Debian Lenny based equivalent builds
- PostgreSQL support for TKLBAM
- Part 2 of the TurnKey with dozens of new appliances
- 64-bit support (it's about time!)
That's all folks! Any questions?
Comments
Congratulations
The webmin module for TKLBAM is the most interesting new thing :) . Also using LVM by default is a good decision.
KUDOS!
Nice work! we were expecting the post. Today I'm testing the Core Appliance, and for what I've seen it's behaving real nice! But I'm writing down my feedback and I'll post it soon.
For me, the nicest features are the dialogs on first boot, the webmin module and LVM. Of course fixing the network bug and removing byobu is also very welcome!
Great achievement!
Looking mighty fine!
Very nice work guys!
I particularly like the way that TKLBAM is so tightly integrated that it definately looks like a fundamental part of the appliance (as it indeed now is)!
The new versioning scheme makes lots of sense (although personally I didn't mind the date based system). The config questions running on first boot, makes install and config consistent regardless of the image format which is very nice!
All-in-all an awesome RC release!
Congratulations
Congratulations on yet another milestone (they seem to come so rapidly - amazing given the human hours available). I'm digging in straight-away with dokuwiki, and I'll see where I'm taken from there. Looking forward to giving any that I can find a use for a go.
Great work,
Rik
---update---
Installed the Lucid RC of Dokuwiki into a VMware VM using Workstation. Wonderful stuff. Love the integration of TKLBAM and how it's worked into both confconsole and, more brilliantly, into Webmin. Love the opportunity to configure TKLBAM on first boot after installation along with the root password. Installing from the ISO to VMware was more painless than installing from repos onto a Lucid server on bare metal, which I have found to be nothing short of a pain in the neck (yeah, could be a problem between my brain and the keyboard - but the point stands that there's no such problems with TKL's appliance).
One thing I've been thinking about is providing easy instructions for changing the default passwords. I know for many of us, it's a few clicks away and possibly a search in Google. For some, however, that may not be the simple process it appears to be from our perspectives. I've thought about this particularly since I'm hosting appliances at our students' site and realize things may be smoother for the end user if we provided more than the default credentials: actual guides to changing and setting passwords so the appliances are more likely to be secure; even if those guides are just links to other documentation, it might secure one more user who may otherwise stay vulnerable. Of course, we haven't done this because of time constraints. Obviously it's more than likely that neither of you have the discretionary time for documenting something like what I'm describing here. Just some thinking I thought maybe I'd share.
Good thinking on passwords Rik
I reckon thats a great idea. I imagine that the best place for specific resetting password info and other similar stuff would be on the Appliance Specific Wiki pages.
I think it would be great if all appliances had their own specific appliance wiki page (even if its only a blank placeholder page to start with) with a link from the (dev produced) appliance page. I guess it may be a little frustrating for some if they go to the Wiki and it's blank, but perhaps all the blank ones could include a basic template offering some suggestions of where to start looking for info and perhaps some links to relevant forums, wikis, base info, etc that are relevant to that appliance. Just adding my 2c in the mix :)
I also second the idea of appliances pages.
I suggested the same thing some time ago, you can see my last comment on this thread. There are some ideas for the webpage including the ones you're proposing here. I hope I'll receive some feedback from the guys, once they come alive again on the forums. And yes, it's a lot of work. But we can do it, it's a matter of stablish a standard "template" so we can contribute but keep consistency on the information.
I forgot about that post of yours Adrian
I remember reading it and agreeing wholeheartedly with many of your suggestions. I had intended to post a response but obviously got sidetracked...
I will try to remember to read through it properly again soon and post back over there (and bump the thread up at the same time).
In a way I think that's why using a Wiki (or similar CMS) is the go, it means that we don't rely on the Core devs to do all the work, we can do some of it too!
It'd be great if Alon and Liraz got some more time here on the forums but I fear with the current appliance rollout we probably won't see a lot of them for a little while (a few weeks at the least).
Yep, the website needs some more work...
The website is overdue for another round of improvements. We're in complete agreement on that. How soon we get to it is mostly a matter of priorities (e.g., spend a few weeks refreshing the web site or work on 64-bit builds).
Resetting passwords after firstboot...
Sounds like a great idea!
Now that all that stuff runs at first boot rather than install thats a good option IMO. Cool!
Initial / resetting passwords
I'm a little late to the discussion, none-the-less...
I think TurnKey's mantra: what can be easy, should be easy. applies here. I'd like to add inithooks that allow the user to set the application passwords on first boot instead of having default passwords. This would alleviate the need for much documentation.
But, there is a catch - we still need to support deployment of appliances where the user does not have direct access to the console on firstboot, such as on Amazon EC2 and other hosted options. We can make the process seemless using the Hub and pre-seeding (but that hasn't been implement as of yet).
So, we would still need default passwords and have them documented some where. But this is still a step forward.
We could also add an option to the confconsole as Liraz mentioned allowing the user to reset passwords, but that means the inithooks need to be aware (or at least take into consideration) the state of when they are being run - as services might need to be stopped and/or restarted.
Some quick feedback
Great feedback
Regarding the removal of the boot from hard-disk option. This was my idea. Alon protested and said he found it useful during development. I argued that we should optimize the user experience for the common user/newbie as he's the one that needs our help the most. A more experienced user will figure it out. IMHO having just these two options make it clearer that you have to make a choice between installation mode and demo mode. The third option was fake. It didn't even do anything useful until after installation. Also, I don't like how it creates two equal but different ways of booting into an appliance after installation (e.g., remove the CD or choose the third option in the bootloader). One way is easier to explain.
I admit I just went with my gut here. If other people would like to weigh in either way it would be useful to get more perspectives.
Good reasoning
My son suggested that if Alon (and/or others) insist on having a 3rd option you could perhaps make it "Doh I forgot to take out the CD: open CD tray and reboot"! :p
But seriously; I initially missed the option, but I think you make a good case for removing it. And it certainly makes the options available more obvious. To make it really shine though I do think you want to auto eject CD after install and/or tell users to eject the CD.
I think the official ubuntu
I think the official ubuntu instalation tells the user to remove the CD. That would be the way to go. Believe me people would find themselves in a loop if they finish the installation and then they are presented with two options: install or try live!
Good advice from your son! That could be the message for the TKL Homer edition LOL.
Eject disc post installation (done)
This was a tough cookie! But it has now been implemented and works whether the installation is run via the boot menu option, or the confconsole when running in live demo mode.
The hardest part was interfacing with the user after the system has gone through the shutdown procedure (filesystems have been unmounted) and right before the actual shutdown. It gets even more tricky once the installation media has been 'ejected', which means we need to cache the relevant executables and libraries in memory, make the tty 'sane'.
Oh, and of course switch back to the correct VT console where the shutdown procedure is happening.
Anyway, the cdrom will be ejected if the installation completed successfully, and the user will be prompted to remove the disc (if any) and hit ENTER, or until the timeout is reached.
Awesome work!
Thanks Alon. Have you refreshed the ISO download or will we just have to be patient until the final release?
You'll have to wait, patiently.
We'll be releasing the final 11.0 (part I) soon once we squash all the outstanding bugs and issues, and create all the build targets. I also want to squeeze in setting of application passwords during firstboot (with preseeding support) if we have the time.
Warnings during installation
Great feedback Adrian!
I'm going to tackle the warnings during installation. From what I recall installing the MBR generates output which is displayed in the di-live console. Are there any other places during the install that generates warnings that should be fixed?
Don't display warnings to the user in di-live (done)
I solved the issue globally in di-live by redirecting stderr of all commands executed to the logfile (/var/log/di-live.log). The installer will still raise an exception for bad exit codes, but warnings will not be displayed. This also provides useful debugging information when needed (no need to watch warnings flying across the screen and hope that all is well.
Anyway, thanks for the push to fix this, it's long over due.
Thanks for the encouragement and feedback guys!
I only wish there were more hours in the day so we could be more active even in the middle of the deepest development cycles.
Providing non-S3 target locations for TKLBAM
When users are running TKL in their internal network (in a physical hardware or Citrix server ), it would be nice if the TKLBAM support target location like FTP Servers and WEBDAVs or even Rackspace Cloud file (since it is more cheaper that S3). In my humble openion , there should be a "drop down list" or "radio button" to select target location type :)
Hope to get to this before the release
If anyone wants to jump in and send patches that add support for "manual" mode in a way that doesn't compromise too much on usability, that would be a good starting point.
Thanks for your work
I stumbled upon Turnkey a few months ago but only downloaded the first Appliance yesterday.
I realize now that I should have done this sooner.
I like it very much and am looking forward to using the final 11.0 release.
Keep up the good work!
PS. Also thank you to Adrian & Basil for working real hard to provide additional Appliances.
2nd for non-S3 targets for TKLBAM
First, excellent work with these appliances, they are fantastic.
I agree with Basil Kurian, it would be great to have other target options for TKLBAM. I will be using these appliances at work and we already have an offsite backup solution in place. However, the TKLBAM interface (GUI and CLI) are much nicer than our current backup software interface, so I'd much rather use that to send the backup to our SAN via SMB which is already set for nightly offsite backups.
It would also be nice to have options like NFS and iSCSI or even over an SSH tunnel.
Thanks and keep up the good work!
Just to clarify...
You may already realise. but it is possible to use TKLBAM to backup to targets other than S3. You do still need to use the Hub, part of which involves signing up for an S3 account (and providing credit card details - although you never have to actually use it).
TKLBAM will automatically use S3, but if you use the (CLI only) --address option you can send your backups pretty much anywhere. From the TKLBAM FAQ:
I assume Basil was just asking for those options to be available easily from the Webmin TKLBAM interface rather than having to use the CLI.
Thx
Thanks, I was just looking in the manual pages. The address option is listed but the description is not as good as the FAQ description.
I need to update the manual pages
>> I assume Basil was just
>> I assume Basil was just asking for those options to be available easily from the Webmin TKLBAM interface rather than having to use the CLI.
Yes :)
An idea: backing up/restoring from the filesystem?
So maybe rather than trying to support every protocol under the sun, we could just add support in webmin-tklbam for backing up/restoring from the filesystem. Let users take care of the mounting part separately.
OTOH, if you can mount filesystems you are probably comfortable invoking tklbam's cli interface. Needs more thought.
Thanks to all of you
I've probably done a testimonial like this before, but I can't find it and from my perspective it's worth repeating in light of Liraz's expression of appreciation:
I hold TKL in very high esteem, in part because my reliance on their products, which are used daily in some way or another. For two weeks of each year I do a unit on collaborative writing and editing. For that I rely heavily on a VM running Turnkey's Wikipedia appliance. This year I may turn to the Dokuwiki, to keep myself familiar with different wikitext conventions. In any case, TurnKey Linux makes that unit possible. To use our Head of School's language, our IEPs are one our primary products: the quality of the IEPs in the eyes of the stakeholders can make or break us. We've dependend on TKLPatch and TKLBAM to run IEP-IPP as a virtual machine in VMware Server. Flawless implementation and outstanding services from TKLBAM. On bare metal, we run an Ampache server built with TKLPatch and installed from the ISO. That's increasingly becoming part of the technology we use in the classroom daily to serve our students' diverse learning needs. The Limesurvey VM produced with TKLPatch is used to get some quantitative data for our ed research and I hope will eventually be used for student evaluation of courses. I rely on TKL's Joomla appliance (in a VM) for prototyping sites for which I've been asked to be part of the design process. For the products alone, I'm tremendously thankful; I am dependent on them and see that dependence increasing (our VMware Server is being supplemented by an ESXi server in coming days).
But I am thankful too for the opportunity to contribute: at my level, it's been tough finding projects that can use my expertise - even when I offer to document. At TKL, I found an opportunity to contribute - it was a struggle and a reach and I've come out knowing far more than I expected. I'm more thankful for the warm welcome the students have received. I've been proud of their success, and that takes me to my next point.
The community here is for the most part outstanding. Having the Ubuntu Code of Conduct as a reminder of the culture we're aiming for is commendable. But the encouragement from others - JedMeister constantly reminding us of what we're capable of if we just reach; Liraz's honest challenges felt chilly at first, to me, but the students took the challenges and read them as very productive dares. They were spurred on by Liraz in every case, in all his ethical honesty. Liraz has gone above and beyond and offering suggestions on how to garner interest in our projects from local communities, and offering honest critiques of my write-ups. Alon never flagged in his support, always came to our aid when we found ourselves stuck in patch development (and in other areas). Alon's kindness and eagerness for us to succeed is an obvious reflection of his passion for TKL and its community to succeed.
So, yeah, thanks. Thanks for the curriculum material. Thanks for the opportunities for service learning hours (which the students would deserve if I pursued that). Thanks for the opportunity to contribute something meaningful and authentic - not only to TKL, but also to Ampache, Limesurvey, Elgg, and now IEP-IPP (over 50 d/ls now excluding direct downloads). And thanks for the appliances that serve me daily, either as instructional tools or administrative tools.
Thanks to you too Rik
AWESOME WORK
Is there any other tutorial other than the IRC chat training which was recorded a couple of months ago? I am really jazzed about your project and the different virtual machines which have been created. GREAT WORK...!
Do you mean other TKLPatch info/tutorial?
If so, there is some quite extensive TKLPatch documentation here in the Wiki. If that wasn't what you meant, please elaborate.
MySQL appliance
Sorry, I wasn't sure where to post this since it's an RC. I've been messing with the MySQL appliance, and it seems that the InnoDB storage engine, among others, is missing? Thinking something went wrong with the installer, I ran from the live cd and it is also missing from there. In fact comparing the 8.04 appliance with the new 11.0 RC, half of the storage engines are missing. Granted some were disabled in the 8.04 appliance.
Thanks!
Alon will look into this
Which engines are you missing?
Hi John,
Unless I misunderstand you, I'm unable to reproduce the issue you're reporting. What engines are you missing? This is what I get:
Missing Engine?
I'm using the LAMP Lucid RC and I seem to be missing an engine that Mahara wants. Following Alon's model, I dropped to MySql prompt and got roughly the same results, minus InnoDB. I'm new to MySql so this may be a misunderstanding. But since I'm unfamiliar with Postgres (I've gotten Mahara to work on LAPP RC), we'd much prefer to work with MySql. What do I do with a missing engine? Or, if it's the case, an apparently missing engine?
Got it - had to rename the log files
Followed the advice on another thread about this. Renamed the log files, and suddently Innodb showed up as an engine. Curious.
InnoDB woes...
InnoDB is an excellent transactional storage engine, and works great. But, I've hit issues in the past where the logfiles are considered corrupt, and therefore the engine will be disabled during MySQL startup.
I spent quite a bit of time reading over documentation, forum threads and experimenting - trying to figure out why it happens, but haven't been able to pin point the exact reason. I've tested on vanilla installations as well as populated databases. Anyway, just take care when renaming/deleting the log files so you don't loose any information.
Thanks Alon
I'm hitting the problem immediately after install of LAMP rc, so there's presumeably no data at risk. If I understand your post correctly, it's possible the corruption may happen later, once there is data at stake. With this in mind, is it your feeling we should base Mahara patch on LAPP stack instead? It'll take some research to learn how to create the db and user, set privs from a bash script, but we're up for that challenge if you think we're better off going in that direction.
InnoDB follow-up
A little late replying to the discussion... but I also confirmed some things. One, that the MySQL plugin for webmin totally hoses the my.cnf file. I did a fresh install and just hit save and restart and MySQL would no longer even start up. It just threw a bunch of errors. I restored the my.cnf file and it started right up.
Second, that a rename or delete of ib_logfile0 and ib_logfile1 restored InnoDB to my list of engines.
And third, the ib_logfiles seem to get corrupt during install. I setup a totally fresh VM and installed the MySQL appliance and right from the start these must be corrupted, because InnoDB did not show on my fresh install until I renamed/deleted those two logs and restarted MySQL.
Anyway, aside from this engine issue, I've been loving the new appliances. I've used this one, the core, and the LAMP. I've had no issues. Keep up the great work! :-)
Where are the download pages for testing?
I received your email announding the beta - and testers needed. I go to your blog link, but I can't find the actual Files to test. Is this all on Turnkey Hub? I am wanting to test in a VM envirionment but not on the cloud.. (I read in your intro that you have VM's built).
What is the schedule for full release?
thank you all for your great work!
Downloads of RCs
It's my experience and understanding that the downloads of the RCs are available as ISOs on the pages of the individual appliances. So, for example, LAMP appliance RC is available here: http://www.turnkeylinux.org/lamp and labeled "Download 11.0 RC". From there, install to fresh VM. Someone will always be around to address questions, I'm certain.
Rik
RC1 LAMP
Great job, love my new box and webshell, until now no server issues. Webmin is more enjoyable somehow and index page is perfect for backoffice admin implementation.
No issues with php or mysql until now also (50.000 records tables)
One thing, will Lenny builds come out before Part 2 and the new appliances?
PS - really... I'm olding back my self, because my wish is to migrate and put my new turnkey to intranet production... now :)
Potential Schedule for full 11 releases?
Hi,
I am wondering what the game plan is for full 11.0 Release packages? Do you anticipate many changes? Will the Package Sets be updated so that 11rc sets move into full Installs?
Lots to ask but all has to do w/ implementation plans.
Cool stuff!
The rails 3 console isn't
The rails 3 console isn't working in the new appliance. The readline.so doesn't seem to be compiled. It's an easy enough fix. Download the ruby source, run apt-get install libncurses5-dev libreadline5-dev (if they're not installed, they're commonly missing from what I could see), run ruby extconf.rb form <ruby_source>/ext/readline, and then run make and make install.
Of course, would be nice if it worked by default, you're bound to forget about readline.so not being there until you need the console, and then have to go searching for the solution again =)
Good catch! Fixed.
By installing libreadline5-dev (and its dependencies) the rails console now works - excellent catch! I've committed the fix and it will be included in all rails based appliances in the upcoming 11.0 release.
BTW, I never found the need to download/compile the ruby source after installing the required libraries.
Side notes:
Again, thanks for reporting the issue!
I actually didn't test to see
I actually didn't test to see if the console worked after installing libreadline5-dev, I just followed some guide I found online, which installs it on a standard Ubuntu 10.04 rather than a turnkey appliance. Either way, great that was an easy fix, and that it'll be included in the appliance :)
As for Rails 2 vs. Rails 3, I don't think there's a need for an appliance for both. Easier would be to keep the appliance on Rails 2, and possible add a note about upgrading to Rails 3. Upgrading is easy enough, just run the gem update command.
Looking forward to the release of the new appliances though, they're a bit easier to work with compared to the older ones, especially if you want to run rails 3 ;)
nameserver is another name for DNS server
So you can either set it as your local DNS server (if you have one), your router/gateway (assuming that forwards DNS requests) or set it directly to you external DNS provider (your ISP or a public one like OpenDNS or Google).
Firstly sounds like you're not using TKL
And although v11.x TKL is heavily based on Ubuntu 10.04 it has some differences. So if you were on TKL I'd suggest that you install the Webmin-BIND module but that won't work unless you're on TKL...
So I suggest you post on the Ubuntu forums...
Regardless Apache should still work regardless of that though. Usually it has a defailt "It works!" page, but it doesn't look like your install does. Try putting an html/php file called index (ie index.html or index.php) in your document root (/var/www by default).
Otherwise you could install TKL LAMP and then start a new thread to address your DNS woes (this is a pretty old blog post).
Pages