New release candidates for TurnKey Linux 11.0 (part 1)

The webmin module for TKLBAM is the most interesting new thing :) . Also using LVM by default is a good decision.

Nice work! we were expecting the post. Today I'm testing the Core Appliance, and for what I've seen it's behaving real nice! But I'm writing down my feedback and I'll post it soon. 

For me, the nicest features are the dialogs on first boot, the webmin module and LVM. Of course fixing the network bug and removing byobu is also very welcome!

Great achievement! 

Very nice work guys!

I particularly like the way that TKLBAM is so tightly integrated that it definately looks like a fundamental part of the appliance (as it indeed now is)!

The new versioning scheme makes lots of sense (although personally I didn't mind the date based system). The config questions running on first boot, makes install and config consistent regardless of the image format which is very nice!

All-in-all an awesome RC release!

I reckon thats a great idea. I imagine that the best place for specific resetting password info and other similar stuff would be on the Appliance Specific Wiki pages.

I think it would be great if all appliances had their own specific appliance wiki page (even if its only a blank placeholder page to start with) with a link from the (dev produced) appliance page. I guess it may be a little frustrating for some if they go to the Wiki and it's blank, but perhaps all the blank ones could include a basic template offering some suggestions of where to start looking for info and perhaps some links to relevant forums, wikis, base info, etc that are relevant to that appliance. Just adding my 2c in the mix :)

I remember reading it and agreeing wholeheartedly with many of your suggestions. I had intended to post a response but obviously got sidetracked...

I will try to remember to read through it properly again soon and post back over there (and bump the thread up at the same time).

In a way I think that's why using a Wiki (or similar CMS) is the go, it means that we don't rely on the Core devs to do all the work, we can do some of it too!

It'd be great if Alon and Liraz got some more time here on the forums but I fear with the current appliance rollout we probably won't see a lot of them for a little while (a few weeks at the least).

The website is overdue for another round of improvements. We're in complete agreement on that. How soon we get to it is mostly a matter of priorities (e.g., spend a few weeks refreshing the web site or work on 64-bit builds).

I suggested the same thing some time ago, you can see my last comment on this thread. There are some ideas for the webpage including the ones you're proposing here. I hope I'll receive some feedback from the guys, once they come alive again on the forums. And yes, it's a lot of work. But we can do it, it's a matter of stablish a standard "template" so we can contribute but keep consistency on the information. 

Now that all that stuff runs at first boot rather than install thats a good option IMO. Cool!

I'm a little late to the discussion, none-the-less...

I think TurnKey's mantra: what can be easy, should be easy. applies here. I'd like to add inithooks that allow the user to set the application passwords on first boot instead of having default passwords. This would alleviate the need for much documentation.

But, there is a catch - we still need to support deployment of appliances where the user does not have direct access to the console on firstboot, such as on Amazon EC2 and other hosted options. We can make the process seemless using the Hub and pre-seeding (but that hasn't been implement as of yet).

So, we would still need default passwords and have them documented some where. But this is still a step forward.

We could also add an option to the confconsole as Liraz mentioned allowing the user to reset passwords, but that means the inithooks need to be aware (or at least take into consideration) the state of when they are being run - as services might need to be stopped and/or restarted.

Congratulations on yet another milestone (they seem to come so rapidly - amazing given the human hours available). I'm digging in straight-away with dokuwiki, and I'll see where I'm taken from there. Looking forward to giving any that I can find a use for a go.

Great work,

Rik

---update---

Installed the Lucid RC of Dokuwiki into a VMware VM using Workstation. Wonderful stuff. Love the integration of TKLBAM and how it's worked into both confconsole and, more brilliantly, into Webmin. Love the opportunity to configure TKLBAM on first boot after installation  along with the root password.  Installing from the ISO to VMware was more painless than installing from repos onto a Lucid server on bare metal, which I have found to be nothing short of a pain in the neck (yeah, could be a problem between my brain and the keyboard - but the point stands that there's no such problems with TKL's appliance).

One thing I've been thinking about is providing easy instructions for changing the default passwords. I know for many of us, it's a few clicks away and possibly a search in Google. For some, however, that may not be the simple process it appears to be from our perspectives. I've thought about this particularly since I'm hosting appliances at our students' site and realize things may be smoother for the end user if we provided more than the default credentials: actual guides to changing and setting passwords so the appliances are more likely to be secure; even if those guides are just links to other documentation, it might secure one more user who may otherwise stay vulnerable. Of course, we haven't done this because of time constraints. Obviously it's more than likely that neither of you have the discretionary time for documenting something like what I'm describing here. Just some thinking I thought maybe I'd share.

We'll be releasing the final 11.0 (part I) soon once we squash all the outstanding bugs and issues, and create all the build targets. I also want to squeeze in setting of application passwords during firstboot (with preseeding support) if we have the time.

Thanks Alon. Have you refreshed the ISO download or will we just have to be patient until the final release?

This was a tough cookie! But it has now been implemented and works whether the installation is run via the boot menu option, or the confconsole when running in live demo mode.

The hardest part was interfacing with the user after the system has gone through the shutdown procedure (filesystems have been unmounted) and right before the actual shutdown. It gets even more tricky once the installation media has been 'ejected', which means we need to cache the relevant executables and libraries in memory, make the tty 'sane'.

Oh, and of course switch back to the correct VT console where the shutdown procedure is happening.

Anyway, the cdrom will be ejected if the installation completed successfully, and the user will be prompted to remove the disc (if any) and hit ENTER, or until the timeout is reached.

I think the official ubuntu instalation tells the user to remove the CD. That would be the way to go. Believe me people would find themselves in a loop if they finish the installation and then they are presented with two options: install or try live! 

Good advice from your son! That could be the message for the TKL Homer edition LOL. 

My son suggested that if Alon (and/or others) insist on having a 3rd option you could perhaps make it "Doh I forgot to take out the CD: open CD tray and reboot"! :p

But seriously; I initially missed the option, but I think you make a good case for removing it. And it certainly makes the options available more obvious. To make it really shine though I do think you want to auto eject CD after install and/or tell users to eject the CD.

Regarding the removal of the boot from hard-disk option. This was my idea. Alon protested and said he found it useful during development. I argued that we should optimize the user experience for the common user/newbie as he's the one that needs our help the most. A more experienced user will figure it out. IMHO having just these two options make it clearer that you have to make a choice between installation mode and demo mode. The third option was fake. It didn't even do anything useful until after installation. Also, I don't like how it creates two equal but different ways of booting into an appliance after installation (e.g., remove the CD or choose the third option in the bootloader). One way is easier to explain.

I admit I just went with my gut here. If other people would like to weigh in either way it would be useful to get more perspectives.

I solved the issue globally in di-live by redirecting stderr of all commands executed to the logfile (/var/log/di-live.log). The installer will still raise an exception for bad exit codes, but warnings will not be displayed. This also provides useful debugging information when needed (no need to watch warnings flying across the screen and hope that all is well.

Anyway, thanks for the push to fix this, it's long over due.

Great feedback Adrian!

I'm going to tackle the warnings during installation. From what I recall installing the MBR generates output which is displayed in the di-live console. Are there any other places during the install that generates warnings that should be fixed?

I only wish there were more hours in the day so we could be more active even in the middle of the deepest development cycles.

If anyone wants to jump in and send patches that add support for "manual" mode in a way that doesn't compromise too much on usability, that would be a good starting point.

When users are running TKL in their internal network (in a physical hardware or Citrix server ), it would be nice if the TKLBAM support target location like FTP Servers and WEBDAVs or even Rackspace Cloud file (since it is more cheaper that S3). In my humble openion , there should be a "drop down list" or "radio button" to select target location type :)

I stumbled upon Turnkey a few months ago but only downloaded the first Appliance yesterday.

I realize now that I should have done this sooner.

I like it very much and am looking forward to using the final 11.0 release.

Keep up the good work!

PS. Also thank you to Adrian & Basil for working real hard to provide additional Appliances.

Thanks, I was just looking in the manual pages.  The address option is listed but the description is not as good as the FAQ description.

>> I assume Basil was just asking for those options to be available easily from the Webmin TKLBAM interface rather than having to use the CLI.

Yes :)

You may already realise. but it is possible to use TKLBAM to backup to targets other than S3. You do still need to use the Hub, part of which involves signing up for an S3 account (and providing credit card details - although you never have to actually use it).

TKLBAM will automatically use S3, but if you use the (CLI only) --address option you can send your backups pretty much anywhere. From the TKLBAM FAQ:

But you can also backup to any storage target supported by TKLBAM's back-end Duplicity including the local filesystem, NFS, Rsync, SSH, FTP, WebDAV, Rackspace CloudFiles and even IMAP.

I assume Basil was just asking for those options to be available easily from the Webmin TKLBAM interface rather than having to use the CLI.

So maybe rather than trying to support every protocol under the sun, we could just add support in webmin-tklbam for backing up/restoring from the filesystem. Let users take care of the mounting part separately.

OTOH, if you can mount filesystems you are probably comfortable invoking tklbam's cli interface. Needs more thought.

First, excellent work with these appliances, they are fantastic.

I agree with Basil Kurian, it would be great to have other target options for TKLBAM.  I will be using these appliances at work and we already have an offsite backup solution in place.  However, the TKLBAM interface (GUI and CLI) are much nicer than our current backup software interface, so I'd much rather use that to send the backup to our SAN via SMB which is already set for nightly offsite backups.  

It would also be nice to have options like NFS and iSCSI or even over an SSH tunnel.

Thanks and keep up the good work!

I've probably done a testimonial like this before, but I can't find it and from my perspective it's worth repeating in light of Liraz's expression of appreciation:

I hold TKL in very high esteem, in part because my reliance on their products, which are used daily in some way or another. For two weeks of each year I do a unit on collaborative writing and editing. For that I rely heavily on a VM running Turnkey's Wikipedia appliance. This year I may turn to the Dokuwiki, to keep myself familiar with different wikitext conventions. In any case, TurnKey Linux makes that unit possible. To use our Head of School's language, our IEPs are one our primary products: the quality of the IEPs in the eyes of the stakeholders can make or break us. We've dependend on TKLPatch and TKLBAM to run IEP-IPP as a virtual machine in VMware Server. Flawless implementation and outstanding services from TKLBAM. On bare metal, we run an Ampache server built with TKLPatch and installed from the ISO. That's increasingly becoming part of the technology we use in the classroom daily to serve our students' diverse learning needs. The Limesurvey VM produced with TKLPatch is used to get some quantitative data for our ed research and I hope will eventually be used for student evaluation of courses. I rely on TKL's Joomla appliance (in a VM) for prototyping sites for which I've been asked to be part of the design process. For the products alone, I'm tremendously thankful; I am dependent on them and see that dependence increasing (our VMware Server is being supplemented by an ESXi server in coming days).

But I am thankful too for the opportunity to contribute: at my level, it's been tough finding projects that can use my expertise - even when I offer to document. At TKL, I found an opportunity to contribute - it was a struggle and a reach and I've come out knowing far more than I expected.  I'm more thankful for the warm welcome the students have received. I've been proud of their success, and that takes me to my next point.

The community here is for the most part outstanding. Having the Ubuntu Code of Conduct as a reminder of the culture we're aiming for is commendable. But the encouragement from others - JedMeister constantly reminding us of what we're capable of if we just reach; Liraz's honest challenges felt chilly at first, to me, but the students took the challenges and read them as very productive dares. They were spurred on by Liraz in every case, in all his ethical honesty. Liraz has gone above and beyond and offering suggestions on how to garner interest in our projects from local communities, and offering honest critiques of my write-ups. Alon never flagged in his support, always came to our aid when we found ourselves stuck in patch development (and in other areas). Alon's kindness and eagerness for us to succeed is an obvious reflection of his passion for TKL and its community to succeed.

So, yeah, thanks. Thanks for the curriculum material. Thanks for the opportunities for service learning hours (which the students would deserve if I pursued that). Thanks for the opportunity to contribute something meaningful and authentic - not only to TKL, but also to Ampache, Limesurvey, Elgg, and now IEP-IPP (over 50 d/ls now excluding direct downloads). And thanks for the appliances that serve me daily, either as instructional tools or administrative tools.

If so, there is some quite extensive TKLPatch documentation here in the Wiki. If that wasn't what you meant, please elaborate.

Is there any other tutorial other than the IRC chat training which was recorded a couple of months ago?  I am really jazzed about your project and the different virtual machines which have been created.  GREAT WORK...!

I'm hitting the problem immediately after install of LAMP rc, so there's presumeably no data at risk. If I understand your post correctly, it's possible the corruption may happen later, once there is data at stake. With this in mind, is it your feeling we should base Mahara patch on LAPP stack instead? It'll take some research to learn how to create the db and user, set privs from a bash script, but we're up for that challenge if you think we're better off going in that direction.

A little late replying to the discussion... but I also confirmed some things.  One, that the MySQL plugin for webmin totally hoses the my.cnf file.  I did a fresh install and just hit save and restart and MySQL would no longer even start up.  It just threw a bunch of errors.  I restored the my.cnf file and it started right up.

Second, that a rename or delete of ib_logfile0 and ib_logfile1 restored InnoDB to my list of engines.

And third, the ib_logfiles seem to get corrupt during install.  I setup a totally fresh VM and installed the MySQL appliance and right from the start these must be corrupted, because InnoDB did not show on my fresh install until I renamed/deleted those two logs and restarted MySQL.

Anyway, aside from this engine issue, I've been loving the new appliances.  I've used this one, the core, and the LAMP.  I've had no issues.  Keep up the great work!  :-)

InnoDB is an excellent transactional storage engine, and works great. But, I've hit issues in the past where the logfiles are considered corrupt, and therefore the engine will be disabled during MySQL startup.

I spent quite a bit of time reading over documentation, forum threads and experimenting - trying to figure out why it happens, but haven't been able to pin point the exact reason. I've tested on vanilla installations as well as populated databases. Anyway, just take care when renaming/deleting the log files so you don't loose any information.

Followed the advice on another thread about this. Renamed the log files, and suddently Innodb showed up as an engine. Curious.

I'm using the LAMP Lucid RC and I seem to be missing an engine that Mahara wants. Following Alon's model, I dropped to MySql prompt and got roughly the same results, minus InnoDB. I'm new to MySql so this may be a misunderstanding. But since I'm unfamiliar with Postgres (I've gotten Mahara to work on LAPP RC), we'd much prefer to work with MySql. What do I do with a missing engine? Or, if it's the case, an apparently missing engine?

Hi John,

Unless I misunderstand you, I'm unable to reproduce the issue you're reporting. What engines are you missing? This is what I get:

mysql> show engines;

+------------+---------+
| Engine     | Support |
+------------+---------+
| InnoDB     | YES     |
| MRG_MYISAM | YES     |
| BLACKHOLE  | YES     |
| CSV        | YES     |
| MEMORY     | YES     |
| FEDERATED  | NO      |
| ARCHIVE    | YES     |
| MyISAM     | DEFAULT |
+------------+---------+

Sorry, I wasn't sure where to post this since it's an RC.  I've been messing with the MySQL appliance, and it seems that the InnoDB storage engine, among others, is missing?  Thinking something went wrong with the installer, I ran from the live cd and it is also missing from there.  In fact comparing the 8.04 appliance with the new 11.0 RC, half of the storage engines are missing.  Granted some were disabled in the 8.04 appliance.

Thanks!

It's my experience and understanding that the downloads of the RCs are available as ISOs on the pages of the individual appliances. So, for example, LAMP appliance RC is available here: /lamp and labeled "Download 11.0 RC". From there, install to fresh VM. Someone will always be around to address questions, I'm certain.

Rik

I received your email announding the beta - and testers needed.  I go to your blog link, but I can't find the actual Files to test.  Is this all on Turnkey Hub?  I am wanting to test in a VM envirionment but not on the cloud.. (I read in your intro that you have VM's built).

What is the schedule for full release?

thank you all for your great work!

Great job, love my new box and webshell, until now no server issues. Webmin is more enjoyable somehow and index page is perfect for backoffice admin implementation.

No issues with php or mysql until now also (50.000 records tables)

One thing, will Lenny builds come out before Part 2 and the new appliances? 

PS - really... I'm olding back my self, because my wish is to migrate and put my new turnkey to intranet production... now :)

Hi,

I am wondering what the game plan is for full 11.0 Release packages?  Do you anticipate many changes?  Will the Package Sets be updated so that 11rc sets move into full Installs?

Lots to ask but all has to do w/ implementation plans.

Cool stuff!

I actually didn't test to see if the console worked after installing libreadline5-dev, I just followed some guide I found online, which installs it on a standard Ubuntu 10.04 rather than a turnkey appliance. Either way, great that was an easy fix, and that it'll be included in the appliance :)

As for Rails 2 vs. Rails 3, I don't think there's a need for an appliance for both. Easier would be to keep the appliance on Rails 2, and possible add a note about upgrading to Rails 3. Upgrading is easy enough, just run the gem update command.

Looking forward to the release of the new appliances though, they're a bit easier to work with compared to the older ones, especially if you want to run rails 3 ;)

By installing libreadline5-dev (and its dependencies) the rails console now works - excellent catch! I've committed the fix and it will be included in all rails based appliances in the upcoming 11.0 release.

BTW, I never found the need to download/compile the ruby source after installing the required libraries. 

Side notes:

• The 11.0 rails based appliances use ruby enterprise edition (for performance), instead of the stock ruby packages from upstream.
• You mentioned rails 3. The 11.0 appliances are based on the 2.x branch as a lot of the apps (e.g. redmine) aren't compatible with rails 3 yet. We might create a rails3 appliance along side rails2 if there is demand (and time).

Again, thanks for reporting the issue!

The rails 3 console isn't working in the new appliance. The readline.so doesn't seem to be compiled. It's an easy enough fix. Download the ruby source, run apt-get install libncurses5-dev libreadline5-dev (if they're not installed, they're commonly missing from what I could see), run ruby extconf.rb form <ruby_source>/ext/readline, and then run make and make install.

Of course, would be nice if it worked by default, you're bound to forget about readline.so not being there until you need the console, and then have to go searching for the solution again =)

So you can either set it as your local DNS server (if you have one), your router/gateway (assuming that forwards DNS requests) or set it directly to you external DNS provider (your ISP or a public one like OpenDNS or Google).

And although v11.x TKL is heavily based on Ubuntu 10.04 it has some differences. So if you were on TKL I'd suggest that you install the Webmin-BIND module but that won't work unless you're on TKL...

So I suggest you post on the Ubuntu forums...

Regardless Apache should still work regardless of that though. Usually it has a defailt "It works!" page, but it doesn't look like your install does. Try putting an html/php file called index (ie index.html or index.php) in your document root (/var/www by default).

Otherwise you could install TKL LAMP and then start a new thread to address your DNS woes (this is a pretty old blog post).