TurnKey Linux Virtual Appliance Library

Beta of TurnKey Core on Debian Lenny (+ Ubuntu vs Debian)

We've just uploaded to SourceForge our first ever Debian-based virtual appliance: a beta of TurnKey Core on the rock stable Lenny release.

DOWNLOAD BETA:

 

It has about the same features as the Ubuntu 10.04 based Core beta we released a couple of weeks ago with a few minor exceptions (e.g., grub instead of grub-pc, byobu not included).

Ubuntu-Debian chimeras considered harmful

Most users probably don't realize it but a handful of our current crop of "Ubuntu based" appliances are actually Ubuntu-Debian chimeras. The package management system (APT) is technically capable of mixing packages from different distributions. We took advantage of that to configure some TurnKey appliances to get security updates directly from Debian for certain packages which were not supported on Ubuntu.

Unfortunately it's a relatively complicated hack that relies on poorly documented, rarely used, and consequently buggy APT functionality.

It hasn't back-fired yet, but we'd rather not wait for that to happen.

TurnKey appliances are configured to auto-update security fixes by default so safety and robustness is a key concern. We don't want to risk breaking anything in the future. Better safe than sorry!

So from now on, no more chimeras. The upcoming Ubuntu Lucid based appliances will be 100% Ubuntu, even if that means some packages don't get security updates.

Are Debian based appliances worth the trouble?

This brings us to our dilemma. Guaranteed security updates for all packages are a big deal, at least for us. And only Debian provides that.

Which got us thinking. How much extra work would it take to also build a Debian-based TurnKey Core? And would the interest from the community justify the effort?

Bottom line: it was a bit harder than we anticipated but we made it happen and now we need the community's help in figuring out if it matters.

Though we haven't committed to it yet, we are seriously considering Debian-based builds of all TurnKey Linux appliances. But that depends on the feedback we get from you and the level of interest in this.

Frankly, we don't have the resources to thoroughly test both Debian and Ubuntu based builds of all TurnKey appliances.

That means to pull this off we'll need all the help we can get testing Betas, providing feedback on issues that come up, filing and triaging bug reports, etc.

If you care about TurnKey on Debian, we'll need you to step up to the plate and help us make it happen.

Ubuntu vs. Debian: the story so far...

So far TurnKey has been known as an Ubuntu based open source project so this move towards Debian may come as a surprise to some, but those of you who have been following closely know that Debian support has actually been in our sights since TurnKey's conception.

One of our first polls asked: would you prefer virtual appliances based on Ubuntu or Debian?

Results so far (based on 763 votes):

  • 62%: Ubuntu for both client and sever roles
  • 23%: Debian for server roles, Ubuntu for client roles
  • 15%: Debian for both client and server roles

Despite a clear preference for Ubuntu (which is better known due to its popularity on the desktop), a significant 38% still prefer Debian for server roles.

This resonates with us because when TurnKey was still on the drawing board a couple of years ago we debated Ubuntu vs. Debian extensively. In the end Ubuntu won by a slim margin but it was a tough call!

I'll talk a little bit about the thought process behind that because despite a couple of years going by the big picture hasn't really changed.

Back then we were mainly using Ubuntu on our desktops and Debian on our servers and frankly Debian seemed like a more natural choice for a server-oriented virtual appliance library.

The "main" problem with Ubuntu is that only a subset of packages in the "main" component are officially supported with security updates.

By contrast, Debian supported all 25,000 packages with carefully backported, well tested security fixes that could be safely applied to a production system. Debian is also rock solid in terms of stability, which is something you usually want in a server operating system, even when it comes at the expense of having the latest package versions.

On the other hand Ubuntu 8.04 (Hardy), a Long Term Support version had just been released and Debian Lenny was still a work in progress. Stability comes at a price, and it's one of the main reasons for Debian's notoriously slow release cycle.

But we didn't want to wait who knew how long or start a new project on an old distribution...

Plus, we shared Ubuntu's values regarding making open source accessible to everyone, not just savvy experts. That meant encouraging a community atmosphere in which everyone was welcome and treated with respect. Which is why we adopted the Ubuntu Code of Conduct. Unfortunately, the Debian community had a reputation for being more withdrawn and elitist.

Ubuntu's popular appeal was also a factor. Let's face it, today Ubuntu has far better name recognition than Debian, though I think that's mostly due to superior marketing and more effective leadership. Having a wealthy benevolent dictator that can bankroll the operation definitely has a few advantages (and disadvantages!).

But keep in mind that though Ubuntu and Debian do inevitably compete for users in some areas, they aren't really in direct opposition. In fact, every 6 months a new version of Ubuntu begins its life from a snapshot of the unstable Debian version in development.

Certainly Ubuntu deserves credit for pushing the envelope in areas like usability, but it's Debian's self-governed volunteer workforce of over 1600 developers that do much of the unglamorous heavy lifting.

Long story short, try the beta and tell us what you think. Obviously we have immense respect for both distributions and we'd like to hear your views on where we should take it from here.

You can get future posts delivered by email or good old-fashioned RSS.
TurnKey also has a presence on Google+, Twitter and Facebook.

Comments

IRC Chat log

I expressed my thoughts on #turnkey as follows:

<tinjaw> alonswartz, you say...
<tinjaw> "Frankly, we don't have the resources to thoroughly test both Debian and Ubuntu based builds of all TurnKey appliances."
<tinjaw> Fine. Spend the resources to build an automated test suite. Better yet...
<tinjaw> I am sure that the various applications that make up the appliances have their own test suites
<tinjaw> Turnkey would be wise to spend the time creating a meta test structure.
<tinjaw> Along with community based testing, that should produce a fairly solid product line.
<alonswartz> tinjaw: true
<alonswartz> up till now the appliances go through testing during development, and after building the ISO's, VM's, and EC2 builds
<alonswartz> but testing is more than making sure the software in the appliances work as expected
<alonswartz> we also need to test the integration of the parts
<tinjaw> that is where the meta testing would come into play
<alonswartz> I agree, if it could all be automated would be great - we could learn something from the ubuntu iso-testing team
<tinjaw> Now, I am in NO WAY committing, but ...
<tinjaw> I have felt that I *need* to learn (and use on my own projects) Selenium. So,
<tinjaw> Maybe working on a Selenium-driven test suite for TurnKey appliances ***might*** be something I would be willing to invest my time with.
<tinjaw> We might be able to even contribute some of that back upstream
<alonswartz> tinjaw: that would be great!
<alonswartz> if you could add a comment to the post on what we discussed, it might spark more discussion and input from the rest of the community

What does the rest of the community think about this?

Liraz Siri's picture

Need to test for the things we don't know can go wrong

I find test suites work best for catching regressions. For example, maybe we figure out how to setup a VM automatically and run a battery of simple tests against it (e.g., compare open ports to the ports that should be open, etc.) but that sort of approach can only get you so far.

For everything else you still usually need some kind of user testing, because you usually don't have the luxury of knowing in advance what is going to fail, otherwise you would have prevented that already.

Yes, you said explicitly what

Yes, you said explicitly what I was implying. Unit tests, black box testing, is best used for regression testing. It is also good for configuration and integration ("thread") testing.

Doesn't matter for me if the appliance works fine

Hi, I'm a Turnkeylinux fun, and I use the appliances a lot mostly as virtual appliances.

For me, it is the same if they use Debian or Ubuntu, since the most important part is the application that is mounted on the appliance, and if this works fine, it is not important for the end user like me what is behind... so I think it is a matter of preference of the developers (you!).

Thanks again for the great appliances.

Liraz Siri's picture

If you use TurnKey in production security updates matter

We understand that a big part of the advantage of using an appliance is that you don't have to care about the underlying operating system too much. So long as the application works.

But a big part of that advantage relies on the operating system level maintaining itself with security updates and such. Otherwise you do have to be aware of what components are in your appliance and make sure to update them yourself in case there are security issues.

That is, assuming you are using the appliance in circumstances where security matter and not just for internal testing and such.

Debian vs Ubuntu are closely related but they make different trade offs with regards to security.

Working fine and working quick

Installed quickly to a VMWare virtual machine on the first install. It sluggish with one security update, but recovers immediately to bring up confconsole.

I used ssh to apt-get myself a lamp stack up and running. I then installed subversion and tried it out. Worked perfectly. What struck me was how responsive it was. Tomorrow I'll be working on bugs for an application in the virtual machine, so I'll be able to say more. So far, absolutely nothing for me to be concerned with.

Update: Web app installed with subversion and configured without issues. Es ist guile!


debian/ubuntu & TurnKey Linux

I am trying debian now, my thoughts are as follows: Most of turnkey is driven by web applications (shell-in-a-box is what has drawn me to this distro time and time again...) - so the 'usability' of ubuntu almost never even comes into play. Additionally, the web applications that drive TurnKey should be tested with some kind of methodology, the OWASP guide to testing (http://www.owasp.org/images/5/56/OWASP_Testing_Guide_v3.pdf) would make a good starting point... ;-) However, I also agree that security testing is but only one kind of testing that should be done. Testing is a very important part of quality, and it is that quality that ubuntu saw in debian that resulted in the ubuntu team choosing debian as its foundation - maybe there is an important message in that choice. In my humble opinion; TurnKey would be also wise to used debian as the foundation and build their differentiation from that base for all the reasons already stated. But also so you can focus on making TurnKey better, and focusing the already limited resources on that differentiation.
Liraz Siri's picture

Ubuntu usability advantages more prominent on the desktop

Not too long in the future we're hoping to expand TurnKey into client-side applications as well, starting with a client-side analog of TurnKey Core (I.e., "TurnKey Core Client").

Client-side appliance-type applications would include any role-specific application that needs a front-end. Off the top of my head that would be stuff like media centers, kiosks, multi-media editing, thin clients, gaming emulators, and desktop virtualization platforms (e.g., VirtualBox).

For these sorts of applications, Ubuntu's usability enhancements might be more of an advantage.

Jeremy's picture

Ooh that sounds exciting!

I think you're right that Ubuntu may well be a better fit for these style of appliance. Although could there be room for Deian there to? Perhaps there may be some specialised client roles that are better fulfilled by a Debian base? Maybe security is particularly important for a specific client role? As long as they have inter-appliance continuity (by way of Core theme & apps) I think a blend of Ubuntu & Debian appliances may still work. It should also be relatively easy to achieve that continuity, as Debian and Ubuntu are so close. Perhaps with these client appliances, it be better to just select some to have a Debian twin? Rather replicating the whole range (I think the server appliances are somewhat different).

Off on a tangent; Multiseat is something I have been looking at over the last few months. I'd be great to have "TKL Multiseat Client"!

Perhaps stretching the vision further than you've been thinking; but I reckon a range of TKL 'live tool' appliances would fit nicely alongside the existing range. This too would be consistent with the TKL aim of bringing the best of open source to the public. TKL cleanup, backup,  recovery, HDD tools, offline antivirus/malware scanning (aimed at use with Win &/or Linux desktops) could all be great appliances.

Another potential plus for using Ubuntu as a base for TKL Clients is GRUB2. By leveraging GRUB2's ability to boot from .iso, a generic TKL 'live tool' appliance could be setup as an 'installed' (but not really) recovery/troubleshooting/live boot option for the "Client" appliances? This live environment could have packages updated leveraging TKLPatch, and upgraded to the latest release simply by downloading the new .iso (and updating GRUB2 menu - which ideally would all be handled by a script).

Sorry, waffling! :)

Liraz Siri's picture

Lots of good ideas!

Multiseat: is a concept I had never considered before. Very interesting but I imagine you would need special hardware to get it to work?

Live tools: Looking at the adhoc todo list in my notes I see that we have had some of the "live tools" ideas (e.g., "system rescue" type appliance) in there for a while now though I admit I hadn't taken the concept as far as you have. The first step in actually implementing this stuff would be to get a solid Core Client with a front-end first. It will then be easier for us (with the help of the community of course) to play with these ideas. Sort of "throw stuff out on the wall and see what sticks" type approach.

Meta live tool?: I admit I don't feel I've fully wrapped my mind yet around your ideas on a live appliance that can run other live appliances. Do you mean that in this case each appliance would be a sort of modular package in a system that can boot any of them? Then you roll the ISOs into a mega all-in-one live tool appliance?

Jeremy's picture

A little more explanation of my ramblings :)

Multiseat - no extra hardware required beyond an additional kvm set (keyboard/video(monitor)/mouse) for each extra user (USB keyboard/mouse) and of course a video outlet per user on the PC. Most video cards are dual head these days, even entry level ones, so an average PC with a vid card will support 2 independant users without any additional outlay (other than kvm). Apparently its quite possible to have up to 10 concurrent users all using the same hardware. I think this could be a niche open source win with relevance for education, public access, kiosks and even family homes if an easy to setup, preconfigured distro was availble (read TKL) to install straight to hardware.

Live Tools - sounds like we're on the same page there then!

Meta Live Tool - That wasn't actually what I was thinking but that's probably an even better idea! AFAIK it is possible to have a CD boot GRUB2, then each Live tool could be a separate (.iso) entity. If GRUB will do that (which I've been lead to believe it will) then that approach could also be used to create TKL Meta "Demo" CDs with multiple versions of TKL. This could help tech savy/IT pros 'sell' TKL to potential clients and the general public. The beauty of having each install as an .iso is that they are each complete and independant, thus avoiding many of the potential problems that can occur with most All-In-One type solutions.

User Friendly Live Recovery 'partition' - was more what I had in mind. To be specific I was thinking a "Windows user friendly" Linux based recovery 'partition'. A value added feature that could be included with TKL Client OSs and/or possibly 'installed' (GRUB2 would need to be installed but the Live Recovery OS would simply reside as an .iso) alongside another OS. Stretching this idea even further, the Australian Federal Police last year made a (very carefully worded) statement urging all users of online banking to only do online banking using a Live environment (rather than an installed OS) to mitigate security concerns. Perhaps a TKL "Secure Online" Live Client that could run from CD, USB or be 'installed'?

Liraz Siri's picture

Now I get it

multiseat sounds very neat, if you can get it working. For some reason I never considered connecting multiple keyboard / mice via USB. It's exactly the kind of application that would be a good fit for TurnKey - easy to use once it's up but needing expert help to get there. I wonder if there's an easy way to detect extra sets of input devices and start X sessions that are associated to them only. If I'm not mistaken usually any input device works with everything. When the time is right, we can start working out the details in a new thread, hopefully with the help of someone who's done this before and has compatible hardware!

Recovery partitions - now I get it. Brilliant idea. Copying the ISO to the hard-drive and booting it from GRUB saves you the trouble of needing a CD/USB. But, it's less secure. An attacker that has access to the Windows end of the system will still be able to tamper with the ISO, or anything else on the hard disk for that matter (e.g., grub). For this to work you'll need to prevent the untrusted machine from accessing everything (e.g., running it under virtualization). A nice solution could be a  TKL meta appliance that's optimized for running desktop type operating systems (rather than just head-less) servers.

Two Other Concerns - My Ignorance of Debian is a given here.

I have two related concerns, both prefigured in an earlier post. I don't know the debian community, but I understand the reputation. And I know the Ubuntu community's reputation. And I can attest: when I've support, it was always ultimately a Ubunu person helping me or a group committed to Ubuntu. And in most every case the support was not only solid and methodically sound - it was also truly "supportive" and encouraging.

My second concern also comes out of an ignorance about Debian. I know Ubuntu is concerned about accessibility; I'm aware the accessibility team feels Lucid is no where near where it should be. But I know the commitment and appropriate priority is given to those that benefit from alternative input and output devices. I have no understanding of Debian's commitment to accessibility; but if it's waning, inconsistent, or outright oblivious, I would prefer my appliances to work on the platform that gives due diligence to accessibility.


Debian Accessibility

I personally can not attest to Debian Accessibility what I can tell you is that the only full-time Debian developer I have ever met is blind. I can also tell you that at my university, in my masters program, we have a blind student whom also chooses to run debian. Additionally, since 2003 Debian has a special internal project named Debian Accessibility Project (http://debian.org/devel/debian-accessibility/index.en.html). A mailinglist (http://lists.debian.org/debian-accessibility/) with over 300 users is used to coordinate activities. There is also a list of software (http://debian.org/devel/debian-accessibility/software.en.html) that is relevant to accessibility.
Liraz Siri's picture

Debian does good work, it just doesn't market it as well

From my forays into Debian, I've often discovered there is a lot of good work being done quietly in the background that never gets the attention it deserves. Having dedicated PR/marketing people to champion the work and evangelize it to others can be a big help, but that's not the kind of work volunteers usually sign up for...

Also, I may be showing my ignorance here but I've always found it a bit awe inspiring that the blind could overcome the inherent challenges in using a computer, which for non-blind users is mostly a visual experience.

And to not only become a proficient computer user but actually a developer, I think that's just super-neat.

I imagine this is where open source can really shine. Most open source software starts out it's life in an attempt to scratch your own itch. In the proprietary world if it doesn't look like there is enough money to be made, it won't get done (unless mandated by law).

But with open source you can crack open the source code and fix it yourself, with the help of other people with similar needs and interests.

Liraz Siri's picture

Ubuntu Code of Conduct is a marvelous social innovation

What I love most about Ubuntu is that it's not just "Linux for human beings" in the technical sense and I think the seed of that positive social dynamic was the UCoC (Ubuntu Code of Conduct), which we adopted also for TurnKey.

The UCoC didn't come out of a vacuum. For a while Debian was really bad in that regard with the loudest, meanest members of the community poisoning the atmosphere. Free speech was overriding all other considerations. Nobody was ever punished for being particularly nasty. IMHO, it got to the point where it was really damaging the community experience and off-putting to newcomers.

When Ubuntu started, the leadership said - enough with that. We'll hold people up to higher standards. It seems to have worked!

Jeremy's picture

As you've stated Liraz

There are positives and negatives for both distros. So I think the idea of having Debian and Ubuntu appliances concurrently and allowing community members to choose is a good option.

Having said that: Whilst I'm aware of many of the valid points you have raised above, I'm not sure whether there is or will be enough need or desire for the option of Debian vs Ubuntu for appliances that are almost identical in most ways. I guess that's part of the idea of this beta release? - To test that theory.

Here's an idea: What about doing Debian appliances as x86_64 only and Ubuntu appliances as x86? Its still probably not going to work for everyone but may provide enough differentiation to make it worth the effort? As you've mentioned you ultimately intend 64 bit support at some point, imagine the support load when that happens! Assuming you replicate all appliances as Debian & Ubuntu / 32 & 64 bit and increase the current appliance list to include additonal ones (say 10+ for arguments sake). That will require support for 200+ appliances (double that if you differentiate .iso & VM formats) which actually only fulfill 50+ appliance roles. Sounds like huge additional effort for minimal extra value!

Sorry to sound so down on an idea that I was so supportive of not that long ago! Don't get me wrong, I'm certainly not anti-Debian!

One other consideration: Whilst Squeeze isn't about to ship any day soon (and as you suggest Liraz, Debian are notorious for pushing back release dates, missing deadlines and ultimately just releasing "when ready") it is edging ever closer to release. I'd hate to see you guys put a huge amount of effort into releasing a new raft of Debian Lenny based appliances only to have Debian release Squeeze as 'stable' a few months later. Have you considered releasing the Debian appliance range based on Squeeze and retaining the 'beta' tag until it becomes 'stable'? (I'm guessing not as stability and consistency seem to be things you guys favour). I guess the irony of going that way is that once Squeeze is 'stable' it will actually have newer packages than those of the Ubuntu 10.04 based appliances - at least until 12.04 LTS.

Liraz Siri's picture

Thoughts

64-bit support and our overhead: We don't yet know how 64-bit support will effect our workload. We're hoping it will mostly increase the aspects of the development process that can be batch automated. The real bottleneck is human labor, not computer resources. If that holds then there won't be a good reason to only support 64-bit on one platform.

Number of images: 50 appliances in 32-bit, 64-bit, Debian and Ubuntu versions. Then you have ISO, VM format, EC2 AMI, UEC, and Xen builds. That's about 1000 images! So yes we will soon be getting into crazy territory with regards to the number of images we are going to be pushing out. But we'll also be expanding how far we leverage cloud computing and automation in the build process so it shouldn't really make that much of a difference if we push out 100 or 1000 images.

Debian Lenny vs Squeeze: you make a good point with regards to the upcoming Squeeze release, but we're not inclined to go with Squeeze while it's unstable. There are still many release critical bugs I wouldn't want to expose users to. So we're going with Lenny for now but how far we go down that road depends on how much labor will go into "porting" over the appliances. Here we believe the main bottleneck will be testing, hence our reliance on the community to help pull through. Yes, the beta is an experiment. If it proves to be more than we can handle we may not release Debian based appliances for everything. Maybe only those applications that don't give us too much trouble.

Note that it doesn't look like squeeze is going to be released any time soon. According to the latest news from the release team they haven't even frozen the distribution yet. If I had to guess I'd say squeeze was at least 6 months away, maybe longer. So in the meantime, it would be nice to attract people from the Debian camp to TurnKey and help out with the first batch of Debian based appliances. I don't know if it will work but it would be nice to get the foot in the door with Lenny so by the time squeeze comes out we've expanded our community a bit with more people from the Debian community and make TurnKey Debian-based appliances a first class citizen.

Jeremy's picture

Debian Squeeze Frozen!

I just discovered that Debian Squeeze was frozen a few days ago. See the full article here

In summary it will run the Linux 2.6.32 kernel and include (in its repos) Apache 2.2.16, PHP 5.3.2, MySQL 5.1.48, PostgreSQL 8.4.4 and Samba 3.4. It will also have interpreters and compilers for all common languages such as Python 2.6 and 3.1, Perl 5.10, GHC 6.12 and GCC 4.4.

I wonder how long it will be frozen for? Who knows? Some have reported that testing was frozen for 6 months prior to Lenny being released as stable! Hopefully it won't be that long!

Beta Feedback

I have run both the Ubuntu & the Debian Betas now. (On a 5 yearl old macbook - but this is another long story...) and without a question the Ubuntu distro seems more 'mature'; however, I like my machines lean and Debian weighs in 44mb less. However, 44mb isn't much diskspace in todays world - but I like to be able to keep my whole OS and userspace on my USB keys, as well as in the cloud, so every MB counts. The other thing that strikes me is how many fewer resources are used by debian compared to ubuntu. On startup debaian had 22 processes and ubuntu has 102!!! Debian is much, much more responsive; ubuntu seems to be 'laggy' by comparison - although it is not slow. My mac runs quite a bit hotter under ubuntu that it did under debian. With debian the security updates installed without flaw, and ubuntu failed to set up networking out of the box. I am seriously not a debian fan boy (nor an ubuntu fan boy) I had no opinion one way or another, but if I did have an opinion before my test, to be honest I was leaning toward ubuntu. But my testing is changing my mind - this is just my own objective experience on my old and ageing hardware. (my local hardware runs fedora) However, I have long been looking to move my entire platform to USB & cloud... and I so far I think that TurnKey is the most promising of achieving that... I guess my opinion is that TurnKey is really starting to be a distribution of its own, in its own right - and I appreciate that this is a very important decision as a small business with limited resources. And although I really have been having a much better experience with Lenny, I would still tend to lean toward Lynx from a purely business based perspective. Shuttleworth is not done dumping money into linux and Ubuntu is already based on Debian - so you inherit all the work of two distros. But it would be really, really sweet if you could deliver a TurnKey Core based on Ubuntu minimal (13mb) (https://help.ubuntu.com/community/Installation/MinimalCD) that has a much reduced foot print and resource requirement. My security thoughts run slightly different, when I think of my images in the cloud - all the security people I know are much more impressed with debian for a whole host of reasons I dare not elaborate here as this post is getting really long, but suffice to say much of what goes into ubuntu is eye candy that we are not exercising in TurnKey... That is my 2 cents, I hope it is useful feedback - unfortunately I spent 25 years consulting so I can hold two opposing views at the same time in my head. ;-)
Liraz Siri's picture

Ubuntu vs Debian Betas, Minimal CD and security

Why TurnKey Core is 44% larger on Lucid: The increased size of the TurnKey Core beta on Lucid is something we're looking into and may soon be able to improve on but the main problem seems to be that the new version of Ubuntu simply specifies a more extensive set of dependencies, even when all we want is the same functionality that takes 44% less space in Debian Lenny and in Hardy, the previous version of Ubuntu.

This disappoints me because it should have been possible to make many of these "dependencies" (e.g., plymouth) optional (I.e., "recommends"), rather than mandatory. The proof is in the pudding: we have no intention of using plymouth for server appliances in TurnKey yet we're still forced to pull it in to satisfy dependencies. Unless we figure a way around this every Ubuntu based TurnKey appliance will include packages such as plymouth which we neither need nor use.

This is something that I doubt could happen in a Debian release for a number of reasons but mainly because Ubuntu's organizational structure allows for top-to-bottom mandates, even when they are very unpopular. There are pros and cons either way.

Responsiveness: the perceived speed difference between the Betas of Core on Lucid and Lenny may be primarily due to byobu, which is absent on the Lenny beta. This is a temporary difference. Based on the feedback we've received on performance issues and problems with nesting byobu will not be the default in future versions of TurnKey on Ubuntu either.

Update: there also seems to be a known issue with the framebuffer.

Minimal type CD not a viable base for a production system: Ubuntu's minimal CDs are pure network installers. If you dissect what's in there you'll discover that unlike a typical Live CD there is no root filesystem, just an initramfs. In the initramfs is a busybox based Linux environment just powerful enough to run debian-installer which then fetches everything else from the network.

It's utterly impractical to build TurnKey appliances using that as a basis. Keep in mind that TurnKey Core it itself built on TurnKey bootstrap which is already the smallest "legal" Ubuntu system that can boot and install new packages using apt-get.

Jeremy's picture

TKL Lucid Core larger

So Canonical have also increased 'dependancies' for apps? I was aware that as of 10.04 they have made install 'recommends' the default behaviour when installing. But to also increase dependancies seems like a strange move to me. I can see that for many "average Joe" end users it may be quite helpful to install 'recommends' by default (as long as how to avoid this behaviour is well documented and fairly straight forward - which it is). But making additional 'dependancies' (that the application can function without - ie aren't really dependancies) as well just seems pure crazy to me. What is the point of that? Bloat for bloats sake??

Liraz Siri's picture

IMHO, some of these dependencies qualify as bugs

You're right. I'm not at all happy about all of these additional unnecessary dependencies. I'm guessing maybe now that recommended packages are installed by default the developers have become less sensitive to the distinction between recommended packages and required dependencies because out of the box Ubuntu just installs both anyway...

But in my book they're bugs and I've even talked to Alon about maybe reporting them on Launchpad. Not that it would do much good for the current release.

Right now, we're looking for workarounds that don't require us to fork too many packages, or break the package management system (too much).

Jeremy's picture

Yes bugs indeed!

I think reporting them is a good idea. Not that it probably makes a lot of difference, but I'd be happy to say that "this bug affects me too" if you let me know the bug numbers. I'm sure many other TKL users would be happy to do likewise.

As you say though it doesn't help us right now. Do you not think that some of the package maintainers/devs that it affects might be willing (perhaps even keen) to fix these issues prior to 10.04.1?

I don't know a lot about packaging but shouldn't it be fairly easy for them to fix? Sounds like a bit of a mess really, and a huge PITA for you guys. I guess its potentially fairly easy to fix each package, but once you fork it then no more updates eh? (unless you repackage each one - that sounds painful). Good luck with it. I for one would be interested to hear how you go, and what choices you make and why. Perhaps a blog post?

Liraz Siri's picture

That's about right - huge PITA

Reporting the bugs is the easy part. The hard part is convincing the Ubuntu developers to do something about it with everything else they have to do. All that back and forth communication can be very labor intensive and distracting.

If we're lucky they'll fix it before 10.04.1, but there's a real risk they won't fix it until the next LTS, maybe not even then.

FYI: tmux is much faster than byobu... ;-)

tmux offers several additional advantages over screen (byobu):
- a clearly-defined client-server model: windows are independent entities which may be attached simultaneously to multiple sessions and viewed from multiple clients (terminals), as well as moved freely between sessions within the same tmux server;
- a consistent, well-documented command interface, with the same syntax whether used interactively, as a key binding, or from the shell;
- easily scriptable from the shell;
- multiple paste buffers;
- choice of vi or emacs key layouts;
- an option to limit the window size;
- a more usable status line syntax, with the ability to display the first line of output of a specific command;
- a cleaner, modern, easily extended, BSD-licensed codebase.
Liraz Siri's picture

tmux is nice. Thanks for the reference!

I hadn't tried tmux before. Thanks for recommending it. In my defense Lucid is the first version of Ubuntu that tmux is available for via apt. So far I've just played with it a bit but my first impression is that it looks considerably more flexible than screen and with saner default key bindings (muscle memory be damned).

I don't know about making it the default, but there's a good chance I'll be adding tmux to my personal toolset.

tmux.conf

This guy configured tmux to behave exactly like screen - his config is here:

http://github.com/harajune/dotfiles/blob/master/.tmux.conf

I'm both an ubuntu and debian developer

I'm both an ubuntu and debian developer, and I have to tell there's no   big difference from a packaging standpoint, as you might be aware. I believe you've chosen the right distro for server as well, because canonical is making a huge breakdown on the server market, so even myself switched my servers from debian to ubuntu, i believe influenced the most by several topics which might have you to consider far beyond the "popularity" stuff:

a) company-backed security updates
b) long-term support for LTS releases
c) certified and trained support personal is an option
d) enterprise support, such as cloud and remote management.

That would basically summarize the ubuntu pluses against debian for the server market, so my personal point of view, is ubuntu for the server and ubuntu for the desktop, that's it.

Liraz Siri's picture

Canonical's support for Ubuntu is a big advantage

We're aware of the advantages of Canonical's support for Ubuntu but it's still helpful to hear that this is an important advantage from our users perspective.

BTW, I'm surprised that as a Debian developer you don't mind taking a clear stand in favor of Ubuntu over Debian. I would have thought that might antagonize part of the Debian community...

I don't believe any debian

I don't believe any debian user/developer will feel bad about what I've said, especially if you point out that debian is ubuntu's upstream, which might not have been better choice ever, that debian is far from dead, and would have probably became the market leader if company-backed.

But I do like things as they're. It lets ubuntu focus on its specific goals (such as user experience, social desktop, restricted drivers, cloud services, etc.), while having half the work already done. There's no ubuntu without debian, and in the near future i believe there's no debian without debian, so you might start thinking on both, as the greatest alliance in the free software world, that at least I've seen.

When you consider that, and you're probably more aware than I about your audience, is that you're providing a solution for many users network infrastructure concerns. To be honest, even myself, being a certified linux and unix network administration, have gotten out of trouble by using your appliances from time to time.

But in the end of the world, they're both linux, and as long as it increases the linux market share, I'm ok with anything that runs the with little penguin's tongue out ;)

So, the way I see your question at a glance is like this: If you have the personal to develop appliances based on both distributions that's definitely great, if you don't (and iirc a chat with swartz that's the case), then you have to evaluate carefully where are your users, and why they're your users, I believe you're providing a pretty small but increasingly popular alternative to the virtual appliance marketplace, and I'm strong believer that a huge part of that popularity is about you making the choice to ubuntu.

Another topic to consider is, like I said before, you're in the server business, eventually those users, - and maybe that's a broad enterprise PoV, can't help it, that's me - will want/have to escalate, and there's where ubuntu shines.

You can do almost everything with debian, but it happens that with ubuntu canonical has made a lot for you already, you have cloud and virtualization support, remote management, network storage and synchronization, a development and support platform and personal, certification and training courses, etc.

I'm just exposing my points, I love both distros equally, but I believe that for your business right now, ubuntu is a better choice.

You won me with Debian

I have seen and tried a few of turnkey appliances, but never used them in production.  I run KVM and OpenVZ servers via Proxmox VE.  I love Debian on the server for only running what you really need and the security.  When trying to use resources efficiently, Debian really shines.  I like the idea of appliances, but I still build my own with Debian.  It takes a little more time, but I get what I want.

I love Ubuntu.  I have it on my desktop.  However, all of the snazzy features come at a performance cost. My notebook runs Debian.  The old Debian desktop isn't as snazzy, but I believe the performance is better.  I would think that for even specialized desktops, performance would be something desired.

Just my two cents.  I admit, I know little about what it takes to build appliances.

Thanks for listening.

Patrick Binder's picture

Where, bomgar, logmein, & webex have the market we should create

I have looked and looked for free open source solutions to the common problem of needing remote access software that clients can dl and it unistalls when it it done or one that has jump client that install on every computer in the network so that administrators can manage the network. Every single one cost thousands per user license. I think that it would be easy enough to develop our own live bootable OS that allows remote access into, WIN, MAC, Linux, and smartphones of all flavors that runs a LAMP stack for management. and I believe strongly that the turnkey core has the necessary starting point to develop it. I would love to have some people help me and put our heads together. if nothing else it will tear down the cost of the other companies software to a much lower cost. I think that if we sucessfully develop a OS/VM solution that hundreds of small and starting out companies would benefit greatly from such a software leap. Now obviously we will have to have mutliple platform gurus working on this project to develop a platform that supports all the above. SO please can we create a development site to start designing this incredible software. I would like to see it have the functionality of all 3 software titles. bomgar, logmein, and webex combined. so lets start dicussing features and concepts.


Jeremy's picture

In many respects I think you are right

but one thing many of these services offer which is not so easy to replicate in a free model is the unattended NAT/firewall tunneling they provide for non-tech users - and lazy ppl like me :). These solutions rely on a central man-in-the-middle server which connects the remote user to the system behind the NAT/firewall.

Personally I think this sort of big idea is admirable but is probably outside the scope of TKL.

OTOH there is probably freely available cross platform open source software that already provides most of the functionality you are looking for. Then perhaps a (yet to be designed) TKL gateway/firewall/VPN appliance that levarges DynDNS (or similar) to provide that same level of userfriendly NAT/Firewall tunneling. This could allow relativly non-tech users to set something similar up without the need to provide the third party man-in-the-middle server.

Liraz Siri's picture

If it's useful enough we might run a NAT negotiating server

Personally I would just love to see a TurnKey open source alternative to all the convenient to use yet distressingly proprietary remote access programs.

It's not just an issue of price, or even liberty. It's about security. An especially sensitive subject when you're dealing with NAT-traversing remote access. Proprietary, close-source software the community can't examine for backdoors doesn't sit well with me.

So if the benefits are strong enough I wouldn't rule out TurnKey running the required infrastructure. That's sort of what we're doing with the Hub...

Patrick Binder's picture

Well if we can compile the already existing CPOSS

Well if we can compile the already existing freely available cross Platform Open Source Software together with already existing gateway,firewall,VPN software into one single appliance then have it set to defualt boot to runlevel 2 so that it is manageable through webmin primarily as all tkl is and design the webmin to manage all of the above functions. Is that something that will be relatively medium to compile?


Jeremy's picture

Sounds like a good start

There is already an OpenVPN TKLPatch here that I imagine would be extremely useful (means you don't have to worry about that aspect). I haven't checked it out closely but perhaps thats getting close to what you're thinking?

As for other software you may want to include I'm not really sure. I'm happy to assist a bit and throw my 2c in but ATM I'm pretty snowed under so I probably won't be able to put in a lot of time (above and beyond my usual busy life, I'm also working on some TKL ideas).

The above patch apparently includes a Webmin OpenVPN module and the rest of the TKL hosted Webmin modules you can find here. (they should install into a TKL appliance with apt-get). If that still doesn't meet your needs then perhaps have a look on the Webmin site to see if they have a plugin (it'l be .tar.gz but we can worry about that later).

If your serious about taking this idea further then start another forum topic (I'd suggest in the General area).

Patrick Binder's picture

created a general topic on it

The general topic can be found here.


It it ain't broke, don't fix.

I just composed a whole post and somehow hit the back button on my browser and lost it, so I'm going to keep this attempt short..  A few thoughts:

  1. If the chimera hadn't caused a problem yet, why not stick with it and get the best of both worlds?  You know a ticking time bomb will go off because we've seen it happen everytime (except on 24 where they always stop it with 1 second left,) but is an undocumented 'relatively complicated hack' enough of a reason to lose the best of both worlds?
  2. In choosing a future path (core) for Turnkey, I would urge you to choose one based not on the marketing capability of an organization, but based on continuing to offer simple yet powerful appliances.  I would urge you to choose ONE path, don't choose the Debian and Ubuntu at the same time path.  That would, in essence, be cutting your resources in half.  No one wins in that situation.
  3. Reading about a possible thin client is great news.  Without getting into details, I have developers that will be starting on a thin client OS for VERY VERY low cost (almost disposible) hardware for client access to a new cloud based service we will be offering which will be developed using Turnkey appliances and the Turnkey Hub.  A Turnkey Thin Client would be a perfect match to go with the development work being done on the appliances.  I doubt there is, but is there a time frame for such a client appliance?

Thanks and sorry for rambling, but rest assured, this post is about 10% of my original one, so at least you don't have to read through that whole thing :)

Marc

Jeremy's picture

Off topic...

If you use Firefox, have a look at Lazarus Form Recovery. It has saved me a number of times. My only frustration is when I forget to install it on a new PC and loose a post or email (when using webmail) etc.

re: Off topic...

Thanks for the suggestion, I'll install that.  I use the Mozilla Add-On Collector (https://addons.mozilla.org/en-US/firefox/pages/collector) to maintain a number of collections to make it easy remember to install all the extensions I want when setting up Firefox.  And then there is FavBackup (http://www.favbrowser.com/backup/) which I've used to create a # of standard profiles that have all the extensions and scripts I use.  I just keep that on a USB key as well as a zip in my GDocs account and just to be triple sure, in my Dropbox account also.

It's makes setting up Firefox for both myself and clients a snap:)

Thanks for the advice.

Marc


encoding problem on Turnkey Lenny

Hello,

I'd like to +1 this issue regarding default encoding for the Debian Lenny Turnkey beta image :

https://bugs.launchpad.net/turnkeylinux/+bug/496746

Basically, it would be nice to have the default LANGuage settings as en_US UTF8 for instance... that is, something in UTF8 at least. This has gotten me issues in Postgresql creating non-utf8 databases because default language is "C" and openstreetmap requiring UTF8 databases only.

Also dpkg-reconfigure breaks abnormally, as explained in the issue.

Take care,

Jonathan-David Schröder

Squeeze just set as stable

Looks like Squeeze is now the stable release, any possibility of this release being updated to Squeeze?  If not, what is involved in making one of these core releases?

I prefer Debian on my servers and I love how easy it is to experiment by just taking this Debian core beta.


Liraz Siri's picture

Squeeze is in our sights

The latest Debian release has not escape our attention. Supporting Debian has been in our sights for a while now and I hope we get to it real soon. I can't give a schedule though. We believe it's done when it's done is the best fit for an open source project. If you want to try your hand at upgrading the previous Debian based core beta yourself, you are more than welcome to give it a shot. TKLPatch is your friend.

I will give it a shot but I

I will give it a shot but I am a novice with linux and am even newer to TKLPatch.  When you guys get around to doing this will you just start with the current beta and run a dist upgrade or will you start with a clean install ISO?


Jeremy's picture

I would suggest a clean install from ISO

Then using a bit of google-fu find yourself a tutorial for updating Lenny to Squeeze, test it on your clean install (documenting as you go). Once you have it working successfully put the instructions you used into the conf script of a tklpatch! Then package, test and post to a new ('general') forum topic.

I used my google-fu and found

I used my google-fu and found the build system that Debian supports for custom live cds, here is a link to the home page:

http://live.debian.net/

It took me a while to figure out how to use it but I have a working live cd with tklpatch installed.  I am starting a thread in the general forum for some assistance on making it more inline with how the existing live cds work.

Here is a link to my forum post.

http://www.turnkeylinux.org/forum/general/20110214/debian-squeeze-pre-alpha


Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)