TurnKey Linux Virtual Appliance Library

Packages from Repositories vs up-to-date packages

Hi all,

My name is Mike and I'm working on the OTRS project. One of our customers pointed us to the availability of a Turnkey VM for OTRS. I like that idea a lot, and there would be certainy a use for a 'quick deploy' OTRS solution.

However, since you're based on Ubuntu 8.04 your appliance ships OTRS 2.2.x, which was first available in the summer of 2007. Our major version release cycle is usually one year, so you'll miss on version 2.3.x (summer 2008) and 2.4.x (summer 2009) which have lots and lots of new features. This makes the Turnkey VM not so attractive: "Here, try out this TurnKey appliance, it makes it really easy to set up a very outdated version of OTRS!".

Is there something we can do to help Turnkey run the 'Latest' OTRS version? Of course, Ubuntu Lucid (10.04 LTS) has OTRS 2.4.x in the repositories, but that does not help much as you'll have the very same issues again in just a couple of months' time.

In my opinion, if you're serious about OTRS, you should not set up the version from your Linux's package manager, because you would only get the version that was stable when the distribution was published, plus security updates, rather than an up-to-date version.

This would also apply for other software as well, for instance the Bugzilla appliance running 3.0.x while the up-to-date version is 3.4.x, so it is a general issue which in my opinion deserves a good solution.

--

Mike.

Liraz Siri's picture

Installing from upstream vs the package manager: pros and cons

Hi Mike, thanks for reaching out.

The reason we install OTRS from the package manager is that it's easier to maintain software that way and you get a well tested, stable version that is often also supported with security updates which are safe to install automatically - because they only backport the security fix itself with no other changes that could break your deployment.

The downside of course is that you don't get the latest version for upstream. It's a compromise.

We would consider including the upstream version of OTRS in future versions of the appliance if OTRS could somehow update itself safely without relying on the package management system. WordPress does that now, so the WordPress appliance includes the latest version from upstream.

An alternative route would be to maintain your own security updates repository for specific versions. Providing new versions isn't enough as it's unstable. You would need to commit to providing packages with backported security fixes to a version once you release it. You could either do it in-house or maybe sponsor the work done by the Debian OTRS maintainer. If there's a new version in a stable supported version of the OTRS package in Debian, we might use that instead of the version in Ubuntu.

Would a PPA be an option?

So, I'm still puzzled? What would be the actual advice here?

We provide security updates for up to four minor versions. We don't have security issues often, but fot the last one we released patches for 2.1, 2.2, 2.3 and 2.4. We create a new minor release version (aka 2.4.x) but also just point to the actual diffs for the fix.

As I pointed out, debian Testing currently has the latest version of OTRS, just as Lucid. But whenever we release the next version of OTRS, we will have the same problem again.

Also, there's an issue with the way Debian handles files, and the way we do it. One of the most powerful features in OTRS is the package manager. You can install additional functionality such as a knowledge base, or an integration with a system monitoring system such as Nagios or OpenNMS by installing it from our web based repositories. On Debian, this is broken because of the fact that privileges have to be raised to a level Debian does not allow. Of course, for the regular package on my server I can understand this, but for the main package on my server, in this case OTRS, it is not really helpful. The debian Wordpress package also suffers from a broken plugin management, and disfunctional attachments upload, because of the same reason.

This might be yet another reason to switch away from the OTRS package to something else. So my question is, would you think it would be a viable option if we create a PPA for OTRS and you use that to get OTRS on the TurnKey appliance?

Regards,

Mike.

Liraz Siri's picture

Debian's packaging policy is "the law" for a good reason

Mike, I've learned the hard way that you have to be very careful before you consider breaking Debian packaging rules because you can easily screw things up that way. For example, if OTRS includes its own package management system, that could conflict with the Debian package management system in unexpected ways. If so, it may be better to redesign that part of OTRS to leverage the regular package management system somehow. At least on Debian/Ubuntu. There is rarely a good reason to break packaging policies, especially if you don't fully appreciate the ramifications. It may require more work on your end but there should be a way to achieve your all of your goals without breaking "the law".

We're not too eager to diverge from the Debian OTRS package, especially if your package implements its own package management system in a way that violates the Debian Policy Manual.

We would be open to using your PPA as the source for OTRS if:

  1. You can produce a package that doesn't violate packaging policy in a potentially harmful way.
  2. You could commit to backporting security updates the way Debian does.

Again, the main advantage of using a Debian package in the stable distribution is that the Debian security team have committed to backporting security fixes to the version that was shipped with the release. This policy is what makes it safe for us to configure an appliance to auto-install packages from Debian's security updates repository.

Out Of Date Packages

Hello There!

First, I'd like to congrats to all the team of turnkey, the work you are doing is awesome, and I hope you all achieve your personal goals with the project.

Its the first time for me using linux.. so the questions could be very dumb.. sorry about that. Im using the "Version Control Appliance" and I was trying to obtain(update) the newest versions for the soft in the appliance to start with my projects. The outdatings I care are: apache 2.2.8 to 2.2.15 / subversion 1.4.6 to 1.6.11 / websvn 2.0 to 2.3.1. All the efforts I tried with: apt-get update, install, downloading, etc were fails, and finally I found this thread.

Dont know anything about package managers, packaging policies, etc. So, I dont understand if theres any way to update those packages, or if that will generate incompatibility with other packages. I would appreciate if you can explain me the difference between this methodology and the one from (for example) "Bitnami Stack SVN" (that sticks to the latest versions).

Besides this issue, Im very happy with the appliance, It have all I can want, is very easy to use and config, and specially and most important, works like a charm!

Very thanks in advance.

Kind Regards.

Jeremy's picture

I assume there are feature you want/need in the later versions?

If so then the current version of TKL appliance may not be for you. The current appliances are built on Ubuntu 8.04 and the program versions are the latest available for that release. (hence why you couldn't update them using apt-get). If it is security you are concerned about then no worries as all security patches are backported so even though the versions are older they are stable and secure.

As mentioned by others, there is always a trade off between stability and 'latest & greatest' version. Linux generally handles that quite differently to Windows. There are advantages and disadvantages for both approaches. Linux generally favours an approach which maximises stability and security (with minimal need for end user effort) over the 'latest and greatest' versions. Generally the Windows way allows for greater (and easier) access to 'latest and greatest' but at the potential expense of stability and the reliance on the end user being aware of security flaws and constantly addressing them themselves (ie keeping abreast of potential security issues, checking for new versions, and manually downloading and updating programs). Some Windows app builders are developing ways to keep their apps up to date, but its all very ad-hoc.

OTOH Linux generally uses unified libraries of apps called repositories (repos for short). These contain apps that have all been tested to work nicely together (a few bugs slip through from time to time, but generally). These allows Linux apps to be very modular in nature instead of being huge monolithic programs. For the life of that particular release the apps will not change version. Security issues and bugfixes are backported into that specific version (rather than include a new version which may conflict with other packages or raise new bugs or security risks). Occasionally new versions of apps will be backported to earlier OS version (you can find them by enabling the 'backports' repository). This is not recommending in production though as secuirty fixes for these newer backported versions are not always released in a timely fashion (or sometimes at all) as they are community supported and rely on volunteers. Another option is to use a PPA (Personal Package Archive). You can find some great up to date software in these but again they should generally not be used in production (unless of course it's your own PPA which you maintain) for the same reasons.

As per usual with general rules, there are exceptions; some Linux distros have what is known as a 'rolling' release, ie they do not release a specific 'stable' version, they just constantly update packages. Again this has positive and negative aspects, but i won't go into that here as its getting a bit off topic.

Bottom line is that the few options I see are:

  • Be patient and put up with the old versions until the next release of TKL (which will be based on Ubuntu 10.04) with updated apps.
  • Find some other repo (such as backports or a PPA that will fulfill your needs). To some degree this will also take security out of your hands but it will still allow for bulk updates using apt (assuming the repo maintainers produce patched/updated versions).
  • Choose a different distro/OS and roll-your-own solution. BitNami have a universal (Windows style) Linux installer that runs on Ubuntu 10.04 (Desktop) or you could just use 10.04 and install and set it up yourself. Debian Squeeze (still in testing) may be another option?

Regardless, good luck!

Liraz Siri's picture

So true. You can't have your cake and eat it too

Very well put! As you say there is a natural tension between various goals so you can't always have your cake (e.g., auto-installing security updates that won't break anything) and eat it too (e.g., latest and greatest package versions).

These days I tend to prioritize stability and security over being on the cutting edge. Sure, occasionally there's a new feature I hear about in a new version that I miss but  speaking as a former Gentoo user what I don't miss is having to constantly spend precious mental energy figuring out why things broke on the latest update.

Gentoo doesn't do releases, they just roll out new ebuilds for individual packages so it was always happening. Eventually I realized it was inevitable and that's the price you have to pay for making that choice. So I made a different choice, moved to a Debian and Ubuntu based Linux distributions and never looked back.

Note that if you really want to be on the cutting edge with regards to new packages (and understand the consequences) you can point APT to an unstable version of the Ubuntu or Debian package repositories. I've never tried this with Ubuntu but in my experience Debian "testing" is more usable on average than the "releases" of other distributions.

Anyhow, everyone that wants more up-to-date stable packages is going to get that soon with the Ubuntu 10.04 based Betas. We're still working on that (and on a few other things!) but sit tight, it's in the works.

But I expect it will only take about 6 months after that till we add another round to this thread (or another like it). C'est la vie.

Perhaps when we come out with Debian based appliances we can tell users that want newer packages to add the "testing" repository to their APT configuration and cross their fingers.

I'm confused

First you say:

We would consider including the upstream version of OTRS in future versions of the appliance if OTRS could somehow update itself safely without relying on the package management system. WordPress does that now, so the WordPress appliance includes the latest version from upstream.

But then when he mentions that they have an internal plugin/package system for adding additional options to the software you state that going outside of the distribution's package system is "a bad thing". If Wordpress doesn't use the package system I'm not sure how this adds up?

Either way I think that if OTRS could update internally without having to be packaged (since it does have a source option available) this could be ideal, but probably non-trivial at the moment.

Jeremy's picture

The devs make decisions on a case by case basis

So each appliance is considered separately.

The reasons for choosing to install from upstream vary. In some cases the repo versions are so far out of date that they are missing in features to the point that they are not fulfilling the needs of the community; occasionally a repo version of a package is a bit buggy, or doesn't work that well with other software included (Moodle with php5.3 springs to mind here) or that like you say WordPress has an internal update mechanism that is sufficently robust and the additional features are sufficently desireable.

So it's not that non repo versions as 'bad' as such it's just that it's not a decision that can (or at least should) be made lightly, and the full implications need to be thought through before a decision to deviate from the repo version is made.

And by my understanding; the point that Liraz makes, is not that 'going outside of the distribution's package system is "a bad thing"'. AFAIK, non-package management updates can be ok, but they need to be done carefully and not conflict with Debain package management policy.

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)