Observium

This documentation page provides Information related to the TurnKey Linux Observium appliance. The current version of this doc was written specifically with TurnKey version 18.x in mind, but should remain compatible with newer versions.

Please provide feedback and/or problems regarding this doc page via the TurnKey Forums (requires free TurnKey Lnux website user account). Alternatively documentation bugs can be reported via our consolidated TurnKey Linux issue tracker.

Known issue: "Unable to add device manually via web, Could not ping <hostname>"

As noted on the TurnKey Linux consolidated issue tracker, this issue can be resolved if desired. However as also noted (in a comment in the issue) the upstream Observium docs recommend that you don't do that because of security implications; and instead use the CLI to add devices. So it's up to you...

Updating Observium

All of these update instructions are based on the official Obervium Update Documentation. There are additional details noted here specific to the TurnKey Observium appliance. Regardless, you may wish to consult the official documentation page as well - especially if you upgrade to a Subscription Edition.

Contents:

Before you start; Backup!

An automated backup regime (e.g. using TKLBAM) should always be configured for any system that contains data that you care about. System data loss is always a matter of when, not if. The Observium upgrade process is expected to go smoothly, so a recent backup may be a enough piece of mind. Regardless, a manual backup immediately prior to a software upgrade is always recommended.

The Observium upgrade process(es) documented below keep a copy of your old Observium files. However it does not keep a copy of your original database. Without a proper backup, you may not be able to rollback to a previous version if something goes wrong. The only scenario where a backup is not as important is if you have only just set your Observium server and have no existing data. Even then, a backup could still save you time if things go wrong and you need to roll back.

An important note for AWS Marketplace users

Most TurnKey servers default to using the 'root' user. If that's you, read on. For those using TurnKey launched from the AWS Marketplace, they default to an 'admin' user. So you'll need to switch to the 'root' user before these instructions will work. You could prefix every command with 'sudo', but that's a pain. Instead I recommend swithcing to the 'root' user for this process. You can do that like this:

sudo su -

When you're done, switch back to the 'admin' user by issuing the 'exit' command.

Updating Observium CE using update script

Backup your system

Perform a manual backup as noted above.

Updating via the 'turnkey-update-observium' script - included in the TurnKey Observium appliance

This script automates the Manual update instructions, plus creates a local backup of the Observium files (/opt/observium). The script is written in BASH and the source of the version included with your server can be viewed at /usr/local/bin/turnkey-update-observium or the latest source code on GitHub.

To run the script from the CLI:

turnkey-update-observium

To date there has been limited feedback related to this update script, although there have been no reports of errors or problems. However, due to limited feedback when the script is run it reports that is "experimental". Despite that, it should work reliably (see below note). As per any update or significant configuration changes, it is always recommended that you have a backup (e.g. TKLBAM) in case something goes wrong.

As noted when running the script:

Note that this script will check the current version from data created at TurnKey appliance build time. If Observium has been updated via an alternate method (or the file doesn't exist) it will always download and "update" to the latest Observium CE (community Edition). It will do this, even if you are already running the latest CE version; or have installed via an alternate channel!

Manually updating Observium CE

These instructions are based on the official Observium CE update docs - with some additional detail specifically relating to the TurnKey Linux Observium appliance. All commands should be run from the commandline as the root user.

Backup your system

Perform a manual backup as noted above.

Disable Observium cronjobs (optional)

Cron jobs are automated tasks that run on a schedule. Generally cron jobs are related to Observium "house keeping". Disabling the cron jobs is optional. However if you don't do this step, you may get some cron errors noted in your logs. If you do disable the cron jobs, ensure that you also re-enable them after the update (re-enablign cron jobs covered later in these instructions).

To disable Observium cron jobs:

Either manually edit the cron file - /etc/cron.d/observium - and comment out all active cron lines. I.e. add a '#' character to every line which does not already start with a '#'. If you are not experienced with this process, it is suggested that you add a '#' to every line (so all existing comments are prefixed with '##').

Alternatively run the sed command immediately below from the commandline. Please note that this command will add a '#' to every line in the cron file (as per above manual instructions). This command is not idempotent, so do NOT run it more than once! If you do, then you will need to manually edit the cron file to uncomment the cron lines and leave the actual comments with a '#' prefix.

sed -i "s|^|#|" /etc/cron.d/observium

Replace old files with the new version

The following commands move the current Observium files out of the way, download the latest version and then copy back the required files (config file and rdd & logs directories) from your original Observium install.

Run the following commands (as root):

cd /opt
mv observium observium_old
wget -Oobservium-community-latest.tar.gz https://www.observium.org/observium-community-latest.tar.gz
tar zxvf observium-community-latest.tar.gz
mv /opt/observium_old/rrd observium/
mv /opt/observium_old/logs observium/
mv /opt/observium_old/config.php observium/

Update DB schema

Update the DB schema for the new version:

/opt/observium/discovery.php -u

Force rediscovery (optional)

If it has been a very long time since you've updated (12 months or more), you may want to force an immediate rediscovery of all devices to make sure things are up to date :

/opt/observium/discovery.php -h all

Re-enable observium cronjobs (optional)

Re-enabling cron jobs is only required if you disabled them before doing the Observium update. If you skipped that initial step ensure that you do NOT follow this step! Instead skip straight to the next section.

To manually re-enable the cron jobs, reverse the process done in the disable cron jobs section above. I.e. uncomment the cron command lines in the Observium cron file - /etc/cron.d/observium .

If you disabled the cron jobs via the sed line noted above - in the disable cron jobs section above, then you can re-enable them with the below sed line. It will remove the (first) leading '#' character from every line. Do NOT do this unless you used the previous sed command to dis-able the cron jobs. As per the previous sed command, it is not idempotent, so do NOT run it more than once! If you do, then you will need to manually edit the cron file to comment out (add a leading '#' to) the intentional comment lines in the cron job.

sed -i "s|^#||" /etc/cron.d/observium

Final cleanup (optional)

Once you have confirmed that the update worked as expected, you may now delete the old Observium files created in the file update process above. I.e. remove the observium_old directory: 

rm -rf /opt/observium_old

Updating Observium CE to Observium Pro/Enterprise edition

These instructions are based on the official Observium CE -> Observium Pro docs - with some additional detail specifically relating to the TurnKey Linux Observium appliance. They are relevant to both Pro and Enterprise subscription editions. All commands should be run from the commandline as the root user - so many TurnKey users are good to go. AWS Marketplace users will need to switch to the root user first. I.e.: 

sudo su -

As noted on the relevant Observium doc page:

Observium has been designed with ease of upgrading and seamless migration in mind, allowing users to transition smoothly from the Community Edition to the Professional or Enterprise edition.

Backup your system

Perform a manual backup as noted above.

Install subversion

Observium subscription editions deliver the initial install code and future updates via the SVN (subversion) version control system. The TurnKey Observium appliance does not include the required subversion software by default. However it can easily be installed via apt (as root user):

apt update
apt install subversion

Disable Observium cron jobs (recommended)

Cron jobs are automated tasks that run on a schedule. Generally cron jobs are related to Observium "house keeping" although include other functionality. Disabling the cron jobs during upgrade is recommend, although strictly speaking is optional. However if the Observium poller (triggered by a cron job) runs during the code checkout process or while moving the rdd directory it may have unintended consequences. If you do disable the cron jobs, ensure that you also re-enable them after the update (re-enabling cron jobs is covered later in these instructions).

To disable Observium cron jobs:

Either manually edit the cron file - /etc/cron.d/observium - and comment out all the cron job lines. I.e. add a '#' character to the start of every cron job line. It is recommended you add a '#' to every line. That will mean that existing intentional comments are prefixed with '##' and will make re-enabling more reliable and less prone to mistakes.

Alternatively you can run the sed command immediately below via the commandline. Please note that this command will add a '#' to every line in the cron file (as per recommendation in manual instructions). This sed command is not idempotent - meaning that re-running this script will cause additional changes. So do NOT run it more than once! If you do, then you will need to manually fix/edit the cron file to re-enable cron jobs.

sed -i "s|^|#|" /etc/cron.d/observium

Migrate to SVN repository code - replace old files with the SVN version

To migrate to the SVN repository-delivered code, you need to check out a new copy of the code and transfer your existing config.php and rrd directory to the new copy.

The commands below will install the stable train. If you prefer to use the rolling train, replace branches/stable in the third command with trunk. See the Observium docs for more detail.

During the below update process, you'll be prompted for a username and password. These credentials can be found in the Observium Subscription Portal.

The following commands move the current Observium files out of the way, download the latest stable subscription version of Observium via SVN version and then copy back the required files (config file and rdd & logs directories) from the original Observium install.

cd /opt 
mv observium observium-old
svn co https://svn.observium.org/svn/observium/branches/stable observium
cp observium-old/config.php observium
mv observium-old/rrd observium-old/logs observium
cd observium
./discovery.php -u

Re-enable Observium cron jobs (recommended - required if disabled above)

Re-enabling cron jobs is required if you disabled them prior to the Observium upgrade - as recommended. If you skipped that initial step ensure that you do NOT follow this step! Instead skip straight to the next section.

To manually re-enable the cron jobs, reverse the process done in the disable cron jobs step above. I.e. remove the '#' from the start of each line (except the intentional comments) in the Observium cron file - /etc/cron.d/observium .

If you disabled the cron jobs via the sed command noted in the initial disable cron jobs step above, then you can re-enable them with the below sed line. It will remove the (first) leading '#' character from every line. Do NOT do this unless you used the previous sed command to dis-able the cron jobs. As per the previous sed command, this command is not idempotent and will make additional changes if run more than once. So do NOT run it more than once! If you do, then you will need to manually edit/fix the cron file to ensue that the intentional comments are prefixed with '#'.

sed -i "s|^#||" /etc/cron.d/observium

If need be, the default TurnKey Observium cron file can be consulted if you need to fix your cron file.

Final cleanup (optional)

Once you have confirmed that the migration upgrade worked as expected, you may now delete the old Observium files created in the upgrade process above. I.e. remove the observium_old directory: 

rm -rf /opt/observium_old

Upgrading Subscription Edition to latest

This process only applies if you have already upgraded Observium to a subscription edition.

Backup your system

Before starting, perform a manual backup as noted above.

Disable cron jobs (optional but recommended)

Disable Observium cron jobs as documented above.

Update to latest stable version

Updating subscription editions of Observium using SVN is simple. To upgrade Observium Subscription Edition to the latest stable release, use the commands below:

cd /opt/observium
svn update
./discovery.php -u

Alternate/Additional Observium Pro/Enterprise upgrade options

Please note that webapp software does not generally support downgrading to a previous version. I can't confirm that is specifically the case for Observium, but if you stick to that rule of thumb then you are less likely to hit issues. So creating a manual backup as noted above is always recommended and considered best practice.

Update Observium Subscription Edition to specific revision

Rather than updating to latest stable, you may wish to upgrade to a specific revision, i.e. a specific place in the stable release timeline. Upgrading Observium to specific revision is very similar to the process and commands noted in the upgrading instructions above. However, the 'svn update' command is appended with '-r <revision number>'. I.e.:

svn update -r <revision number>

For example, let's say you want to install/upgrade version 0.12.4.3049. The SVN revision is the last number, so run the following:

svn update -r 3049

Please see the offical Observium Subscription Edition upgrade docs for more options and details.