TURNKEYGNU / LINUX
English

Forum archive

how do I get tklbam to work with custom packages and APT repositories?

Supporttklbam

tklbam seems not to work ??

today i have been testing tklbam ... I did follow the instructions in the case study:

1 - launch a turnkey appliance from amazon console (turnkey apache tomcat lucid)

2 - run tklbam init ; tklbam backup

2 - install some packages ... nxserver, lxde, chrome and sun java.

3 - run tklbam backup

4 - And now ... I start another instance of the same appliance  

5 - run tklbam init ; tklbam backup ; tklbam restore {id of the backup of the other appliance}

6- restart

The backup does not show any error but the instance seems no to work, in particular , I tried 

the command java -version , and i got the message "java command not found"

I do not know if I am doing something wrong. Please could any one bring some light ?

 

Thank you and kind regards.

Omar

Replies (8)

Need more information

Reply 1
Could you share the backup and restore logs? Also it might be helpful to understand how exactly you installed additional packages. Was it all done through the package management system? Did you add any custom repositories?

Hi Liraz Those are all the

Reply 2

Hi Liraz

Those are all the commands I typed in the server in order to install them mentioned packages (from the ./bach_history)

 

Init tklbam
----------
tklbam init
tklbam bakcup
tklbam backup test
 
 
Install MX
-------------
mkdir NX
apt-get install libaudiofile0
dpkg -i nxclient_3.4.0-7_i386.deb
dpkg -i nxnode_3.4.0-14_i386.deb
dpkg -i nxserver_3.4.0-14_i386.deb
nano /usr/NX/bin/nxwrappe
#!/bin/bash

nxnode=$(ps -o ppid= -p $PPID)

nxagent=$(pgrep -P $nxnode -u $USER -x nxagent)

${1+"$@"}

exec kill $nxagent
sudo chmod 755 /usr/NX/bin/nxwrapper
mv /usr/NX/bin/nxwrappe /usr/NX/bin/nxwrapper
chmod 755 /usr/NX/bin/nxwrapper
 
Installing lxde
---------------
apt-get install aptitude
aptitude install --without-recommends lxde-core
aptitude install gksu gnome-icon-theme manpages-dev openbox-themesaptitude install gksu gnome-icon-theme manpages-dev openbox-themes
sudo aptitude install synaptic
passwd
vim /usr/NX/etc/server.cfg
EnableAdministratorLogin = "1"
shutdown -r now
 
Installing Chrome
-----------------
tklbam backup
vim /etc/apt/sources.list.d/sources.list
(adding at the end)deb http://dl.google.com/linux/deb/ stable main
apt-get update
apt-get install google-chrome-unstable
 
Install java sun
----------------
tklbam backup
vim /etc/apt/sources.list.d/sources.list
(adding at the end)deb http://archive.canonical.com/ lucid partner
aptitude update
aptitude install sun-java6-jdk
update-alternatives --config java
update-alternatives --config javac
java -version
 
tklbam backup

here is the restore log (the last bit of it)

Reply 3

 

    /etc/.git/objects/61/6a7d2e87b5647a20d4a42817a8377b20c661f8
/etc/.git/objects/61/a606d91b875d021bf62197eb0bd4ef18e1dde9
/etc/.git/hooks/commit-msg.sample
/etc/.git/hooks/pre-commit.sample
/etc/.git/hooks/applypatch-msg.sample
/etc/.git/hooks/post-update.sample
/etc/.git/hooks/pre-applypatch.sample
/etc/.git/hooks/pre-rebase.sample
/etc/.git/hooks/update.sample
/etc/.git/hooks/pre-commit
/etc/.git/hooks/prepare-commit-msg.sample
/etc/.git/hooks/post-receive.sample
/etc/.git/hooks/post-commit.sample
/etc/.git/logs/HEAD
/etc/.git/logs/refs/heads/master
/etc/.git/refs/heads/master
/etc/hal/fdi/policy/preferences.fdi
/etc/webmin/system-status/info
/etc/webmin/system-status/history/swapused
/etc/webmin/system-status/history/cpuidle
/etc/webmin/system-status/history/load
/etc/webmin/system-status/history/procs
/etc/webmin/system-status/history/load5
/etc/webmin/system-status/history/netcounts
/etc/webmin/system-status/history/maxes
/etc/webmin/system-status/history/cpuuser
/etc/webmin/system-status/history/diskused
/etc/webmin/system-status/history/load15
/etc/webmin/system-status/history/bout
/etc/webmin/system-status/history/cpukernel
/etc/webmin/system-status/history/memused
/etc/webmin/system-status/history/cpuio
/etc/webmin/system-status/history/bin
/etc/xml/sgml-data.xml.old
/etc/xml/catalog
/etc/xml/docbook-xml.xml
/etc/xml/docbook-xml.xml.old
/etc/xml/xml-core.xml.old
/etc/xml/sgml-data.xml
/etc/xml/rarian-compat.xml
/etc/xml/catalog.old
/etc/xml/xml-core.xml
/etc/menu-methods/openbox
/etc/default/dbus
/etc/default/google-chrome
/etc/default/inithooks
/etc/dbus-1/session.conf
/etc/dbus-1/system.conf
/etc/dbus-1/system.d/hal.conf
/etc/gamin/gaminrc
/etc/logrotate.d/aptitude
/etc/gconf/gconf.xml.defaults/%gconf-tree.xml
/etc/gconf/2/path
/etc/gconf/2/evoldap.conf
/etc/gconf/gconf.xml.mandatory/%gconf-tree.xml
/etc/init.d/dbus
/etc/init.d/nxsensor
/etc/init.d/nxserver
/etc/init.d/inithooks
/usr/local/lib/python2.6/dist-packages/ec2metadata.py
/usr/local/lib/python2.6/dist-packages/ec2metadata.pyc
/var/spool/postfix/etc/resolv.conf
/var/spool/postfix/etc/hosts
/var/spool/postfix/etc/services
/var/spool/postfix/etc/localtime
/var/spool/postfix/etc/nsswitch.conf
/var/spool/postfix/etc/ssl/certs/ca-certificates.crt
/var/spool/postfix/usr/lib/zoneinfo/localtime
/var/spool/postfix/pid/master.pid
/root/.bash_history
/root/.penv
/root/nxserver_3.4.0-14_i386.deb
/root/nxclient_3.4.0-7_i386.deb
/root/core
/root/nxnode_3.4.0-14_i386.deb
/root/.Xauthority
/root/.dbus/session-bus/e603ec161102f448e8977af94d1de84c-1003
/root/.ssh/known_hosts
/root/.ssh/authorized_keys2
/root/.ssh/authorized_keys
/root/.config/google-chrome/Local State
/root/.config/google-chrome/First Run
/root/.config/google-chrome/Default/History
/root/.config/google-chrome/Default/Favicons
/root/.config/google-chrome/Default/Web Data
/root/.config/google-chrome/Default/Current Session
/root/.config/google-chrome/Default/Bookmarks
/root/.config/google-chrome/Default/Cookies
/root/.config/google-chrome/Default/Top Sites
/root/.config/google-chrome/Default/Visited Links
/root/.config/google-chrome/Default/Archived History
/root/.config/google-chrome/Default/Preferences
/root/.config/google-chrome/Default/Current Tabs
/root/.config/google-chrome/Default/History Index 2010-12
/root/.config/google-chrome/Default/User StyleSheets/Custom.css
/root/.config/pcmanfm/main.lxde
/root/.config/openbox/lxde-rc.xml
/root/.pki/nssdb/cert9.db
/root/.pki/nssdb/key4.db
/root/.pki/nssdb/pkcs11.txt
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/stats
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/errors
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/clients
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/session
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/options
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/authority
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/scripts/authority.applications
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/scripts/authority
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/pids/monitor
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/pids/agent
/root/.nx/C-tomcat-apache-1003-BEF127BF09AF94CECA3274EBC1AF956D/pids/applications
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/stats
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/errors
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/session
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/options
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/authority
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/scripts/authority.applications
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/scripts/authority
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/pids/monitor
/root/.nx/F-C-tomcat-apache-1001-3ADE78FBEBC1AC9AA7F98BB1F1A74EF4/pids/agent
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/stats
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/errors
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/session
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/options
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/authority
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/scripts/authority.applications
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/scripts/authority
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/pids/monitor
/root/.nx/F-C-tomcat-apache-1002-D994574136E0E9F9CFE75265F346ECFF/pids/agent
/root/.gnupg/pubring.gpg
/root/.gnupg/gpg.conf
/root/.gnupg/secring.gpg
/root/.gnupg/random_seed
/root/.aptitude/config

POST-OVERLAY FIXES:

mkdir -p /opt/google/chrome/extensions
mkdir -p /root/.config/lxpanel
mkdir -p /root/.config/lxpanel/LXDE
mkdir -p /root/.debtags
mkdir -p /root/.gnome2
mkdir -p /root/.gnome2/keyrings
mkdir -p /root/.local
mkdir -p /root/.local/share
mkdir -p /root/.local/share/openbox
mkdir -p /root/.local/share/openbox/sessions
mkdir -p /root/.nx/cache-desktop
mkdir -p /root/.thumbnails
mkdir -p /root/.thumbnails/normal
mkdir -p /root/Desktop
mkdir -p /root/NX
chmod 0700 /opt/google/chrome/extensions
chmod 0700 /root/.aptitude
chmod 0700 /root/.config/google-chrome
chmod 0700 /root/.config/google-chrome/Default
chmod 0700 /root/.config/google-chrome/Default/User StyleSheets
chmod 0700 /root/.config/lxpanel
chmod 0700 /root/.config/lxpanel/LXDE
chmod 0700 /root/.dbus
chmod 0700 /root/.dbus/session-bus
chmod 0700 /root/.gnome2
chmod 0700 /root/.gnome2/keyrings
chmod 0700 /root/.local
chmod 0700 /root/.local/share
chmod 0700 /root/.local/share/openbox
chmod 0700 /root/.local/share/openbox/sessions
chmod 0700 /root/.nx/cache-desktop
chmod 0700 /root/.pki
chmod 0700 /root/.pki/nssdb
chmod 0700 /root/.thumbnails
chmod 0700 /root/.thumbnails/normal
rm /etc/nologin

Restoring databases
===================

database: mysql
table: mysql/time_zone_transition
table: mysql/proc
table: mysql/tables_priv
table: mysql/func
table: mysql/host
table: mysql/columns_priv
table: mysql/time_zone_name
table: mysql/procs_priv
table: mysql/help_category
table: mysql/help_relation
table: mysql/help_topic
table: mysql/event
table: mysql/plugin
table: mysql/db
table: mysql/user
table: mysql/time_zone_transition_type
table: mysql/time_zone_leap_second
table: mysql/servers
table: mysql/ndb_binlog_index
table: mysql/help_keyword
table: mysql/time_zone

the last bit of the backup log

Reply 4

 

 
A var/spool/postfix/pid/unix.cleanup
A var/spool/postfix/pid/unix.local
Uploading s3://s3-eu-west-1.amazonaws.com/tklbam-oraursstloxj3ll7/duplicity-inc.20101231T150823Z.to.20110105T234217Z.vol1.difftar.gpg to STANDARD Storage
Processed volume 1
Uploading s3://s3-eu-west-1.amazonaws.com/tklbam-oraursstloxj3ll7/duplicity-new-signatures.20101231T150823Z.to.20110105T234217Z.sigtar.gpg to STANDARD Storage
Uploading s3://s3-eu-west-1.amazonaws.com/tklbam-oraursstloxj3ll7/duplicity-inc.20101231T150823Z.to.20110105T234217Z.manifest.gpg to STANDARD Storage
 
--------------[ Backup Statistics ]--------------
StartTime 1294270938.68 (Wed Jan  5 23:42:18 2011)
EndTime 1294270942.93 (Wed Jan  5 23:42:22 2011)
ElapsedTime 4.25 (4.25 seconds)
SourceFiles 2867
SourceFileSize 135552974 (129 MB)
NewFiles 264
NewFileSize 4313037 (4.11 MB)
DeletedFiles 14
ChangedFiles 85
ChangedFileSize 14062736 (13.4 MB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 363
RawDeltaSize 5253140 (5.01 MB)
TotalDestinationSizeChange 2805919 (2.68 MB)
Errors 0
-------------------------------------------------

From my understanding...

Reply 5

AFAIK TKLBAM leverages the package management system to install any extra packages from default appliance state (ie any extra packages that aren't installed originally).

I'm not sure but I suspect that as this package isn't available from a repo, TKLBAM doesn't know where to get it from and hence its not included. It's a scenerio which probably should be accounted for.

If that's the case, you may be able to work around it by manually including the app path as part of the backup. Another possible workaround may be to host a repo locally for that deb (and any other custom ones).

I have found that the nx

Reply 6

I have found that the nx server has not been backup, maybe due to the way i installed it :

 

 

dpkg -i nxclient_3.4.0-7_i386.deb
dpkg -i nxnode_3.4.0-14_i386.deb
dpkg -i nxserver_3.4.0-14_i386.deb
 
Please can anyone confirm that this type of manual installation wont be backup, so we can take care of them in another way ? 
 
Thanks 

Configure repositories before restoring

Reply 7
Try and configure the package repositories before performing the restore. If you want this to be done automatically, look into setting up a custom hook that runs on restore in the "pre" state. If packages are only available as downloadable *.deb files and are not installable through an APT repository (e.g., NX) I recommend you save the packages in /root/debs and install them from the custom hook in the "post" state.

$ cat /etc/tklbam/hooks.d/example

#!/bin/bash -e
# This is a disabled hook example.
# To enable, make it executable: chmod +x /etc/tklbam/hooks.d/example

# hooks are always called with two arguments
op=$1
state=$2

if [ "$op" = "restore" ]; then
    echo -n "A restore operation called this hook "
elif [ "$op" = "backup" ]; then
    echo -n "A backup operation called this hook "
fi

if [ "$state" = "pre" ]; then
    echo "right BEFORE duplicity was invoked"
elif [ "$state" = "post" ]; then
    echo "AFTER the $op operation finished"
fi

# `false` returns a non-zero exitcode
# Uncomment the next line to raise an error

#false

As JedMeister correctly suspected, when you tklbam-restore from a new installation, TKLBAM is not aware of the new package repositories on your old installation and it doesn't attempt to synchronize them. This is by design, as otherwise we wouldn't be able to use TKLBAM to upgrade between different versions of Ubuntu or migrate between Ubuntu and the upcoming Debian versions, etc.

TKLBAM hook example

Reply 8

One of my cloud servers uses packages from multiverse, which is not enabled by default in TKL, which means even though TKLBAM knows about them in the backup, they are not installed when I launch a new server from the backup.

I wanted everything to be automated, so I created the following TKLBAM hook, and thought that others might find it useful:

#!/bin/bash -e
# hooks are always called with two arguments
op=$1       # backup / restore
state=$2    # pre / post

if [ "$op" == "restore" ] && [ "$state" == "post" ]; then
    sed -i "/multiverse/ s/# //g" /etc/apt/sources.list.d/*.list
    export DEBIAN_FRONTEND=noninteractive
    apt-get update
    apt-get -y install PACKAGE1 PACKAGE2
fi

Note that just enabling multiverse in state=pre and letting TKLBAM take care of the package installation won't work on a newly launched server because the backed up filesystem is not restored until after package installation.