TurnKey Linux Virtual Appliance Library

TKLBAM restore issues

derek.schwartz's picture

Hi.

I'm close to finally going live my employer's new TKL Drupal 6 appliance on Amazon EC2 via the TurnKey Hub.

Everything with the TK Hub seems to be working great, but I decided to try a disaster recovery scenario and was disappointed by the result.  I'm hoping that I'm simply missing something.

I came up with the following recovery procedure for my boss, in case I'm not available when disaster strikes.  Am I missing anything here?  I included step three because, without it, Drupal keeps trying to launch its installation procedure.  I'm confused as to why this happens if all original data is supposedly restored.

In the TurnKey Hub interface:
1 - Launch a new server like this one
2 - Actions > Elastic IP > Associate existing
3 - Browse to the displayed IP and click "Install Drupal in English" to complete the installation sequence.  (Anything entered will be replaced when the backup is restored, so don't bother writing down the password that you set during this step!)
4 - Open the command console from the TurnKey Hub interface and enter the following commands in order:
  1. apt-get update
  2. apt-get install tklbam
  3. tklbam-init {API key}
  4. tklbam-restore 1
  5. chmod +x /etc/cron.daily/tklbam-backup
  6. reboot

Post recovery, the Drupal site looks fine.  Once I login with the administrator account, I see the following errors which are fixed by browsing to /admin/settings/performance and clicking the "Clear cached data" button.  Once again, shouldn't the data restore replace the cached data, along with everything else in the database?  Why would I need to do this if all data is being restored?

Is it possible that not all modifications to the TKL appliance are being captured during backup?  The Drupal core version has also reverted to 6.14, but it was 6.19 before the initial TKLBAM backup.  What am I missing?

 

Login successful.
  • warning: include_once(user.pages.inc) [function.include-once]: failed to open stream: No such file or directory in /usr/share/drupal6/includes/theme.inc on line 613.
  • warning: include_once() [function.include]: Failed opening 'user.pages.inc' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drupal6/includes/theme.inc on line 613.
  • warning: include_once(user.pages.inc) [function.include-once]: failed to open stream: No such file or directory in /usr/share/drupal6/includes/theme.inc on line 613.
  • warning: include_once() [function.include]: Failed opening 'user.pages.inc' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drupal6/includes/theme.inc on line 613.
  • warning: include_once(user.pages.inc) [function.include-once]: failed to open stream: No such file or directory in /usr/share/drupal6/includes/theme.inc on line 613.
  • warning: include_once() [function.include]: Failed opening 'user.pages.inc' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drupal6/includes/theme.inc on line 613.

 

Thank you for any suggestions,

-Derek

Liraz Siri's picture

Did you put the theme in a naughty place?

Good idea to test your backups to make sure they work as you expect. Everyone should always do that.

I think what you're missing is the concept of what directories in the appliance you are "allowed" to manipulate directly (e.g., anything in /etc, /usr/local, /var/www, etc.), vs what directories in the appliance are the responsibility of package management - touching these is naughty (e.g,. /usr/share). These "rules" aren't arbitrary, but are inherited from the Debian way of doing things. TKLBAM builds on these assumptions, so if you break them you will need to at least understand the consequences. I recommend you just don't break them.

In your case I'm guessing you installed your theme to the "wrong" place (e.g., /usr/share/drupal6/themes instead of /etc/drupal/6/sites/all/themes).

That didn't work because by default TKLBAM only backs up changes to directories that are not usually maintained by package management (e.g., apt-get) to avoid conflicts.

If you haven't done so already I recommend installing webmin-tklbam package:

apt-get install webmin-tklbam
That should make it easier to understand how TKLBAM's configuration works (e.g., which directories are backed up by default according to the profile for TurnKey Drupal6, etc.).

To fix this problem, I recommend moving your theme on the backed up machine to the right location.

From the sound of it you also probably made the mistake of trying to overwrite files in the drupal6 Debian package by extracting a new Drupal6 tarball from upstream to the same location.

If you insist on maintaining drupal6 files directly rather than through package management add /usr/share/drupal6 to the overrides, but make sure you understand what this means. It's generally a bad idea to mess with files from package management as package upgrades will overwrite them. This is what /usr/local is for. It will be a bit more work, but if you really want to install Drupal6 from upstream, remove the drupal6 package (e.g., apt-get remove drupal6), and extract the tarball directly to /var/www.

Hope this helps!

Guest's picture

Had no idea about naughty places!

I noticed that my backup was much smaller than it was supposed to be, and after some googling i ended up here!

Since the default web folder is var/www, i had place a couple of other webapps in /var, and they were not picked up by TKLBAM!

Moving them to /usr/local fixed the whole thing.

I guess this is proof that TKLBAM is not idiot proof. :)

 

Derek Schwartz's picture

No naughty theme location, but manual upgrades to Drupal core!

Hi, Liraz.

Thanks for the quick reply and for clearing up what I was missing.

I do have my theme installed in the correct place:  /etc/drupal/6/sites/all/themes/zen

I think the errors were due to old cache data being used, since flushing the Drupal performance cache took care of them.

Unfortunately for me, I had no idea that I could update the Drupal core with the package manager.  For the last nine months, I have been manually updating the Drupal core on my TKL appliances as directed by the Drupal documentation.  Now, it makes sense as to why the Drupal core version reverted -- it was never backed-up!  I should have used the package manager to upgrade the Drupal core before running the TKLBAM restore.

Now that you explained what I was missing, letting the appliance handle application upgrades, I'm going to give TKLBAM another shot.  I'm not going to mess with this currently functioning system, but the first AMI termination will present an opportunity to restore everything the right way!

Thanks for your time,

-Derek

Guest's picture

naughty places

Hi, 

I am brand new to this and  I don't understand a lot of what you're talkinga about.  For exaample; everyone talks about installing this file or that, such as, /etc/drupal/6/sites/all/themes/zen, above. If I were attempting to install a style it would be enormously helpful to know the path I would need to get it there. 

I do all the moves that Filezilla tells me, and it tells me that the transfer is complete.   When I open phpBB3 styles, however, it is never there.

I can understand "upstream".  I don't, however, know what a tarball is.  What is a Drupal  and what is a TKLBAM, and if I am to write to them how do I get to where I would find them?  I have the same problem with phpBB3 help.  If one is not a tech, how can use the information.  Every guide I have read, assumes that it is written to very advanced compute people.  All I want to do is create a particular forum for in my area, and all others welcome, but, I must say, so far, I am pretty frustrated.

I Do want to learn to do these things for myself as well, but I live in a place where there is no one with the knowledge to help me.

I hope I haven't been whimpering too much, but  I'm desperate and would appreciate suggestions and a couple answers to these questions.

Thanks a bunch

Butch

Jeremy Davis's picture

Are you using TurnKey Linux?

If so then you're almost in the right place. If not then I suggest that you try the forums for your operating system.

TBH I don't think that this thread is really that relevant to you.

I suggest you start your own thread. Post the output of the "turnkey-version" command and any other relevant details (e.g. where it is running, etc).

Then tell us what you are trying to accomplish; what you've tried already and what the result was. Posting links to other threads/posts (within TurnKey or elsewhere) is fine e.g. if you are trying to follow the steps from a particular post. If you get any error messages, paste them verbatim.

Atlasssian Confluence Backup Size

I'm using the turnkey hub backups of atlassian in confluence system using the turnkey postgrees appliance. Backup works okay but I noticed that the size of the backup is incorrect. Checking the log files it seems /var/atlassian/application-data/confluence/ is not being backed up.

I understand that I have to add an override but can you confirm thi ssyntax

 /var/atlassian/application-data/confluence/
-/var/atlassian/application-data/confluence/backups/

the first line includes the confluence directory  and the second line excludes the backup folder ?

also my concern is that other dependent folders are not included any ideas how to establish if I have a full backup ?

Regards

Paul


Jeremy Davis's picture

Hi Paul

Apologies on delayed response.

You have the theory correct. Mention of a path adds it to what get's backed up; a path proceeded with a '-' is excluded from backup.

All appliances inherit their TKLBAM profile from Core (many of them are actually just blank). FWIW you may find consulting the default Core profile useful?. In that you'll note that most of /var is not backed up; only some specific white-listed places...

If you would like to go over all the files that are included in your backup, use the '-s' (--simulate) switch. That will initiate a backup and collect all the files in /TKLBAM; but it will stop prior to creating the archives and uploading them. So you can essentially inspect the contents of your backup.

The only thing I'm not 100% sure about is whether --simulate always creates a full backup or whether it will create an incremental (if it's not yet time for a full backup)? You could use the '--full-backup' switch to ensure that it's a full backup.

So your command would look something like this:

tklbam-backup --full-backup 1D --simulate
You can safely delete /TKLBAM when you're done! :)

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)