TurnKey Linux Virtual Appliance Library

TKBLM Not Showing Latest Backups...

My TKBLM Dashboard is not showing the latest backups. Shows only the backup from 11/8/2011 and none from 11/9/2011. Is there an issue anyone is aware of?

Dashboard not updating ...

Same issue here -- TKBLM Dashboard not updating ... multiple system backups missing.  Logs on each system indicate that the backups were successful.

Here too.

root@core ~# tklbam-status
TKLBAM:  Backup ID #10, Updated Wed 2011-11-09 21:31
root@core ~# tklbam-list
# ID  SKPP  Created     Updated     Size (MB)  Label
...
  10  No    2011-08-31  2011-10-26  58.75      Drone
...


 

appears to only be a problem with turnkey-core-11.2-lucid-x86

at least for me.  I have three of these and none of their backup statuses are updating, but my other machines (redmine, lamp) appear to be OK (though I haven't checked them all)

Looks like it is now updated

Looks like it is now updated today, and shows the ones that were missing from yesterday. Might have just been a glitch on TKLBAM yesterday.


L. Arnold's picture

I posted on the "general side' before seeing this.

i don't see the Backups from 11-10-11 AM on my Hub Account. Sounds like they are taking.  My question is what this does to the incremental backup system if one of the images is missing.

I don't think you can delete one image from your backup set and i don't know what is controlling duplicity, though i imagine it is the TKL install itself rather than the hub.

if TKLBAM thinks there is a backup, what will it do when it does not find it during restore?

Jeremy's picture

Others are reporting that all backups listed now

Are all your backups now listed on TKLBAM too? If so then all good.

In answer to your question, if an incremental backup is missing from the chain then that is very bad! The backup can not be restored unless all of the links in the chain (incremental backups) exist from the incremental backup you are trying to restore, all the way back to the preceeding full backup. That is why it is important that the chain isn't too long.

By default TKLBAM is set to monthly full backups (with no incrementals). I turn daily incrementals on, but set full backups to run every 2 weeks (see here). That way the chain is only ever 13 links long (13 incremental daily backups + 1 full backup = 14 days worth of backups). If you wanted hourly incrementals, personally I'd be looking at full backups every 12 hours. I'm not sure whether I'm being overly paranoid, but bottom line is that the shorter the chain the better IMO. I think it's also good practice to test your backups once every few full backs (so for my setup ideally I'd do it once every month or 2).

I'm not 100% sure what happened to the Hub (re not showing up all backups) but AFAIK the Hub is only displaying info, the important part is that TKLBAM itself is seeing all the incremental backups. Which from my understanding it did.

And yes Duplicity is controled by the local instance of TKLBAM (not the Hub). AFAIK, as long as your local TKLBAM knows what's going on, and the backups exist in your S3 container (assuming you are using the default TKLBAM storage option) then all will be well.

Hub still not updated ...

I also forced a backup on two of my servers to see if that would update what was reported.  My status page is still in error.  The last successful update reported was at Thu, 10 Nov. 2011 10:11:46 UTC.  I checked the hub status from two different ISPs on two different computers using two different browsers.  tklbam-status reports the backups were made.

Jeremy's picture

Not sure what's going on

I just ran a (full) backup (on a new machine) and it showed up in the Hub straight away. So not at all clear what/where the problem is (I haven't tried doing a new backup of an existing server yet).

I'm hoping one of the core devs will be along shortly to try to have a look at this. Perhaps even to clarify whether or not this is a major issue (ie backups not propagating to Hub and therefore dificult to restore if need be - and they need to get straight onto it) or only a minor issue (ie backups still ok, but just Hub not updating properly - but will catchup soon and/or they will fix when they have time).

[update] I suspect that this is related to this bug registered recently on the bug tracker.

Alon Swartz's picture

No need for concern...

We've been performing some Hub maintenance these last couple of days, so the TKLBAM/Hub cache sync was probably not being triggered. The way it works is as a backup is completed, TKLBAM sends a trigger to the Hub to update its cache. Due to the maintenance the trigger was probably not being handled.

Just to be clear, there is no need for concern. If the TKLBAM backup completed successfully then the backup itself was successfull. The Hub's cache is just probably out of date. Once the issue resolved all backup sessions will be displayed.

Sorry for the late reply and reporting the issue!

Alon Swartz's picture

Flushed the queue...

I just flushed the trigger queue so all backups should be updated. If your backups aren't updated, please send feedback via the Hub and I'll manually trigger the cache update for your account.

[correction] the queue is still being processed, so give it a couple of minutes before reporting out-of-sync backup accounts.

Jeremy's picture

Thanks Alon

I forgot to post back that I successfully restored a backup that was not listed correctly on the Hub. It was a fresh backup that I did and the Hub reported (actually still reports) "First backup in progress..." with a filesize of 0 bytes. But i successfuly restored it (migrated it) to a new appliance - no worries.

Actually there were a few things I found during this process, but they are irrelevant to this issue and I have lodged (non-urgent) bug reports.

Waiting for Update

Hi everyone!

You're doing a Great Job!! Congratulations.

But, until now, no updates.

I made 3 backups today, thinking I missing somthing. Until to search the forum for support.

And found this.

When the Hub updates, I'll feedback.

Thanks.

[Adding Info]

[desenv@desenv:~] sudo tklbam list
[sudo] password for desenv: 
# ID  SKPP  Created     Updated     Size (MB)  Label
   3  No    2011-06-03  2011-10-28  524.42     DeepLook
[desenv@desenv:~] sudo tklbam-status
TKLBAM:  Backup ID #3, Updated Fri 2011-11-11 09:41

Updating...

Now the HUB update...


88.6 KB (inc)
Fri, 11 Nov. 2011 11:11:54 UTC
368.7 KB (inc)
Fri, 11 Nov. 2011 11:11:30 UTC
256.1 MB (full)
Fri, 11 Nov. 2011 11:11:21 UTC

Thanks.

L. Arnold's picture

Confirm also, Hub sessions now showing.. Now how to Optimize

Thank you Alon for performing your magic! 

Jeremy, Alon and Team.  So if we want to 'shorten" the number of links can we do that at any point?

More to the question though, how do we "stop" taking Backups to one Backup id, and start issuing to a new id?  Or even better, how might we toggle to a different Backup id, in the case for instance of a Quarter or Year End Closing that we want to have access to but not load data onto on a regular basis?

Backups are great, but whether via TKLBAM or, with a program like Retrospect, keeping track of it all is quite a chore and I am sure the back end data mapping is absolutely Byzantine.

Jeremy's picture

To shorten the chain you just take more regular full backups

But then you end up storing more data. Both methods have advantages and disadvantages, so like most things in life, there's always a trade off. Full backups take up more storage space and take longer to perform. Incremental backups are much faster to perform but are somewhat vulnerable in as much as if one gets corrupted, or goes missing then the chain back to the full backup is lost.

My personal take on that (as I said in my post above) is to make full backups more frequently thus reducing the number of incrementals between them. You don't need to do anything more than that. You don't need to start taking backups to a new ID to reduce the chain length.

But if I get you correctly, you want to 'start again' without all of the old data, but most of the config stuff? That should be doable, but will involve a bit of work. Creating a new ID is as simple as migrating your data to a new server and doing a full backup - a new ID is automatically created. But if you wished to purge all old info, then you would have to manually do that prior to running your first backup. If you document the steps you take, you may be able turn this into a script. So this is the workflow I would suggest to acheive what (I think) you're after:

  1. Run a full backup of Server A (your current primary appliance).
  2. Restore the backup to Server B (your new appliance).
  3. Manually remove all data that is no longer needed. Document this step carefully.
  4. Run a full backup of Server B (which now your new current primary appliance) - the new backup ID is automatically created.
  5. As a final step I would recommend changing the name of the old backup so it is clear what it is (eg add the financial year that it applies to).

Fixed too.

Thank you.

All good for me now as well.

All good for me now as well. Thanks!


Hub Backup Status Page not updating again ...

Yesterday, my backup status page updated (sort-of: all of my backups showed the same time elapsed ... even though they had been made hours apart).  However, this morning (Sat, 12 Nov), it is again not updating.  I checked to ensure that various backups had been made from my servers that are not being reflected on my status page.

Jeremy's picture

Me too

I made 2 backups yesterday (my time - +11 GMT) within about an hour of one another.

The first one, the Hub states occured "Fri, 11 Nov. 2011 02:11:36 UTC"

The second that occurred within an hour (or perhaps 2 at the outside) still says "First backup in progress..." but it took about 2 minutes!

TBH I'm not too fussed personally because I know that the data is there and I have confirmed that backups that the Hub states haven't completed restore fine. But still I could imagine that it may be a little unnerving for some!

Alon Swartz's picture

Queue got stuck, again...

The queue got "stuck" again, argh! I just "flushed" the queue and its running all the updates as I write this. We are still looking into what is causing this, and will issue a fix ASAP. 

[update]: I'm almost certain the issue has been solved for good. The backlog of tasks are running so it should take a couple of minutes to get through all of them. If anyone still has an issue later in the day please let me know...

Queue stuck again??

Good Day, I seem to be experiencing the exact same problem.  I completed a full tklbam with a VM instance on my server, it completed succesfully (shown below)

Backup ID #3, Updated Sat 2013-07-20 06:37

However when I go to Hub it shows first backup in progress with 0 bytes xfer'd. A reload of the webpage does not resolve the issue. I am using an older instance of Mindtouch Deki VM from 2011 this is the first time I have attempted an S3 Amazon backup as I want to migrate the VM to the cloud in Hub.  Plse help.

Jeremy's picture

Have you tries from within the new appliance?

IIRC correctly it was only the Hub that was stuck, not the actual backups themselves... If you have a look at your available backups from the appliance itself (either within Webmin or from the commandline) hopefully the real situation should be apparent...

Also you could inform the devs via the Hub feedback (blue button on the left hand side when logged into the Hub).

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)