Tklbam backup fails after EBS change

Hello there guys,

I have this instance running on EC2. It is a Nginx + Varnish + Mysql + PHP5-FPM, build precisely for my needs using Turnkey core as base.

Turns out I had to increase EBS volume size and that is only possible by the snapshot + create new ebs from snapshot. So I created a new ebs with tripe the size, detached the old ebs volume and attached the new one.

Tklbam backup does not work anymore.

Traceback (most recent call last):
File "/usr/bin/tklbam", line 46, in <module>
CliWrapper.main()
File "/usr/lib/tklbam/cliwrapper.py", line 88, in main
commands[command].main()
File "/usr/lib/tklbam/cmd_backup.py", line 385, in main
if not opt_simulate and not opt_disable_resume and registry.backup_resume_conf == conf:
File "/usr/lib/tklbam/registry.py", line 212, in backup_resume_conf
return BackupSessionConf(simplejson.loads(s))
File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 451, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 420, in raw_decode
raise JSONDecodeError("No JSON object could be decoded", s, idx)
simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)

Any suggestions of anything I could try before having to rebuild the whole thing?

Any help is welcome.

Best regards.

Replies (12)

Hmmm

Reply 1 2014-01-21

TBH I have no idea what might be up for you. If you have a very recent TKLBAM backup then you could try restoring that to a new instance (that has the desired size EBS)... I think that should work...

Regardless I have lodged a bug on the TKL Issue Tracker.

Bugger...

Reply 2 2014-01-19

Well TBH I'm not sure what the issue is... When you say latest, is that latest available via the marketplace (AFAIK it's still v12.1) or latest via the Hub (v13.0). If you are using v12.1 then perhaps double check that you are on the latest TKLBAM. I'm not sure if that will fix the issue (possibly not) but at least then we can confirm whether or not it is an issue in the latest version.

No backups

Reply 3 2014-01-17

Thanks Jeremy.

Turns out I dont have any recent backups (last 15 days). I was out on vacation and I did this EBS change just before I left.

I'll rebuild my env using the last turnkey core.

Best regards.

Perhaps just double check that TKLBAM is at the latest version

Reply 4 2014-01-20

If you do:

apt-get update && apt-get install tklbam

And see what happens... If it says something like "tklbam is already at the latest version", then you have it... Otherwise if it updates then try again. TBH I'm doubtful that will resolve your issue (just a gut feeling) but at least we will know whether it not your issue still occurs with TKLBAM v1.4 (the current stable version).

Jeremy, Thanks for your

Reply 5 2014-01-20

Jeremy,

Thanks for your reply.

I am on one of the release candidates that was released last year.

TurnKey Linux 13.0 / Debian 7.2 Wheezy

I has worked flawslessly until now that I had to "resize" the ebs.

Best regards.

No worries

Reply 6 2014-01-20

Sorry I couldn't help more...

Jeremy, Thanks alot but that

Reply 7 2014-01-20

Jeremy,

Thanks alot but that did not work. It says I am up to date.

I'll survive on Snapshots for backup for the next couple of weeks and as soon as I have the time i'll rebuild the env using the latest Turnkey core.

I'll also do a ebs detach > snapshot > new ebs and attach the new one and see if it happens again.

Best regards.

Not a problem. I am not

Reply 8 2014-01-20

Not a problem.

I am not exactly blaming Turnkey on this one, considering I built this vm myself for my needs. Varnish, Nginx and ngx_pagespeed, all build from source. Too bad the pre-build VMS do not fit my needs.

Best regards and have a nice week.

Corrupt backup resume state?

Reply 9 2014-01-20

>From the exception it looks like the file that stores your backup resume state has been corrupted. Could you post the output from the following command please:

cat /var/lib/tklbam/backup-resume

Then try deleting it:

rm /var/lib/tklbam/backup-resume
tklbam-backup

If that still doesn't work, try deleting your TKLBAM registry and reinitializing TKLBAM:

rm -rf /var/lib/tklbam
tklbam-init

Liraz, Thanks alot. After

Reply 10 2014-01-21

Liraz,

Thanks alot. After doing as you pointed out it works again.

Best regards.

Clone TKLBAM fails (but doesn't say so)

Reply 11 2014-02-19

Several hours later TKLBAM has taken over the CPU. Nothing yet to show for Backups on the Hub though WebMin shows the "empty" Storage Containers. I can't even delete the Empty ones on the Hub interface as they don't show up. 91 does show up. I shouldn't have deleted x0 after all. Deleting the Clone next anyway. Will likely try a restore from X3 below (TKLBAM) next. Nice Feedback in Webmin through all this I must say. Webshell drops the TKLBAM session very quickly it seems now.

x1	No	2014-02-18	-	0.01	TurnKey Joomla 2.5
x2	No	2014-02-18	-	0.01	TurnKey Joomla 2.5
x3	No	2014-02-19	2014-02-19	205.83	TurnKey Joomla 2.5
x4	No	2014-02-19	-	0.01	TurnKey Joomla 2.5
x5	No	2014-02-19	-	0.01	TurnKey Joomla 2.5

11170	root	81.0 %	/usr/bin/python /usr/bin/tklbam-backup
2809	mysql	0.3 %	/usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/m ...
11169	root	0.2 %	/usr/bin/python /usr/bin/tklbam-backup

Wrestling with a cloned Install

Reply 12 2014-02-19

In the past when I cloned an ap/install my method was simply to delete the original Backup set in the hub (Not something I wanted to do).

This tip (from Liraz above, of deleting reference to the backup in the Ap - in the new clone) seems a far better path because it allows the original Backup set to stay in the bank and stay attached to the orig. Yesterday I was especially feeling that because when I went to backup the "forked" set neither the old or the new app/install would backup. I must have tried 5 times before finding this post. When they would show up I was getting Backup sets on the Hub of .01 kb.

So yesterday I "killed" the record and thought I was good. Both Apps were ready for Midnight backup. Now here at morning I see the original took (whew). The new was a third (or was it 5th) .01 kb backup on the hub.

I am trying one more... but at least I can try a Tklbam-restore now again (I will stop deleting backup sets on the hub - resolution 99) (if this backup fails though, this will remain a parachute method)

Clones are not a good idea. So many issues that can come from them... Tklbam-restore would seem the better course. I was sure feeling that after I killed my backup set then couldn't get a new one to take. We have one Full (from the original now). Lets see if # 2 completes. Otherwise Delete and Restore.

Clones are just so fast and bandwidth is Zero. Fast if nothing go's wrong. Thanks for a great method (TKLBAM) Liraz! And thanks for the tip here too!