TurnKey Linux Virtual Appliance Library

Appliance upgrade path?

Hi,

What is the typical migration path for TKL Appliance users?

Concretely, I'm looking to run Redmine on EC2 and am considering TKL Redmine appliance, which is now over 12 months old.  If I install it today and want to upgrade later, what will I have to do?  Is there some magic involved, or does one have to somehow export data out of Redmine and import it into a new Remine instance on the new server running new TKL Redmine appliance?

Thank you,
Otis

Alon Swartz's picture

TKLBAM - TurnKey Backup and Migration

Instead of re-hashing the TKLBAM documentation, I'm just going to provide some links. If you have further questions, feel free to ask.

Appliance upgrade without TKLBAM?

Hi Alon,

This is an old thread, but I think I should rephrase my question.  Yes, I do see TKLBAM lets one backup an existing appliance into S3 and upgrade from there, but I think I'm really more asking about something along the lines of:

If I have version V1 of appliance A, then when V2 of this appliance comes out, how can I install V2 right along side V1 and get data from V1 to V2 locally - without S3?

Basically, something along the lines of http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade.  I'm assuming once you have a TKL appliance you don't want to do the upgrade of the app itself by yourself, as described on that RedmineUpgrade page or else you won't be able to cleanly upgrade to V2?

 

The reason I'm asking is because TKL looks like something that simplifies my life.  But if I have to now make appliance upgrades more complicated (and I'm assuming involving S3 adds another moving part that I currently don't use/want), then I'm not simplifying my life any more.

 

Thanks,
Otis

Liraz Siri's picture

You don't have to use S3

You don't strictly have to use S3 for backup storage, it's just usually easier that way. See TKLBAM Faq > General > Where can I use TKLBAM?

Note that you'll have to sign up for S3 because that's part of the process of signing up for TKLBAM, but S3 is charged by the gigabyte. If you don't use it, you won't pay anything.

Yeah, but there is EBS!

Thanks for the pointer, Liraz.

But what about EBS?  That is, EBS looks like local disk, so the following comment about using local storage (vs. S3) is kind of misleading, no?

 

"The disadvantage is that you won't be able to restore/test your backup in the cloud, or from a VM running in another office branch (for example). Also keep in mind that a physical hard disk, even a RAID array, provides much much lower data reliability compared with Amazon S3."

 

That is, with EBS you get the benefit of "local-looking" storage, with reliability of S3 and advantages of the cloud.  The only disadvantage of EBS I can think of is that an EBS volume can be attached to only 1 EC2 instance at a time, so if you want to upgrade and thus make a backup, you can't just attach that same EBS volume to a new EC2 instance (the one that will host the new version of the TKL appliance) without first detaching the EBS volume from the first/old EC2 instance.

 

Otis

Liraz Siri's picture

S3 vs EBS: we use S3 by default for a good reason

I have to admit I never considered using EBS as "local storage" with TKLBAM, but I don't see any reason why it wouldn't work.

I still don't think it's a good idea though. At least not in typical usage. Besides the limitation that prevents you from accessing an EBS drive from more than one instance, there are a few more advantages to using S3 which you may not have fully considered.

Pros:

  • Data reliability: S3 has much higher data reliability than EBS. EBS drives do occasionally fail, with catastrophic consequences. They're not designed for fault tolerant backups. Google it.
  • More cost efficient: With S3, you don't pay for empty space. With EBS you pay even for the storage space you aren't using as you have to pre-allocate the drive.
  • Flexible and scalable: with S3 you don't need to preallocate or reallocate storage. It grows and shrinks according to your needs.
  • Access from other regions/zones within EC2: S3 storage is accessible across availability zones and regions. EBS drives don't even work across availability zones in the same region.
  • Access from outside Amazon EC2: S3 storage is accessible from outside Amazon EC2, so you can restore an S3 backup to a local machine.
  • Better usability: When you use S3, TKLBAM usability is streamlined.

Cons:

  • IO speed: S3 is slower than EBS (10MB/s vs 36MB/s). Last time I tested this. This may not be true for larger instances though as they have a fatter network connection.

If you're already using Amazon EC2, I can't think of too many good reasons to store the backup on EBS-backed "local storage" rather than to S3.

Am I missing anything?

S3 vs. EBS - a few more points

Hi Liraz,

Thanks for your patient and thorough response.  I think almost everything you wrote makes sense.  I can think of only a few more things/questions:

  • If you have a critical Appliance running in EC2, where does it store the data?  If on the local/ephemeral disk, that's risky (even with daily backups).
  • Can Appliances store data directly in S3 for reliability reasons?  If yes, doesn't that make things slow?  If not, then one would want to use something else that's reliable, like EBS, in which case one needs EBS anyway.
  • EBS is nice because it's like a normal FS you can browse with all the usual commands (which is nice for us command-line people).  It's been a long time since I looked at S3, but I don't think that's doable with S3, so S3 feels more like a black box.

Otis

Hardy to Lucid migration assistance

Hi -

Just a preface to say that I love these appliances.  Been testing out the restore features of TKLBAM and have one or two issues, though.  I've read the announcements, FAQ's, documentation, etc.  Perhaps I missed the answers.

1.  I have a few of the "old" WP appliances that are Hardy based, and would like to migrate them to Lucid.  The TKLBAM restore pulls all the data to the "new" base WP Lucid appliance, but it's not 10.04 LTS anymore.  It's now 8.04 LTS.  Did I do something wrong?

2.  The other thing is that it didn't migrate settings I had in Webmin, like the server hostname.  So I am leery of what other settings it missed.  Data-wise, though, it is fine.

Thoughts?  Thanks in advance.

Liraz Siri's picture

Answers

1) It shouldn't be possible for TKLBAM to downgrade Ubuntu 10.04 LTS packages to Ubuntu 8.04 LTS package. What makes you think it did? If it looks like it, I suspect this might be a case of an overwritten version label.

2) It's OK to feel a bit uneasy with something new until you test it thoroughly. Testing is always a good idea. For technical reasons, by default TKLBAM profile is configured to ignore /etc/hostname. You can add an override to force it to do so. If you want to understand which files and directories are backed up and which are ignored (by default), see the TKLBAM profile for your appliance. It's in /var/lib/tklbam/profile, but you can also access it through the TKLBAM Webmin module (advanced options > overrides > click default profile link).

Also, consult the backup/restore logs for more information on what actually happens when you restore or backup.

Cost of Hub + TLKBAM

OK, so if I understand things correctly, we have:


* TKLBAM dependency on TKL Hub.

* TKL Hub costing 10% of whatever you are paying for the cloud instance (e.g. if 1 EC2 small instance is $70/month, TKL Hub will cost 10% of that, so $7/month, for a total of $77/month)

* TKLBAM stores backup data in S3, so add the S3 fee to the total cost.  This is probably very inexpensive.

Is this right?

Thanks,
Otis

Liraz Siri's picture

Hub backups independent from Hub EC2 server deployment

I just wanted to point out that TKLBAM works with non Amazon EC2 servers too. The Hub's server deployment feature makes it easier for you to deploy TurnKey on Amazon EC2 but it doesn't force you to use it for backups. If you just want TKLBAM it will work with any TurnKey installation, on your own servers, in VirtualBox, in a VPS, etc.

Sorry for the late reply, we've been swamped pushing out the release.

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)