Appliance upgrade path?

Hi Liraz,

Thanks for your patient and thorough response.  I think almost everything you wrote makes sense.  I can think of only a few more things/questions:

• If you have a critical Appliance running in EC2, where does it store the data?  If on the local/ephemeral disk, that's risky (even with daily backups).
• Can Appliances store data directly in S3 for reliability reasons?  If yes, doesn't that make things slow?  If not, then one would want to use something else that's reliable, like EBS, in which case one needs EBS anyway.
• EBS is nice because it's like a normal FS you can browse with all the usual commands (which is nice for us command-line people).  It's been a long time since I looked at S3, but I don't think that's doable with S3, so S3 feels more like a black box.

Otis

I have to admit I never considered using EBS as "local storage" with TKLBAM, but I don't see any reason why it wouldn't work.

I still don't think it's a good idea though. At least not in typical usage. Besides the limitation that prevents you from accessing an EBS drive from more than one instance, there are a few more advantages to using S3 which you may not have fully considered.

Pros:

• Data reliability: S3 has much higher data reliability than EBS. EBS drives do occasionally fail, with catastrophic consequences. They're not designed for fault tolerant backups. Google it.
• More cost efficient: With S3, you don't pay for empty space. With EBS you pay even for the storage space you aren't using as you have to pre-allocate the drive.
• Flexible and scalable: with S3 you don't need to preallocate or reallocate storage. It grows and shrinks according to your needs.
• Access from other regions/zones within EC2: S3 storage is accessible across availability zones and regions. EBS drives don't even work across availability zones in the same region.
• Access from outside Amazon EC2: S3 storage is accessible from outside Amazon EC2, so you can restore an S3 backup to a local machine.
• Better usability: When you use S3, TKLBAM usability is streamlined.

Cons:

• IO speed: S3 is slower than EBS (10MB/s vs 36MB/s). Last time I tested this. This may not be true for larger instances though as they have a fatter network connection.

If you're already using Amazon EC2, I can't think of too many good reasons to store the backup on EBS-backed "local storage" rather than to S3.

Am I missing anything?

Thanks for the pointer, Liraz.

But what about EBS?  That is, EBS looks like local disk, so the following comment about using local storage (vs. S3) is kind of misleading, no?

"The disadvantage is that you won't be able to restore/test your backup in the cloud, or from a VM running in another office branch (for example). Also keep in mind that a physical hard disk, even a RAID array, provides much much lower data reliability compared with Amazon S3."

That is, with EBS you get the benefit of "local-looking" storage, with reliability of S3 and advantages of the cloud.  The only disadvantage of EBS I can think of is that an EBS volume can be attached to only 1 EC2 instance at a time, so if you want to upgrade and thus make a backup, you can't just attach that same EBS volume to a new EC2 instance (the one that will host the new version of the TKL appliance) without first detaching the EBS volume from the first/old EC2 instance.

Otis

You don't strictly have to use S3 for backup storage, it's just usually easier that way. See TKLBAM Faq > General > Where can I use TKLBAM?

Note that you'll have to sign up for S3 because that's part of the process of signing up for TKLBAM, but S3 is charged by the gigabyte. If you don't use it, you won't pay anything.

Hi Alon,

This is an old thread, but I think I should rephrase my question.  Yes, I do see TKLBAM lets one backup an existing appliance into S3 and upgrade from there, but I think I'm really more asking about something along the lines of:

If I have version V1 of appliance A, then when V2 of this appliance comes out, how can I install V2 right along side V1 and get data from V1 to V2 locally - without S3?

Basically, something along the lines of http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade.  I'm assuming once you have a TKL appliance you don't want to do the upgrade of the app itself by yourself, as described on that RedmineUpgrade page or else you won't be able to cleanly upgrade to V2?

The reason I'm asking is because TKL looks like something that simplifies my life.  But if I have to now make appliance upgrades more complicated (and I'm assuming involving S3 adds another moving part that I currently don't use/want), then I'm not simplifying my life any more.

Thanks,
Otis

Instead of re-hashing the TKLBAM documentation, I'm just going to provide some links. If you have further questions, feel free to ask.

• TKLBAM announcement (includes screencast).
• TKLBAM documentation.
• TKLBAM depends on having a TurnKey Hub account.

Sorry for the late reply, we've been swamped pushing out the release.

OK, so if I understand things correctly, we have:

* TKLBAM dependency on TKL Hub.

* TKL Hub costing 10% of whatever you are paying for the cloud instance (e.g. if 1 EC2 small instance is $70/month, TKL Hub will cost 10% of that, so $7/month, for a total of $77/month)

* TKLBAM stores backup data in S3, so add the S3 fee to the total cost.  This is probably very inexpensive.

Is this right?

Thanks,
Otis