Launching EC2 images in Amazon EC2

Note: If you came here from the Amazon subscription pipeline, the activation key is not required because just like our other build targets, our Amazon EC2 AMI's are 100% open source with no proprietary restrictions.

Overview

There are many advantages to using Amazon EC2 to deploy TurnKey Linux appliances on-demand:

  • It's fast: launch an appliance into a high performance cloud in just minutes.
  • It's convenient: No additional hardware or software is required.
  • It's cost effective: Less than $0.1/hour (8.5¢) and you only pay for what you use.
  • Nothing to download: this is especially useful with our larger appliances (e.g., Zimbra).
  • Simple: Basic usage can be managed from a web browser.
  • Powerful: can be managed and automated from a powerful API.

If you have not yet subscribed to the appliance you want to deploy on EC2, please do so now by going to the appliance page and clicking on the "launch to cloud -> Amazon EC2 AMI" link. This will take you to a sign-up form on Amazon. If you do not have an Amazon EC2 account one will be created for you. After subscribing you will be redirected from Amazon back to this page.

Pricing

Update: We've decided to let users try out TurnKey on EC2 for free while we solicit more feedback from the community on pricing here. For now you'll only be paying the regular Amazon EC2 usage fees:

Less than $0.1/hour (8.5¢/hour to be precise)

Getting started

EC2 has many powerful features, which can make it confusing for new users.

This tutorial does not assume previous experience with EC2. We'll be demonstrating how to launch TurnKey LAMP appliance on EC2 using Amazon's management console but the principles are the same even if you use a different interface such as the ElasticFox FireFox extension or Amazon's command line API.

If you're a more experienced EC2 user you may want to skip down to the part where we set up EBS volume for data persistence.

Launching an instance

Log into the Amazon web console.

Launch instance wizard

Select the region where you want to launch an instance (US or EU).

Click on the Lauch Instances button which launches the Launch Instance Wizard.

Select the community AMIs tab. Entering "turnkey" into the text box will give you a list of available TurnKey Linux appliances.



To select the AMI you subscribed to, enter its codename:

Create SSH keypair

Amazon will prompt you to create an SSH keypair you can use later to log into your instance.

Firewall security groups

If you haven't set up a firewall security group the wizard will create a new group that only allows port 22.

Note that to access the full range of services available on your appliance instance you'll later have to add additional ports to the security group.

Finalize and launch

Opening additional ports

Go to the security groups panel.

Here you may want to open any of the following ports:

22 SSH/SFTP
80 HTTP
443 SSL
12320 Web shell
12321 Webmin
12322 Database front-end

Accessing a running instance

Go to instances panel. Clicking on the instance shows its public IP address

"Instance actions -> Connect" shows how you can use the keypair to login with SSH.

You can also log in with the initial root password, which is available in the output of the system log (Instance Actions -> System Log). It's long so you may want to change it to something shorter if you're going to be using webmin or the Ajax web shell.

The initial root password (as well as the MySQL:root and PostgreSQL:postgres where applicable) are set randomly on firstboot for security reasons, and is available in /root/README.

Setting up an EBS volume for data persistence

Go to "Volumes" panel

Create a volume. Make sure you create it in exactly the same availability zone as your instance otherwise you won't be able to attach it to the instance.

Attach it to your instance.

The new volume is "hotplugged" to your instance on the fly (it shows up as device /dev/sdf).

First-time formatting

Log into a shell command line and execute the following commands:

mkdir /vol
mkfs.ext3 /dev/sdf
mount /dev/sdf /vol

apt-get update

# (I.e., in TurnKey LAMP this installs ebsmount-scripts-lamp)
# please note that the ebsmount-scripts are highly experimental
apt-get install ebsmount-scripts-$(cat /etc/hostname.orig)

umount /dev/sdf

# starts the ebsmount service
/etc/init.d/ebsmount start

You can now safely terminate your appliance instance and your data will survive.

How it works

When launched the ebsmount service shuts down running services, mounts the EBS volume and then mount --bind a list of directories (e.g., /etc, /var/lib/mysql) on the instance to directories inside the EBS volume. Which filesystem locations are mounted to the EBS volume varies from appliance to appliance which is why each appliance has its own ebsmount hooks (e.g., ebsmount-scripts-lamp).

If you are mounting a newly created EBS volume then ebsmount will first migrate the instance's data to the EBS volume.

Attaching an instance to a previously created EBS volume

Unfortunately Amazon doesn't yet make it easy to start an instance with the EBS volume already attached. This means attaching an instance to an EBS volume via the AWS management console is a bit cumbersome. We're working on tools to automate this.

Meanwhile to remount an EBS volume to a new instance attach the EBS volume (e.g., using the Amazon web management console) and restart the ebsmount service:

/etc/init.d/ebsmount restart

 

Comments

Guest's picture

AWS instantiation fails following above procedure

 Hi.

I purchased your AMI and opened an AWS account today. On the last step of the AWS Request Instances Wizard, I get the error message "Subscription to ProductCode C1BC2EEF required". I cannot find that module in AWS online docs. Is that your AMI?

What should I do to complete the instantiation process?

Guest's picture

follow-up

 OK, C1BC2EEF is the product code for Turnkey's AMI. So the question is: where do I activate my subscription to your product?

liraz's picture

Currently every AMI has a different product code

Did you maybe subscribe to one AMI and then try to create an instance for another one? If not, which AMI did you subscribe to?
Guest's picture

I subscribed to "TurnKey

I subscribed to "TurnKey Drupal6 AMI for Amazon EC2". When I click the link to the app on my Amazon purchase, I land on this page, which does not ask for a product key (even says a key is not needed - on top of this page!).

Guest's picture

product code

 My Amazon purchase states "TurnKey Drupal6 AMI for Amazon EC2", which is what I'm looking for, but maybe not the AMI stated in the tutorial above?

Guest's picture

OK, working now

OK, my problem was that I needed to select the Drupal AMI, not the LAMP AMI which was given as example. duh.

Thanks for the help! 

joey's picture

Subscription to ProductCode A68DA028 required

 I subscribed to the turnkey-joomla product and when I follow the instructions I cannot continue b/c of the subject error above.  I subscribed to the Joomla product and I am choosing the Joomla product in the AMI choices.

Does it take a certain time period (24 hours, etc) before the Product is registered?  I have confirmation of payment in AWS and they have given me a code, but I see no place to enter it in the instructions.

Thank you!

liraz's picture

No it should work immediately

There is no registration period. Once you are subscribed you should be able to use that AMI immediately. Are you absolutely sure you subscribed to the right appliance (e.g., Joomla) and that this is the same AMI you are selecting? Also which region are you using? If it doesn't work I'll try to setup an experiment and reproduce this error myself. Note that Amazon is reporting other successful subscriptions to the Joomla appliance so it's strange that it's failing on your end.

I apologize for the inconvenience on your end. The workflow is still a bit complex, and we are currently working on developing a web application that will make it easier to get up and running.

Guest's picture

Error in execution of ebsmount service

 At the end of first time formatting, mid-way through execution of the "/etc/init.d/ebsmount start" command, I get some errors in mysql-common:

 

(... everything OK until that point ...)
* Starting web server apache2                                           [ OK ]
executing: core
* Restarting OpenBSD Secure Shell server sshd                           [ OK ]

executing: drupal6

executing: mysql-common

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
 * Stopping MySQL database server mysqld                                 [ OK ]
mysqld_safe: no process killed
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
 * Starting MySQL database server mysqld                                 [ OK ]
 * Checking for corrupt, not cleanly closed and upgrade needing tables.
executing: php5
executing: phpmyadmin
executing: postfix
 * Stopping Postfix Mail Transport Agent postfix                         [ OK ]
 * Starting Postfix Mail Transport Agent postfix                         [ OK ]
                                                                         [ OK ]
 
Should I be concerned about the error messages?

 

liraz's picture

Probably not a problem + lets move this to the forum

What these messages mean is that your current directory was probably moved by ebsmount into EBS while you were in it. Next time you might want to run ebsmount from the root directory. I don't think it matters anyhow.

Also, this isn't really the best place to discuss support issues. That's what the support forum is for.

Guest's picture

Recommended use of PostgreSQL

Hi guys

Thanks for the great walkthrough up above.   We're setting up a Postgres database and will probably use your solution

If it isn't easy to get the EC2 instances to automatically attach to the EBS volumes,  what do you recommend as far as operating a PostgreSQL database?      Should we shut it down when we know we aren't going to use it,  then go through the re-attach when we want to use it again?

Also is it possible to use the 150GB that comes with an intance as database storage?   Or is that unreliable/likely to disappear?

Also,  we're having some debate about which OS to use.   One teammate likes Fedora,  another wants OpenSUSE, etcetc.   Any words of wisdom on how you decided (incidentally, what do your images use?) and how well others work on EC2?

Thanks!

 

neil_2's picture

Ubuntu 8.04 LTS

I dont know about EC2, but the TKL images are based on Ubuntu 8.04 LTS.

http://UnmeteredVPS.net/tkl
Try our TKL appliance hosting FREE for 7 days!

liraz's picture

EC2 is pretty reliable in practice but backup anyway

Though the official line at AWS is that you should plan for failure of your instance, and that's probably true at large scales, in practice it seems to be rare nowadays for an EC2 instance to fail. If you Google you'll find people with EC2 uptimes in excess of a year.

It is possible to use the instance storage for your database but please make sure to setup regular backups. Preferably automated. You should do that with any server, regardless of what platform it's running on. An EC2 server is no exception. Even if you use EBS!

BTW we're actually working right now on making all of that simpler. We want it to be easy for users to migrate instances running on local computers into the cloud, to setup regular backups, etc. If you register for an account on the site we'll let you know when we open that up for a public beta.

And Neil is right. All TurnKey images are currently based on the latest Ubuntu LTS (Long Term Support) version 8.04.3. Debian support is planned. Support for other distributions (e.g., CentOS) isn't currently planned but we're not opposed to it in principle, it's just a matter of resources.

neil_2's picture

CentOS?

What would be involved in making CentOS based ports of the appliances?  I use CentOS for all my servers and am very familiar with it so I could lead that effort.

http://UnmeteredVPS.net/tkl
Try our TKL appliance hosting FREE for 7 days!

liraz's picture

Thanks for volunteering but it's a bit premature

  • First, we would need to upgrade our build Debian-based build infrastructure to support RPM packages...
  • Figure out what needs to be done to port over the custom packages we've developed (e.g., di-live, confconsole)
  • Create a CentOS based TurnKey Core
  • Port over all of the configurations and test them
  • Upgrade the web site to support CentOS based builds

In terms of the effort involved it's similar to migrating 40 different servers from Ubuntu Server to CentOS. Not a trivial effort, and currently a bit premature since there are a few prerequisites which we need to work on first. Volunteering to help does make it more likely we'll pursue that though.

In the meantime, let's move this and other non-EC2 related discussions to the forum.

Guest's picture

AMI images ot showing up on Amazon AWS ...

I cant seem to find your images on Amazon AWS. I put 'turnkey' into the filter box under 'All Images' within the 'Community AMIs' tab and nothing shows up. I even tried to enter a few ami ids (ami-xxxxxxx ) I see on your screenshots but nothing is showing up.... Are you still current on Amazon?

alon's picture

AMI's are region specific

I just checked and everything seems OK. My hunch is that you are connected to the new US-WEST-1 region, which does not yet contain that TurnKey images. The images are available in the other 2 regions US-EAST-1 and EU-WEST-1.

I hope this helps.
Guest's picture

AMIs - Your hunch is correct, sir.

Yep. I did connect to the US-WEST-1. I guess it doesn't really matter what region I use for now. Your response and work are much appreciated.

Guest's picture

EBS boot image

Hi,

Is it possible you convert the instances to EBS boot instances (or offer them as an alternative)? I have done one convert myself of my own custom instance and; although not very easy; if you dive into it; you should have it done within a few hours. EBS boot instance are far easier to backup (click of a button). If you need help / advice let me know; there are a few tricky things but if you get a good manual things shouldn't be too complicated.