Automating EC2 instance setup with user-data scripts

I recently published ec2metadata which provides a simple CLI and Python interface to the metadata of an Amazon EC2 instance. In that post I mentioned that "one of the most useful pieces of data is user-data, which can be used to pass configuration information or even initialization scripts to the instance upon launch".

I received a couple of responses by email asking for more information, so here it is.

But first, let me just thank Eric Hammond (and RightScale) for the idea. Below is a Python implementation which leverages ec2metadata as the communication layer.

Support for user-data scripts (ec2metadata, 25ec2-userdata) is included in all the latest TurnKey AMI's. If you setup the mechanism as described it should work on your own AMIs too.

Usage

It's simple. If the user-data content starts with the two characters #! (shebang), it will be executed. If no user-data is specified, or does not start with #!, nothing happens.

Launching an instance with user-data

When launching an instance, specify the user-data-file to pass to the instance.
For example:

ec2-run-instances --key KEYPAIR --user-data-file USERDATA_FILE ami-cbc12fa2

The above example uses the Amazon EC2 CLI tools, but it's possible to pass user-data using the AWS console as well. The AMI specified is that of TurnKey Core in the US-EAST-1 region, the complete AMI listing is available here.

Example user-data script

#!/bin/bash -ex
install()
{
    apt-get update
    DEBIAN_FRONTEND=noninteractive apt-get -y \
        -o DPkg::Options::=--force-confdef \
        -o DPkg::Options::=--force-confold \
        install $@
}

# installs hello - a highly useful package
install hello

# tell the world what we've done!
echo 'Hello world - I just executed user-data!' > /root/helloworld

The code

The following script depends on ec2metadata, and is executed on firstboot by inithooks on TurnKey AMI's.

/usr/lib/inithooks/firstboot.d/25ec2-userdata

#!/usr/bin/python
# Copyright (c) 2010 Alon Swartz <alon@turnkeylinux.org> - all rights reserved

import os
import shutil
import tempfile

import executil
import ec2metadata

class TempFile(file):
    def __init__(self, prefix='tmp', suffix=''):
        fd, path = tempfile.mkstemp(suffix, prefix)
        os.close(fd)
        self.path = path
        self.pid = os.getpid()
        file.__init__(self, path, "w")

    def __del__(self):
        if self.pid == os.getpid():
            os.remove(self.path)

def main():
    userdata = ec2metadata.get('user-data')

    if userdata and userdata.startswith("#!"):
        fh = TempFile(prefix="ec2userdata")
        fh.writelines(userdata)
        fh.close()

        os.chmod(fh.path, 0750)
        executil.system(fh.path)
        print "# executed ec2 user-data script"

if __name__ == "__main__":
    main()