We leveraged cloud-init and extended an already existing simple management system to allow passing user-data to EC2 instances and OpenStack Nova instances. The use of ephemeral instances proved very powerful and influenced our thinking greatly. We came up with great solutions using these very simple techniques.

Even before I left that job, I longed for an easy way to do the same thing for myself. I played a bit with the AWS CLI tool (the newer python boto based tool) and yes, aws ec2 run-instances –user-data works. I always longed to get the same thing on my home server and on my laptop.

Finally, I figured out how to do this with LXC. Its simple yes, but I finally learned how to do what I want.

tl;dr example:

lxc-create -n crisp-Hadley -t ubuntu-cloud -- -r trusty -S ~/.ssh/id_rsa-.pub -u one.yaml

cat > one.yaml
#cloud-config
output:
 all: "|tee -a /tmp/cloud.out"

bootcmd:
 - rm -f /etc/dpkg/dpkg.cfg.d/multiarch
 - for i in 1 2 3 4 5 ; do curl -s http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add - && break ; sleep 2 ; done
apt_sources:
 - source: deb http://packages.elasticsearch.org/logstash/1.4/debian stable main
 - source: ppa:evarlast/experimental
packages:
 - mongodb
final_message: "The system is finally up, after $UPTIME seconds"
runcmd:
 - service myapp start

Background

Most LXC tutorials that I’ve seen walk the user through using the download template. The download template is not bad for new users, but I want something more powerful. It turns out there are a number of templates available by default in /usr/share/lxc/templates and you can even create your own.

The template I am interested in is the ubuntu-cloud template. These lxc templates are not so much templates at all as they are scripts. Some of them use other scripts called hooks defined in /usr/share/lxc/hooks. The ubuntu-cloud template, defined in /usr/share/lxc/templates/lxc-ubuntu-cloud.

Usage

The help for templates is a little hidden and lxc is a little stupid at letting you view the help. You COULD run lxc-create, use the — to pass options to the template and use -h. That has the unfortunate side effect of creating the container anyway. You’d have to lxc-destroy it even though you only used -h. Instead, it is easier to invoke the template directly and get help.

$ /usr/share/lxc/templates/lxc-ubuntu-cloud -h
LXC Container configuration for Ubuntu Cloud images.

Generic Options
[ -r | --release <release> ]: Release name of container, defaults to host
[ --rootfs <path> ]: Path in which rootfs will be placed
[ -a | --arch ]: Architecture of container, defaults to host architecture
[ -T | --tarball ]: Location of tarball
[ -d | --debug ]: Run with 'set -x' to debug errors
[ -s | --stream]: Use specified stream rather than 'tryreleased'

Additionally, clone hooks can be passed through (ie, --userdata). For those,
see:
 /usr/share/lxc/hooks/ubuntu-cloud-prep --help

Here, we see that if we don’t specify the -r option, it defaults to match the host. I’m running vivid on my host, but I’d really like to stick with trusty inside of containers. The -a is interesting, and I can only guess that it only works where compatible. -a i386 would let me use the i386 cloud image on an amd64 host. I can’t think of any other use where a mixing architecture would work in a container.

But there is nothing here about cloud-init

cloud-init via cloud-prep

The last line of help says clone hooks can be passed through. This is useful and IMO the most important item. Run the help for ubuntu-cloud-prep exactly as suggested.

$ /usr/share/lxc/hooks/ubuntu-cloud-prep --help
Usage: ubuntu-cloud-prep [options] root-dir

  root-dir is the root directory to operate on

  [ -C | --cloud  ]:       do not configure a datasource.  incompatible with
                           options marked '[ds]'
  [ -i | --instance-id]:   instance-id for cloud-init, defaults to random [ds]
  [ -L | --nolocales ]:    Do not copy host's locales into container
  [ -S | --auth-key ]:     ssh public key file for datasource [ds]
  [ -u | --userdata ]:     user-data file for cloud-init [ds]

Options for –userdata and –auth-key. Are those what I think they are? It turns out, yes, they work exactly like choosing a public key and user-data when starting an EC2 or Nova instance.

Putting all this together you can create cloud-config yaml files and specify and ssh key, and starting an LXC is just like starting a public cloud instance.

For example, want a postgresql server running?

$ cat > psql.yaml
#cloud-config
output:
 all: "|tee -a /tmp/cloud.out"
packages:
 - postgresql
runcmd:
 - echo "listen_addresses = '*'" >>/etc/postgresql/9.3/main/postgresql.conf
 - sudo -u postgres createuser -D -R -S myuser
 - sudo -u postgres createdb -E utf8 -O myuser mydb
 - echo host mydb myuser 10.0.0.0/8 trust >> /etc/postgresql/9.3/main/pg_hba.conf
 - service postgresql restart 
^D
$ lxc-create -n mypostgresql -t ubuntu-cloud -- -r trusty -S ~/.ssh/id_rsa.pub -u psql.yaml
$ lxc-start -n mypostgresql
$ lxc-info -n mypostgresql
Name:           mypostgresql
State:          RUNNING
PID:            6899
IP:             10.0.3.32
CPU use:        3.67 seconds
BlkIO use:      168.00 KiB
Memory use:     23.21 MiB
KMem use:       0 bytes
Link:           veth452FOE
 TX bytes:      3.84 KiB
 RX bytes:      20.36 KiB
 Total bytes:   24.20 KiB
$ psql -h 10.0.3.32 -U myuser -d mydb
psql (9.4.2, server 9.3.7)
SSL connection (protocol: TLSv1.2, cipher: DHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

mydb=> \q

One of my favorite things about using the cloud-image like this is that unlike the download images, openssh server is running and listening by default. The user ubuntu has the public key which you provided in its authorized_keys file. Everything is ready to go.

11 lines of config, 373 bytes is not much at all for a running postgresql server.

When I don’t want to use juju, this is my go to option.