Using the haproxy charm

The haproxy charm in the charmstore https://jujucharms.com/haproxy/ is deceptively powerful. I recently had a use case which I thought it would not handle. It turns out, it does.

The details are all in the services config value. In my case, I am replacing the apache charm https://jujucharms.com/apache2/ using balancer and reverseproxy relations.

The apache charm has vhost_https_template and vhost_http_template which gets pasted in as apache httpd config. The haproxy charm services config value has service_options as yaml which works much the same.

In my case, port 80 redirects to 443, so I start with this in yaml:

- service_name: haproxy_service
  service_host: "0.0.0.0"
  service_port: 80
  server_options: maxconn 100 cookie S{i} check
  service_options:
      - 'redirect scheme https code 301 if !{ ssl_fc }'

You’ll notice this is almost identical to the default value for the services config:

- service_name: haproxy_service 
  service_host: "0.0.0.0" 
  service_port: 80 
  service_options: [balance leastconn, cookie SRVNAME insert] 
  server_options: maxconn 100 cookie S{i} check

I only changed the service_options entry with what would redirect haproxy in a frontend config.

This is where the magic of this haproxy charm happens. The haproxy charm knows which config values are for a frontend haproxy section and also for a backend haproxy section. The charm automatically puts the value in the right place.

The next thing which wasn’t obvious to me from reading the haproxy charm readme is that the Juju application related using reverseproxy relation becomes a backend section and its values will be merged from the services config.

e.g.

$ juju add-relation my-app haproxy:reverseproxy
$ juju add-relation kibana haproxy:reverseproxy

I can use defaults, or I can make some tweaks to the my-app and kibana applications.

For my use case, I was using apache httpd config RewriteRule ^/?KIBANA/(.*)$ balancer://kibana/$1 [P,L]

The equivalent in haproxy config looks like this:

    acl path_kibana path -m beg  /KIBANA/
    use_backend kibana if path_kibana

and in the kibana backend:

   reqirep  ^([^\ :]*)\ /KIBANA/(.*)     \1\ /\2

The haproxy charm allows all of this to be configured using services config. The related application is automatically set as a service name and your config must match it via service_name yaml.

I use juju2’s juju config command to set the config directly from a yaml file. If you are using juju1 you’ll need to use juju set command. juju config haproxy services=@haproxy-config-services.yaml

- service_name: my-app
  service_host: "0.0.0.0"
  service_port: 443
  crts: [DEFAULT]
  service_options:
      - balance leastconn
      - reqadd X-Forwarded-Proto:\ https
      - acl path_kibana path -m beg  /KIBANA/
      - use_backend kibana if path_kibana
  server_options: maxconn 100 cookie S{i} check
- service_name: kibana
  service_options:
      - balance leastconn
      - reqirep  ^([^\ :]*)\ /KIBANA/(.*)     \1\ /\2
      - rspirep ^Location:\ https?://[^/]+/(.*) Location:\ /KIBANA/\1
      - rspirep ^(Set-Cookie:.*)\ Path=(.*) \1\ Path=/KIBANA/\2
  server_options: maxconn 100 cookie S{i} check

The implication here is that there is another application, “my-app” which is also related to haproxy. This config tells haproxy to use my-app as the default application, but if the url starts with /KIBANA/, to use the kibana backend instead of the “my-app” backend. For completeness, I am including the equivalant of apache’s ProxyPassReverse and ProxyPassReverseCookiePath. These are the rspirep… Location and rspirep…Set-Cookie lines in the config respectively.

I Welcome Parse Developers to Juju

Hello Parse developers,

I was curious how easy it would be to get the published parse-server-example to run with Juju. The end result is that there is a new juju charm named parse-server available.*

Deploying parse-server is as easy as running these commands in a bootstrapped juju environment. This means that it can run ANYWHERE.

juju deploy cs:~evarlast/trusty/parse-server-0
juju deploy mongodb
juju add-relation parse-server mongodb
juju expose parse-server

You’ll then be able to use the http api at port 1337.

For example:

curl -X POST -H “X-Parse-Application-Id: myAppId” -H “Content-Type: application/json” -d ‘{“whatever”:”data”}’ 10.0.3.247:1337/parse/functions/hello

If you wish to take a look at this charm, its in the charmed branch of my fork of the parse-server-example. I do not recommend using this charm as an example of writing a good production charm. This is an example of a quick and dirty hack of a charm which happens to work.

Part of what makes Juju awesome is the magic of application modeling. While my hack of a parse-server charm isn’t production ready, it is building on a very production ready mongodb charm, which can be scaled out and made HA very easily. Charms are reusable open source ops. The mongodb ops have been captured in the mongodb charm. Any required parse-server ops need to be capture in a parse-server charm. The only ones captured so far are configuring the mongodb relation. While its a hack of a demo charm, it is a start.

 

—-

* The real reason is that I have cloud envy and I saw the azure release at https://azure.microsoft.com/en-us/blog/azure-welcomes-parse-developers/ and I thought to myself, gee that is a lot of clicks, seems like there is a better way.

Converting eth0 to br0 and getting all your LXC or LXD onto your LAN

Wayne has a great post on the new juju lxd work. I’ve been using it a bit and it is awesome. It is super fast and I can create and destroy environments faster than creating and destroying with juju-local.

One thing which I’ve done which has made all LXC and LXD instances more valuable to me, in my home development environment, is to use a bridge to put them directly on my home LAN.

Normally, LXC creates its own device, lxc-br0, which is managed by the lxc-net service. The service creates the device, brings it up, manages the dnsmasq tied to it (which provides DHCP for the 10.0.3.0/24 range).

Bridge your interface

Instead of using lxc-br0, I create a br0. I add my eth0 (and in my case other devices) to that br0. Then I configure LXC and LXD to use br0 instead of lxc-br0. I go as far as stopping the lxc-net service, since I’m not using it.

There is one trick if you are going to do that on a remote home system, e.g. I have an old laptop I leave in the basement and I’m really lazy and I don’t want to walk down there and use its console when I screw up its networking. The trick is to make sure eth0 comes up on br0 when its added there.

Before you do anything, make sure bridge-utils is installed. It probably is if you are already using lxc, but if this is a fresh install, you’ll want to apt-get install bridge-utils

Edit your /etc/network/interfaces and disable eth0 by setting it to manual. Add it to br0 by adding a new br0 section and listing eth0 in bridge-ifaces and bridge-ports.

auto br0
iface br0 inet dhcp
    bridge-ifaces eth0
    bridge-ports eth0
    up ifconfig eth0 up

iface eth0 inet manual

Now run sudo ifup br0. At this point something magical happens, the DHCP lease is renewed but this time the IP address is bound to br0. The magical part is that br0 used the eth0 MAC to make the DHCP request and so you get the same IP address in response and even your SSH session stays open. YAY!

wikipedia network bridge

LXC can use any bridge

Now configure LXC to use this bridge.

apt-get install lxc
sed -i 's/lxc.network.link = lxcbr0/lxc.network.link = br0/' /etc/lxc/default.conf

TADA, now any LXC containers you start with lxc-start will use br0 and get an address from your household DHCP server. They will be accessible from any host in your home.

Now what about LXD?

LXD can use any bridge

It turns out, while LXD is a layer on top of LXC, it doesn’t use /etc/lxc/default.conf for its default config, but instead uses its own settings. These are editable with lxc profile edit default. Change the lxcbr0 in your editor and save and exit. You can check that it is correct by using lxc profile show default.

There you have it. LXD instances starting on your local LAN.

Now go read Wayne’s post again and use the Juju LXD provider.

Juju makes getting to http2 easier

I stumbled upon https://launchpad.net/~ondrej/+archive/ubuntu/apache2 when looking at http2 options for apache and it seemed to me that it might be easy to add support for http2 on any ubuntu system.

I started by forking the apache2 charm, because I just want to play around.

You can try it out:

juju bootstrap
juju deploy cs:~evarlast/trusty/apache2 –to 0
juju set apache2 ssl_cert=SELFSIGNED
juju set apache2 enable_modules=ssl
juju run –service apache2 ‘a2ensite default-ssl’
juju run –service apache2 ‘service apache2 reload’

Now run `juju status` to get the IP address of the node and try it out.

https://10.0.3.1/

Open your browsers dev tools and confirm that http2 was used.

Faster mongodb deploys with Juju

I made a change to the official mongodb charm.

Before the change, the install step, when deploying the charm took 5 minutes.

2014-11-17 20:54:13 Installing mongodb
2014-11-17 21:04:12 Entering config_changed

After the change, the install step, when deploying took a tiny bit over 2 minutes.

2014-11-19 19:14:55 Installing mongodb
2014-11-19 19:16:06 Entering config_changed

Before: Fetched 46.2 MB
After: Fetched 14.6 MB

Why the huge change? The charm was previously running ‘apt-get install mongodb’ which Recommends the mongodb-dev package which Depends on development packages, ultimately pulling in a c++ compiler and boost dev libraries and header files.

I am happy to download 1/4 of what I previously was required to download for the same functionality.

When writing charms for juju, or automating your deploys using any other tool, remember consider what the system is really doing with the commands you give it.

I highly recommend always running apt-get install with the –no-install-recommends option when you are in a server environment. You’ll waste less time.

A note on the time comparison: At first I hesitated to write this, because the hardware on which I tested this is doing other things. I then realized that this is exactly what all hardware in the cloud is doing all the time. This isn’t a benchmark. This isn’t a timing test. This is an example of making a single case of something slow, a bit faster.