Note: you may prefer reading the README.html file in your browser.
Swarmer is an open source project to help people deploying proper configured docker swarm clusters on AWS.
You'll find other and simpler tutorials or github projects to deploy swarm on AWS, but if you don't want your cluster to be exposed on public facing IPs, you'll then have to get your hands dirty on a lot of other things.
This project tries to compile a lot of resources to get a swarm cluster up and running on a private aws cloud.
Swarmer is built on top of the following components:
- Terraform for provisioning the infrastructure
- Packer for building the boxes for various providers (Virtualbox, AWS, Kvm, …)
- Consul for service discovery, DNS
- Docker for application container runtimes, of course
- Vagrant for running the swarm cluster in virtualbox
To use this project you will need at least this list of tools properly installed on your box:
- docker 1.10
- gnu make 4.1
- vagrant 1.8
- virtualbox 5.0
To quickly bootstrap a swarm cluster with vagrant on your local machine, configure the servers.yml file and type the following commands.
$ git clone https://github.com/yanndegat/swarmer
$ cd swarmer
$ vi servers.yml
...
$ vagrant up
==> box: Loading metadata for box 'yanndegat/swarmer'
box: URL: https://atlas.hashicorp.com/yanndegat/swarmer
==> box: Adding box 'yanndegat/swarmer' (v0.0.1) for provider: virtualbox
box: Downloading: https://atlas.hashicorp.com/yanndegat/boxes/swarmer/versions/0.0.1/providers/virtualbox.box
box: Progress: 26% (Rate: 1981k/s, Estimated time remaining: 0:02:24)
...
$ ./setup.sh
try command:
export DOCKER_HOST=swarm-4000.service.swarmer:4000
docker info
or go to http://consul.service.swarmer:8500/ui
$
Go to http://consul.service.swarmer:8500/ui
If you encounter any problem, refer to the next sections.
Refer to the README file.
Now we can play with swarm.
To resolv names, you have to configure a dns service that will forward the requests to consul. A docker-compose file is provided (<dns-proxy-compose.yml>).
This compose file starts a dnsmasq service with net=host that will target $CONSULIP consul agent. This only works with vagrant or when the VPN is setup.
$ # check your /etc/resolv.conf file
$ cat /etc/resolv.conf
nameserver 127.0.0.1
...
$ # eventually run the following command for your next boot
$ sudo su
root $ echo "nameserver 127.0.0.1" > /etc/resolv.conf.head
root $ exit
$ export CONSULIP=192.168.101.101
$ docker-compose -d -f dns-proxy-compose.yml up -d
$ curl registry.service.swarmer:5000/v2/_catalog
{"repositories":[]}
$ ...
You can now use your swarm cluster to run docker containers as simply as you would do to run a container on your local docker engine. All you have to do is target the IP of one of your swarm node.
export DOCKER_HOST=swarm-4000.service.swarmer:4000
docker pull alpine
docker run --rm -it alpine /bin/sh
/ # ...
The private insecure registry which is automatically started on the swarm cluster is registered on the "registry.service.swarmer:5000" name. So you have to tag & push docker images with this name if you want the nodes to be able to download your images.
Examples are available in the examples directory. You can play with them to discover how to work with docker swarm.
- Every component of the system must be able to boot/reboot without having to be provisionned with configuration elements other than via cloud init.
- Every component of the system must be able to discover its pairs and join them
- If a component can't boot properly, it must be considered as dead. Don't try to fix it.
-
a consul cluster setup, which consists of a set of consul agents running in "server" mode, and additionnal nodes running in "agent" mode. The consul cluster could be used :
- as a distributed key/value store
- as a service discovery
- as a dns server
- as a backend for swarm master election
-
a swarm cluster setup, which consists of a set of swarm agents running in "server" mode, and additionnal nodes running in agent mode. Every swarm node will also run a consul agent and a registrator service to declare every running container in consul.
-
an insecure private registry which is started automatically by a random swarm node. It's registered under the dns address registry.service.consul. If this node is down, it will be restarted by another one within a few seconds. On AWS, it is possible to configure the registry's backend to target a S3 bucket.
Some nodes could play both "consul server" and "swarm server" roles to avoid booting too many servers for small cluster setups.
Yes. Because by now, it's the only coreos version that supports docker 1.10.
We may later consider using docker-machine to install & configure the swarm agents. We would then benefit proper & secured configurations.
There are some caveats running the system services as docker containers, even on coreos. The main problem is the process supervision with systemd, as full described in this article. That said, the coreos rocket container engine could be considered as a suitable alternative.
There is no monitoring yet, and no centralized log system configured either.v
It would be nice if the server.yml could be used as input to terraform an AWS setup.