Running Kathara inside of a Docker container

[pg]

In our use-case, we have developed an API (which runs inside a container) with some endpoints whose outcomes include building and modifying a Kathara Lab. We would like the Kathara-managed containers to exist at the root level of our host, not within the API Docker container. Is there a recommended and/or supported way to run Kathara inside of a Docker container?

[Skazza94]

Hi @pg,
thanks for opening this discussion and for using Kathará.

As far as I understand, what you have is a container with some APIs, and you want to run a Kathará scenario on the host instead of inside the container. Am I right?

Mariano.

[pg]

Yes, that is correct. We want API requests to set up a Kathara lab, but we don't want the Kathara-managed containers to live inside the API container (a Docker-in-Docker scenario). Instead, we would prefer the Kathara containers to run on the host at the same level as the API containers.

[Skazza94]

Yes, it is possible to do so.

You have to connect the host Docker daemon inside the API container.

We're still working on supporting it by releasing an official image, but you can build the image on your own.
The following zip file allows to build a Docker image with Docker and Kathará inside:
DockerHost.zip

Surely you have to add the commands of your API Dockerfile to this one, but after building it (e.g. with a hostdocker tag), you can launch a new container with this image and mount the host Docker socket inside the container. Through this socket, Kathará will create the containers on the host. You also need to run the container with --privileged flag to make internal Docker work.

The following command is an example for launching an interactive container (running /bin/bash) with the image, but you can change it as you wish:

docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker.pid:/var/run/docker.pid --privileged -ti hostdocker /bin/bash

The important part is the -v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker.pid:/var/run/docker.pid --privileged

The Dockerfile installs both the Kathará binary, and the PyPI package in a Python3.9 venv (located in /root/venv if you need it).

Hope this is useful!
Mariano.

[pg]

Thanks! This sounds like it should work out well for us. We will give it a try.

[salunkhs]

@Skazza94 :
Thank you !! As recommended , we consolidated our docker file with yours and now we are able to run Kathara managed containers on the host at the same level as API container.

Initially I ran into below error.
docker.errors.APIError: 409 Client Error for http+docker://localhost/v1.41/plugins/kathara/katharanp:amd64/disable?force=False: Conflict ("plugin kathara/katharanp:amd64 is in use")

which is thrown at below line when lab is deployed.
Kathara.get_instance().deploy_lab(lab)

I resolved the error by adding below lines in docker entrypoint.

But same error is appearing when i run the code outside of the docker container.
Any suggestions ?

[Skazza94]

Hi @salunkhs, sorry for the late reply.
Since you mounted the host Docker daemon, you are also sharing the network plugin. From what I can understand, it seems like the internal plugin is not using the xtables.lock file (since it uses iptables with nftables), while the host requires to mount the xtables.lock file (probably you are using the iptables-legacy).

Kathará tries to change the plugin configuration to use/or not use the xtables.lock. So, the plugin is deactivated and reconfigured.
What happens is that this is done in the container while the plugin is used on the host, and viceversa.

So, first check if you are using iptables-legacy on the host, by checking the iptables --version output. If so, switch to iptables-legacy in the container by adding those 2 lines in the Dockerfile after RUN chmod +x /usr/local/bin/rundocker:

[...]
RUN chmod +x /usr/local/bin/rundocker 

RUN update-alternatives --set iptables /usr/sbin/iptables-legacy
RUN update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

[...]

After that, when launching the container, also bind-mount the /var/run/xtables.lock file of the host:

docker run -v /var/run/xtables.lock:/var/run/xtables.lock -v /var/run/docker.sock:/var/run/docker.sock -v /var/run/docker.pid:/var/run/docker.pid --privileged -ti hostdocker /bin/bash

This should solve the problem. If not, just reply here so we can further debug it 😄.

Mariano.

[salunkhs]

Thank you @Skazza94