How to run a sidecar in docker-compose

Context

Sometimes you need to test locally how a service that requires a sidecar pattern, i.e. you want to ship logs from the sidecar to a log aggregator, or you want to run a service mesh sidecar to do the service discovery, or any other use case that requires a sidecar pattern.

In our case we will be using the coredns with consul we built on our last post in order to do the service discovery part for our main service.

network_mode

In order to run a sidecar pattern in docker-compose, we will make use of the service network_mode, which allows us to run a service in the same network namespace as another service.

In our case we will be running the coredns-consul service in the same network namespace as the main service, so that the main service can resolve dns through the sidecar.

docker-compose.yml without sidecar

So let’s take our previous example for docker-compose with consul and coredns, and add the sidecar pattern.

You can also clone the repo and run the example.

by running the following command:

git clone https://github.com/Ilhicas/coredns-consul-docker.git

So now let’s take a look at the docker-compose.yml file:

version: '3.8'
services:
  consul:
    image: consul
    ports:
      - 8500:8500
      - 8600:8600/udp
      - 8301:8301
    command: agent -server -bootstrap-expect=1 -ui -client 0.0.0.0
  coredns:
    image: coredns-consul
    build: .
    ports:
      - 8053:53/udp
    environment:
      CONSUL_ADDRESS: 172.17.0.1
    depends_on:
      - consul

So on this example there are a few things that will be impediments to run the sidecar pattern.

docker-compose.yml with sidecar

First of all we can’t do host port mapping, as the sidecar will be running in the same network namespace as the main service, so that is not compatible with the network_mode service.

So we need to modify our docker-compose.yml to remove the host port mapping and also remove the depends_on, as we will be running coredns not as our main service but as a sidecar.

We also need to add a network_mode to the coredns service, to be on the same network as our yet to be defined main service.

For clarity, we will also be modifying the name of coredns to be sidecar.

version: '3.8'
services:
  consul:
    image: consul
    ports:
      - 8500:8500
      - 8600:8600/udp
      - 8301:8301
    command: agent -server -bootstrap-expect=1 -ui -client 0.0.0.0
  sidecar:
    image: coredns-consul
    build: .
    environment:
      CONSUL_ADDRESS: 172.17.0.1
    network_mode: "service:main"

We can keep all the other services with their host mapped port onto the the bridge network, as they will be running on their own network namespace. In this particular case, it’s only the consul server service.

docker-compose.yml with sidecar and main service

So now we need to add our main service, which will be the one that will be using the sidecar for service discovery.

For simplicity, we will be using a simple service for us that comes with dig installed, but we will be running it as our main service, this way we can test that the sidecar is working as expected.

version: '3.8'
services:
  consul:
    image: consul
    ports:
      - 8500:8500
      - 8600:8600/udp
      - 8301:8301
    command: agent -server -bootstrap-expect=1 -ui -client 0.0.0.0
  sidecar:
    image: coredns-consul
    build: .
    environment:
      CONSUL_ADDRESS: 172.17.0.1
    network_mode: "service:main"
  main:
    image: sequenceiq/alpine-dig
    command: tail -f /dev/null

However on the case of our main service we don’t need to add the network_mode, as this service is the network namespace owner.

Running the example

So now we can run the example with the following command:

docker-compose up -d

Given our dig service is running infinitely, we can run the following command to test that the sidecar is working as expected:

docker-compose exec main dig @127.0.0.1 consul

Which is using the internal loopback interface of the main service, which is the same as the sidecar, and we can see that it is resolving the consul service.

We can also make sure we are not using the default dns server for docker bridges by inspecting the output of our dig command.

...
;; SERVER: 127.0.0.1#53(127.0.0.1)

If we were to rely on the dns provided by docker, by running dig without specifying our dns server address (we could set a default, but for brevity and further detailing on what we are achieving, we will not do that), we would get the following output:

docker-compose exec main dig @127.0.0.1 consul
...
;; SERVER: 127.0.0.11#53(127.0.0.11)

And 127.0.0.11 would be our default docker dns server.

Conclusion

So we have seen how we can run a sidecar pattern in docker-compose, by using the network_mode service, which allows us to run a service in the same network namespace as another service.

This can be useful for testing purposes locally, and allows to easily prototype sidecars.

We could use this to create mesh networks. You can see an example about a presentation I made for consul service mesh.

We could use it for logging sidecars, or any other common practices where sidecar patterns are used.