Load balancing requests across all instances of a service

Before we explore load balancing, we need to have something to balance. We need multiple instances of a service. Since we already explored scaling in the previous chapter, the command should not come as a surprise:

eval $(docker-machine env node-1)

docker service scale go-demo=5

Within a few moments, five instances of the go-demo service will be running:

What should we do to make the proxy load balance requests across all instances? The answer is nothing. No action is necessary on our part. Actually, the question is wrong. The proxy will not load balance requests at all. Docker Swarm networking will. So, let us reformulate the question. What should we do to make the Docker Swarm network load balance requests across all instances? Again, the answer is nothing. No action is necessary on our part.

To understand load balancing, we might want to go back in time and discuss load balancing before Docker networking came into being.

Normally, if we didn't leverage Docker Swarm features, we would have something similar to the following proxy configuration mock-up:

backend go-demo-be
  server instance_1 <INSTANCE_1_IP>:<INSTANCE_1_PORT>
  server instance_2 <INSTANCE_2_IP>:<INSTANCE_2_PORT>
  server instance_3 <INSTANCE_3_IP>:<INSTANCE_3_PORT>
  server instance_4 <INSTANCE_4_IP>:<INSTANCE_4_PORT>
  server instance_5 <INSTANCE_5_IP>:<INSTANCE_5_PORT>

Every time a new instance is added, we would need to add it to the configuration as well. If an instance is removed, we would need to remove it from the configuration. If an instance failed… Well, you get the point. We would need to monitor the state of the cluster and update the proxy configuration whenever a change occurs.

If you read The DevOps 2.0 Toolkit, you probably remember that I advised a combination of Registrator (https://github.com/gliderlabs/registrator), Consul (https://www.consul.io/), and Consul Template (https://github.com/hashicorp/consul-template). Registrator would monitor Docker events and update Consul whenever a container is created or destroyed. With the information stored in Consul, we would use Consul Template to update nginx or HAProxy configuration. There is no need for such a combination anymore. While those tools still provide value, for this particular purpose, there is no need for them.

We are not going to update the proxy every time there is a change inside the cluster, for example, a scaling event. Instead, we are going to update the proxy every time a new service is created. Please note that service updates (Deployment of new releases) do not count as service creation. We create a service once and update it with each new release (among other reasons). So, only a new service requires a change in the proxy configuration.

The reason behind that reasoning is in the fact that load balancing is now part of Docker Swarm networking. Let's do another round of drilling from the util service:

ID=$(docker ps -q --filter label=com.docker.swarm.service.name=util)

docker exec -it $ID apk add --update drill

docker exec -it $ID drill go-demo

The output of the previous command is as follows:

;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 50359
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; go-demo.     IN      A

;; ANSWER SECTION:
go-demo. 600 IN A 10.0.0.8

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 0 msec
;; SERVER: 127.0.0.11
;; WHEN: Thu Sep 1 17:46:09 2016
;; MSG SIZE  rcvd: 48

The IP 10.0.0.8 represents the go-demo service, not an individual instance. When we sent a drill request, Swarm networking performed load balancing (LB) across all of the instances of the service. To be more precise, it performed round-robin LB.

Besides creating a virtual IP for each service, each instance gets its own IP as well. In most cases, there is no need discovering those IPs (or any Docker network endpoint IP) since all we need is a service name, which gets translated to an IP and load balanced in the background.