Load Testing with Docker and Locust - Part 4 :: biscuit.ninja - M7CFW — Ramblings of an IT Infrastructure Engineer

Background

This is the final post in a series where I’m looking at using Locust with Docker to do some load testing against an Apache HTTP Server web server.

Posts in this Series

Part 1 - Creating and Building the Docker Image
Part 2 - Testing the Locust Docker Image
Part 3 - Running Docker Containers With Their Own IP Addresses (macvlan)
Part 4 - Using Docker Compose to Create The Containers

Part 4 - Using Docker Compose to Create The Containers

What is Docker Compose?

Docker Compose allows you to specify, declaratively, a set of containers that work together to perform a task or provide a service. In this specific case, we want to run one Locust master instance and many Locust worker instances that will be coordinated by the master instance. We can define these containers in one Docker Compose file.

We can also describe the macvlan network in that same file. That means, to perform our task, we can simply run docker-compose up and then sit back and relax whilst the network is created and the containers are started, instead of typing the commands that we have used to define the network and start the containers in the earlier parts of this series of posts.

The Docker Compose File

We create a docker-compose.yml file in our working directory, ~/loadtest/. In this file we will define one locust-master and two locust-worker services or containers. We also define the macvlan network that the containers use:

version: "2.4"
services:
  locust-master:
    container_name: locust-master
    hostname: locust-master
    build:
      context: /home/user/loadtest/
      dockerfile: dockerfile
    image: debian:locust
    volumes:
      - "/home/user/loadtest:/locust"
    networks:
      locustnet:
        ipv4_address: 172.16.5.1
    ports:
      - "8089:8089"
      - "5557:5557"
    environment:
      targetHost: "https://somebox.somedomain.sometld"
      locustMode: MASTER

  locust-worker1:
    container_name: locust-worker1
    hostname: locust-worker1
    image: debian:locust
    volumes:
      - "/home/user/loadtest:/locust"
    networks:
      locustnet:
        ipv4_address: 172.16.5.11
    ports:
      - "8089:8089"
    environment:
      targetHost: "https://somebox.somedomain.sometld"
      locustMode: WORKER
      locustMaster: 172.16.5.1

  locust-worker2:
    container_name: locust-worker2
    hostname: locust-worker2
    image: debian:locust
    volumes:
      - "/home/user/loadtest:/locust"
    networks:
      locustnet:
        ipv4_address: 172.16.5.12
    ports:
      - "8089:8089"
    environment:
      targetHost: "https://somebox.somedomain.sometld"
      locustMode: WORKER
      locustMaster: 172.16.5.1

networks:
  locustnet:
    driver: macvlan
    driver_opts:
      parent: eno1
    ipam:
      config:
        - subnet: "172.16.0.0/21"
          gateway: 172.16.0.1
          ip_range: "172.16.5.0/24"

Given the commands we ran and the parameters we supplied to them in parts 2 and 3, most of this files contents can already be explained away.

The locust-master service had an additional port published. This is used for the Locust workers to communicate with the master. We are also setting the locustMode environment variable to be MASTER rather than relying on the default value of standalone.

Conversely, the two locust-worker services have their locustMode environment variable set to WORKER. We also specify the IP address on which the workers can find the locust-master container.

The only other thing to mention is the build parameters we have added to the locust-master service. These parameters mean that Docker Compose knows how it can create our Locust Docker image, using the build file we created earlier if the image does not already exist.

Otherwise, this Docker Compose file contains what we specified on the command line to the original Docker commands back when we were testing the docker image.

Using Docker Compose

From within the ~/loadtest/ folder we can now simply run:

sudo docker-compose up

The output should look something like:

Creating network "loadtest_locustnet" with driver "macvlan"
Creating locust-master   ... done
Creating locust-worker2  ... done
Creating locust-worker1  ... done
Attaching to locust-master, locust-worker2, locust-worker1
locust-master      | [2021-06-28 21:08:32,504] locust-master/INFO/locust.main: Starting web interface at http://0.0.0.0:8089 (accepting connections from all network interfaces)
locust-master      | [2021-06-28 21:08:32,514] locust-master/INFO/locust.main: Starting Locust 1.6.0
locust-master      | [2021-06-28 21:08:32,520] locust-master/INFO/root: Terminal was not a tty. Keyboard input disabled
locust-worker2     | [2021-06-28 21:08:32,733] locust-worker2/INFO/locust.main: Starting Locust 1.6.0
locust-master      | [2021-06-28 21:08:32,742] locust-master/INFO/locust.runners: Client 'locust-worker2_35ba8f2089014273b840224335c5d3a9' reported as ready. Currently 1 clients ready to swarm.
locust-worker1     | [2021-06-28 21:08:32,913] locust-worker1/INFO/locust.main: Starting Locust 1.6.0
locust-master      | [2021-06-28 21:08:32,914] locust-master/INFO/locust.runners: Client 'locust-worker1_23bcb4c7ebaf454ebd811f4144502a25' reported as ready. Currently 2 clients ready to swarm.

Your output maybe a bit longer if, as a result of running docker-compose up, if Docker has had to build an up-to-date version of your Locust image.

If we visit http://172.16.5.1:8089 in a web browser, we should be able to launch a load test. In which case we should see some output along the following lines:

locust-master      | [2021-06-28 21:09:02,767] locust-master/INFO/locust.runners: Sending spawn jobs of 200 users and 2.00 spawn rate to 2 ready clients
locust-worker2     | [2021-06-28 21:09:02,769] locust-worker2/INFO/locust.runners: Spawning 200 users at the rate 2 users/s (0 users already running)...
locust-worker1     | [2021-06-28 21:09:02,782] locust-worker1/INFO/locust.runners: Spawning 200 users at the rate 2 users/s (0 users already running)...
locust-worker2     | [2021-06-28 21:10:42,445] locust-worker2/INFO/locust.runners: All users spawned: MyUser: 200 (200 total running)
locust-worker1     | [2021-06-28 21:10:42,449] locust-worker1/INFO/locust.runners: All users spawned: MyUser: 200 (200 total running)

When we are done, we can find the terminal in which we ran docker-compose up and hit <CTRL>+<C> to stop the services. If we want to clean things up by deleting the network and the containers, we can do so by running:

docker-compose down

Final thing, by way of instruction, it is not always practical to have Docker Compose running our containers in the foreground. To run them in the background we can do:

docker-compose up --detach

docker-compose up -d

Conclusion

This approach does, thus far, seem successful in conducting a load test using Locust with requests hitting a web server from multiple IP addresses. I have been able to verify this using the Apache HTTP Server access logs.

There is still one major shortcoming, which may be down to a lack of knowledge on my part, bad choice of tooling or indeed missing tooling. Individually defining each almost identical Locust worker in the Docker Compose file is far from ideal, if you want more than a handful of them.

In future, I would like to look at solving that problem and not have to do that. Perhaps the answer lies in Docker Swarm, Kubenetes, or an alternative containerisation technology.

I understand some people have solved it by using a separate Docker Compose file and writing a for loop around CLI calls to docker-compose, with some additional parameters passed in to get unique container names etc.. I may employ that technique as a stop-gap measure, but I would ideally prefer a declarative approach that means writing configuration rather than writing code.