The container orchestration tools you need to manage your apps

Debo Ray•November 2, 2022

The container orchestration tools you need to know about #

There are some great container management tools out there. In this article, we talk about a set of tools that are used to manage production apps and workloads in modern tech companies to avoid unnecessary downtime.

But before we get into that, what is the difference between Kubernetes and Docker? And why is a company around remote software development and ephemeral environments writing about container orchestration systems?

Docker Compose vs. Kubernetes #

Kubernetes and Docker are often portrayed as competing tools, but that’s not the case. Kubernetes and docker compose might have some broad similarities but they really solve separate problems. The common unit here is containers, so let’s understand them first.

Containers enable portability and scalability to microservices, apart from bringing a level of standardization (while making them hermetic). Docker is arguably the most popular open-source containerization technology - other examples: BuildKit, LXD, Buildah, etc. Usually, when people talk about Docker, they are talking about Docker Engine, the runtime that allows you to build and run containers. Docker compose uses this underneath to process your Dockerfiles or docker-compose.yml files, and run a bunch of container images on your machine.

When you have a lot of container images stored in Docker Hub or AWS ECR or GCP’s Container Registry (or even your own computer), someone needs to run it on some servers so that it can serve production traffic. While docker-compose can run things on your local computer to understand how a couple of apps can interoperate, the single laptop or machine simply can’t keep up with the use-cases of a production system. The container images solved the problem of packaging and distributing an application, now the application needs to be scaled (auto-scaling, high availability), monitored (logs, metrics) and run. This is where container orchestration systems come in, with the most popular among them being Kubernetes.

The key difference between Docker Compose vs. Kubernetes is that Kubernetes is used to run containers on several real or virtual machines/hosts while Docker Compose can only run containers on a single host machine or node. So, it’s not Kubernetes vs. Docker, but most frequently, Kubernetes AND Docker. And when most people talk about Kubernetes vs. Docker, what they really mean is Kubernetes vs. Docker Swarm - more on this later.

Implications of production configuration systems on local development #

While production workloads are now managed as containers supported by orchestration systems, local development processes are still stuck to servicing the traditional deployment model.

Evolution of how containers are used to manage production workloads (from kubernetes.io)

A majority of production apps and workloads now run on linux (we’ll cover Windows in a separate post) as the baseline operating system, either as docker containers (or something similar) or less commonly, as un-containerized applications, especially as microservices have started to take over. Service discovery (routing, load-balancing, inter-microservice communication, etc) and other environmental configuration in production usually operates differently from how it works on a software developer’s local laptop. To get around these issues, DevOps engineers will oftentimes apply the following stop-gap measures:

replicating production-like ergonomics either within application code (as dev-only config in some YAML file)
some companies that have invested in having clear API contracts can get away with being able to define mock interfaces for downstream dependencies, but we find this to be very infrequent
using docker-compose to run containers of a subset of applications for local development - usually a set of microservices (for multi-container applications) and relevant persistence layers
(quite common) pushing branches to test directly in a dev cluster, or staging (or less frequently, production)
(extremely rare) running minikube to replicate a production kubernetes stack on a single node developer laptop

When an application uses message queues (eg: Amazon SQS), this problem becomes even more complex since message queues can’t really get replicated locally. This problem is similar to having a local database that gets quite disconnected from the production database (eg: Amazon RDS).

Therefore, depending on the scheduling and orchestration of production workloads or the complexity of the system, attempts are made to make the single node (or host) developer laptop behave like the multi-node (or VM) production environment - we try to have enough functionality to ensure that product engineers can have enough confidence in building everyday features. Most of the solutions available today force developers to spend long amounts of time outside of their IDEs, simply because the production environments are not represented well enough for their local development use-cases (here’s a post on how to get the most out of VS Code).

How are applications deployed to production? #

A change or pull request will be merged into the main (formerly known as master) branch after a successful code review. At this point, a set of workflows run to get the change into production (and/or a set of preview environments before production).

Workflow for how code changes get deployed, after being merged

In the final steps of the workflow, tasks will run to take the built artifacts and using a set of CLIs or APIs, call out to the main APIs of the orchestration systems to deploy the workload into the appropriate environment (eg: a kubernetes cluster).

Here’s a detailed post about the various stages of the SDLC.

What technologies are currently being used to manage production workloads? #

There are multiple container scheduling and orchestration systems out there, and we’ll focus on a popular subset: Kubernetes, OpenShift, HashiCorp Nomad, Docker Swarm, Mesos and some proprietary third-party cloud solutions (AWS ECS, AWS Fargate, Azure Container Instances, Google Cloud Run).

We won’t be getting into container runtimes in this post since depending on the orchestration platforms being used underneath, various container runtimes can be supported (Docker, Container Runtime Interface, Busybox, etc) and that’s not super relevant to this article. All these technologies support their own flavors of networking, security, orchestration, scalability and self-healing capabilities.

Kubernetes #

Kubernetes is an open-source, out-of-the-box container orchestration tool. It comes with an excellent scheduler and resource manager for deploying highly available containers more efficiently. It was originally developed at Google (internally, the project was called Borg) and has since been donated to the Cloud Native Computing Foundation (CNCF) and is now open source.

The 2022 GitLab Global DevSecOps Survey showed that by the end of the year, 58% of companies planned to be using Kubernetes.

Components of Kubernetes (from kubernetes.io)

Third-party cloud providers’ container orchestration #

In order to boost cloud adoption, various cloud providers have their own container orchestration systems (apart from the usual managed Kubernetes services). Amazon has Elastic Container Service (ECS) and AWS Fargate, Microsoft’s Azure has Container Instances and Google has Cloud Run.