The container orchestration tools you need to manage your apps

Debo Ray

Cover Image for DevZero

The container orchestration tools you need to know about

There are some great container management tools out there. In this article, we talk about a set of tools that are used to manage production apps and workloads in modern tech companies to avoid unnecessary downtime.

But before we get into that, what is the difference between Kubernetes and Docker? And why is a company around remote software development and ephemeral environments writing about container orchestration systems?

Docker Compose vs. Kubernetes

Kubernetes and Docker are often portrayed as competing tools, but that’s not the case. Kubernetes and docker compose might have some broad similarities but they really solve separate problems. The common unit here is containers, so let’s understand them first.

Containers enable portability and scalability to microservices, apart from bringing a level of standardization (while making them hermetic). Docker is arguably the most popular open-source containerization technology - other examples: BuildKit, LXD, Buildah, etc. Usually, when people talk about Docker, they are talking about Docker Engine, the runtime that allows you to build and run containers. Docker compose uses this underneath to process your Dockerfiles or docker-compose.yml files, and run a bunch of container images on your machine.

When you have a lot of container images stored in Docker Hub or AWS ECR or GCP’s Container Registry (or even your own computer), someone needs to run it on some servers so that it can serve production traffic. While docker-compose can run things on your local computer to understand how a couple of apps can interoperate, the single laptop or machine simply can’t keep up with the use-cases of a production system. The container images solved the problem of packaging and distributing an application, now the application needs to be scaled (auto-scaling, high availability), monitored (logs, metrics) and run. This is where container orchestration systems come in, with the most popular among them being Kubernetes.

The key difference between Docker Compose vs. Kubernetes is that Kubernetes is used to run containers on several real or virtual machines/hosts while Docker Compose can only run containers on a single host machine or node. So, it’s not Kubernetes vs. Docker, but most frequently, Kubernetes AND Docker. And when most people talk about Kubernetes vs. Docker, what they really mean is Kubernetes vs. Docker Swarm - more on this later.

Implications of production configuration systems on local development

While production workloads are now managed as containers supported by orchestration systems, local development processes are still stuck to servicing the traditional deployment model.

Evolution of how containers are used to manage production workloadsEvolution of how containers are used to manage production workloads (from kubernetes.io)

A majority of production apps and workloads now run on linux (we’ll cover Windows in a separate post) as the baseline operating system, either as docker containers (or something similar) or less commonly, as un-containerized applications, especially as microservices have started to take over. Service discovery (routing, load-balancing, inter-microservice communication, etc) and other environmental configuration in production usually operates differently from how it works on a software developer’s local laptop. To get around these issues, DevOps engineers will oftentimes apply the following stop-gap measures:

  1. replicating production-like ergonomics either within application code (as dev-only config in some YAML file)
  2. some companies that have invested in having clear API contracts can get away with being able to define mock interfaces for downstream dependencies, but we find this to be very infrequent
  3. using docker-compose to run containers of a subset of applications for local development - usually a set of microservices (for multi-container applications) and relevant persistence layers
  4. (quite common) pushing branches to test directly in a dev cluster, or staging (or less frequently, production)
  5. (extremely rare) running minikube to replicate a production kubernetes stack on a single node developer laptop

When an application uses message queues (eg: Amazon SQS), this problem becomes even more complex since message queues can’t really get replicated locally. This problem is similar to having a local database that gets quite disconnected from the production database (eg: Amazon RDS).

Therefore, depending on the scheduling and orchestration of production workloads or the complexity of the system, attempts are made to make the single node (or host) developer laptop behave like the multi-node (or VM) production environment - we try to have enough functionality to ensure that product engineers can have enough confidence in building everyday features. Most of the solutions available today force developers to spend long amounts of time outside of their IDEs, simply because the production environments are not represented well enough for their local development use-cases (here’s a post on how to get the most out of VS Code).

How are applications deployed to production?

A change or pull request will be merged into the main (formerly known as master) branch after a successful code review. At this point, a set of workflows run to get the change into production (and/or a set of preview environments before production).

Workflow for how code changes get deployed, after being mergedWorkflow for how code changes get deployed, after being merged

In the final steps of the workflow, tasks will run to take the built artifacts and using a set of CLIs or APIs, call out to the main APIs of the orchestration systems to deploy the workload into the appropriate environment (eg: a kubernetes cluster).

Here’s a detailed post about the various stages of the SDLC. 

What technologies are currently being used to manage production workloads?

There are multiple container scheduling and orchestration systems out there, and we’ll focus on a popular subset: Kubernetes, OpenShift, HashiCorp Nomad, Docker Swarm, Mesos and some proprietary third-party cloud solutions (AWS ECS, AWS Fargate, Azure Container Instances, Google Cloud Run).

We won’t be getting into container runtimes in this post since depending on the orchestration platforms being used underneath, various container runtimes can be supported (Docker, Container Runtime Interface, Busybox, etc) and that’s not super relevant to this article. All these technologies support their own flavors of networking, security, orchestration, scalability and self-healing capabilities.

Kubernetes

Kubernetes is an open-source, out-of-the-box container orchestration tool. It comes with an excellent scheduler and resource manager for deploying highly available containers more efficiently. It was originally developed at Google (internally, the project was called Borg) and has since been donated to the Cloud Native Computing Foundation (CNCF) and is now open source.

The 2022 GitLab Global DevSecOps Survey showed that by the end of the year, 58% of companies planned to be using Kubernetes.

Components of KubernetesComponents of Kubernetes (from kubernetes.io)

Third-party cloud providers’ container orchestration

In order to boost cloud adoption, various cloud providers have their own container orchestration systems (apart from the usual managed Kubernetes services). Amazon has Elastic Container Service (ECS) and AWS Fargate, Microsoft’s Azure has Container Instances and Google has Cloud Run.

Technically, the high-level core differences to Kubernetes are:

  • fully managed solution whereas Kubernetes isn’t that (unless you use a well-configured AWS EKS, GCP GKE, Azure AKS, etc solution)
  • tied to one container runtime (usually, docker applications)
  • some of these solutions (ECS, or Fargate which is built on ECS/EKS) are proprietary

OpenShift

OpenShift is a container management system built on top of Kubernetes. While Kubernetes is open-source and various cloud providers have support for managed Kubernetes services (EKS, GKE, AKS, etc), OpenShift is available both as a commercial product (OpenShift Container Platform), as well as public cloud (called OpenShift Online and OpenShift Dedicated).

Technically, the high-level core differences to Kubernetes are:

  • can’t run containers as root
  • while Kubernetes supports plugins, OpenShift supports networking out of the box (Open vSwitch)
  • tied to Red Hat’s proprietary operating systems, Fedora or CentOS

HashiCorp Nomad

Nomad is an orchestration platform from HashiCorp that supports containers and non-containerized workloads. It shares a similar philosophy to Kubernetes in managing applications at scale, although experiments with Nomad have shown that it can scale better with it passing the 2-million container challenge.

Technically, the high-level core differences to Kubernetes are:

  • support for both containerized and non-containerized workloads
  • ease of service discovery, config management using HashiCorp Consul but load balancing, configuration management, routing, feature gates, and service discovery are not supported natively
  • supports both Windows and Linux-based containers

Docker Swarm

Docker Swarm, from a high-level, has interfaces similar to docker-compose, but has the ability to manage containers across multiple nodes.

Technically, the high-level core differences to Kubernetes are:

  • Swarm supports only the docker runtime (and coordination between various instances of docker engine)
  • lack of pluggability in networking, security, etc

Mesos

Mesos is a cluster management tool that can manage container orchestration. It was created by Twitter for its internal infrastructure before being donated to open source.

Technically, the high-level core differences to Kubernetes are:

  • limited networking and storage mounting capabilities (Marathon’s docker integration enables mapping container ports to host ports, a limited resource)
  • lack of pod-concept, which enables better flexibility to managing infrastructure around containers (sidecars, etc)

Conclusion

There are various companies that will adopt various technologies to manage their production workloads. In order to boost developer productivity, it will be imperative to be able to replicate those environments for development. Trying to mimic the production configuration for local development is a losing battle and the industry at large, within reason, is moving to build pipelines to ensure development environments closely track production dependencies.

Due to the growing pattern of coding locally, then performing the “testing stage” of the development process in a shared “dev” cluster, there’s a lot of companies that are coming up to create ephemeral environments for the developers to use. These environments are built off of production configuration, therefore definitely improving the status quo.

At DevZero, we take this a step further. Not only are we able to give engineers their ephemeral environments, but we move the coding away from the local laptop. Engineers can now directly write software in remote environments contained within that production-like ephemeral environment - all the benefits of “writing code in prod”, without any of the associated risks.

To find out more about DevZero, visit www.devzero.io to book a demo and evaluate the product.

Debo Ray

Debo Ray

Co-Founder, CEO

Share this post


Built for the cloud you use.

Use your established cloud provider, or let us supply one for you.

Get started →
Featured Posts
DevZero Wins 2023 DEVIES Microservices & Serverless AwardIntroducing DevZero. It’s time to upgrade localhost.Kubernetes vs. Terraform: Which DevOps/Platform Engineering Tool Do You Need?Lessons from Slack’s GitHub Theft, CircleCI Breach WarningGift Guide for Developers: 10 Gifts to Boost ProductivityWhy Software Development Environments are Important and How to Manage them EffectivelyThe container orchestration tools you need to manage your appsProtect Your Source Code With Cloud Development Environments