Cluster Autoscaler versus the DevZero node autoscaler

Cluster Autoscaler resizes node groups that you define in advance, such as AWS Auto Scaling groups, Azure virtual machine scale sets, or Google Cloud managed instance groups. The DevZero node autoscaler, also called dzkarp, doesn't use node groups. It picks an instance type for each batch of pending pods and launches it directly through the cloud API.

dzkarp is the engine inside the DevZero Node Operator. It's built on open source Karpenter and runs on Amazon Elastic Kubernetes Service (Amazon EKS), Azure Kubernetes Service (AKS), and Google Kubernetes Engine (GKE). Compared with Cluster Autoscaler, dzkarp chooses from every instance type that your NodePool allows. It replaces underused nodes with cheaper ones instead of only removing them, and it ranks spot capacity by published interruption rates. This page explains each difference.

Who this page is for

This page is for platform and infrastructure engineers who run Cluster Autoscaler and want to know what changes when they move to DevZero. It assumes you know Cluster Autoscaler concepts such as node groups, expanders, and scale-down. It doesn't cover installation. For setup steps, see the Node Operator overview.

If you run Karpenter instead, see Karpenter versus the DevZero node autoscaler.

Summary

The following table compares Cluster Autoscaler with the DevZero node autoscaler:

Capacity model

Cluster Autoscaler works on node groups. Inside a node group, every node has the same capacity and labels. To offer a workload a different instance size, a different architecture, or a different zone, you create another node group, with its own template and minimum and maximum sizes to maintain.

dzkarp works on NodePools. A NodePool lists what's allowed rather than what to launch: instance categories, families, sizes, architectures, zones, and capacity types. One NodePool can cover hundreds of instance types. You set the image, subnets, and security settings in a provider node class: EC2NodeClass, AKSNodeClass, or GCENodeClass.

On Amazon EKS, the following NodePool allows any compute, general purpose, or memory optimized instance type from the sixth generation onward, on spot or on-demand capacity:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m

Scale-up

Every 10 seconds, Cluster Autoscaler checks for pods that the scheduler marked unschedulable. It builds a template node for each node group, checks which groups could fit the pods, and asks an expander to pick one. The expanders are random, most-pods, least-waste, price, priority, and grpc. Cluster Autoscaler then raises that group's desired size, and the cloud provider launches the node.

The node group's shape limits the result. If the pods need 3 vCPUs and the only matching node group has 16-vCPU nodes, Cluster Autoscaler launches a 16-vCPU node. On AWS, when an Auto Scaling group uses a mixed instances policy, Cluster Autoscaler plans with the first instance type in the policy only.

dzkarp batches pending pods, filters every instance type the NodePool allows by the pods' requests and constraints, and launches the cheapest option that fits. On AWS, it calls EC2 Fleet directly with a list of candidate types, so a shortage of one type doesn't stop the launch.

Scale-down and consolidation

Cluster Autoscaler marks a node as unneeded when its pods' CPU requests and memory requests are both under 50% of the node's allocatable resources and its pods can move elsewhere. It removes a node after the node has been unneeded for 10 minutes, and it removes one non-empty node at a time and waits 10 minutes after a scale-up before it looks at scale-down again. By default it doesn't remove nodes that run pods with local storage, such as emptyDir or hostPath volumes, or nodes that run non-DaemonSet pods from kube-system.

Cluster Autoscaler removes nodes but can't replace an oversized node with a smaller one. dzkarp consolidates in three ways:

• It deletes empty nodes.
• It deletes an underused node when the other nodes have room for its pods.
• It replaces one or more nodes with a single cheaper node that fits the same pods.

Pods with emptyDir volumes or in kube-system don't block consolidation by themselves. PodDisruptionBudgets and the karpenter.sh/do-not-disrupt annotation still protect the workloads that need it. You choose a consolidation policy (WhenEmpty, WhenEmptyOrUnderutilized, or Balanced) and a consolidateAfter delay for each NodePool.

DevZero also tunes consolidation for each NodePool based on how long your workloads take to start. For details, see Startup-time-aware consolidation.

Spot capacity

With Cluster Autoscaler, you get spot capacity through each node group's cloud settings, such as a mixed instances policy on an AWS Auto Scaling group. Cluster Autoscaler has no record of which spot types get reclaimed most often.

dzkarp mixes spot and on-demand capacity in one NodePool through the karpenter.sh/capacity-type requirement. It also ranks spot types by the interruption rates that the cloud provider publishes, so a type that costs slightly more but gets reclaimed less often wins over a cheaper, less stable one. For details, see Stable spot.

Cost optimization

Cluster Autoscaler's price expander prefers cheaper node groups, but only among the groups you've defined, and it doesn't know about your commitments.

dzkarp ranks every allowed instance type by price on each launch. On Amazon EKS, it also reads your EC2 Instance Savings Plans and fills unused commitment before it pays on-demand rates. For details, see Savings Plans awareness.

Disruption control

Cluster Autoscaler controls disruption with cluster-wide flags and two annotations:

• cluster-autoscaler.kubernetes.io/safe-to-evict: "false" on a pod keeps its node from being removed.
• cluster-autoscaler.kubernetes.io/scale-down-disabled: "true" on a node prevents scale-down of that node.

dzkarp gives each NodePool its own disruption budgets. A budget limits how many nodes, or what percentage of nodes, can be disrupted at once. Each budget can apply to specific reasons, such as Underutilized, Empty, or Drifted, and can have a cron schedule and duration, so you can block consolidation during business hours and allow it at night.

The following NodePool disruption settings allow 10% of nodes to be disrupted at a time and block underutilization consolidation on weekdays from 09:00 to 17:00 UTC:

spec:
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    budgets:
      - nodes: "10%"
      - nodes: "0"
        reasons: ["Underutilized"]
        schedule: "0 9 * * 1-5"
        duration: 8h

DevZero node policies also support scheduled overrides, which change instance types, capacity types, limits, and disruption settings for a time window.

DevZero features for Cluster Autoscaler clusters

You don't have to switch autoscalers to start using DevZero. The following features work on clusters that run Cluster Autoscaler:

• Simulate packs a node group's running pods onto the instance types you choose and shows the node count, monthly cost, and risks before you change anything.
• Node compaction drains underused nodes with placement-aware plans and leaves them for Cluster Autoscaler to remove.
• Scheduling health shows node startup times, pending pods, and what blocks scale-down.
• Live migration and continuous workload resizing run in the Workload Operator and don't depend on the node autoscaler.

Move from Cluster Autoscaler to dzkarp

dzkarp annotates every node in the cluster, including nodes that Cluster Autoscaler manages, with the image, subnet, security groups, and IAM role or identity that launched it. DevZero uses these annotations to suggest NodePools and node classes that match your existing node groups.

You can run dzkarp alongside Cluster Autoscaler while you move workloads, then scale the old node groups down. For setup steps, see the setup guide for your cloud:

• AWS setup
• Azure setup
• GCP setup

Next steps