KEDA Autoscaling Guide: Setup, Kafka & Best Practices

Last reviewed: August 13, 2026. Examples use KEDA 2.20.2.

A queue worker does not care how busy its CPU looks. It cares about the work waiting to be processed.

When 40,000 messages are backed up in Kafka, consumer lag is a more useful scaling signal than CPU utilization. Waiting for that backlog to create CPU pressure means reacting after the workload is already behind. At the other end of the cycle, keeping workers running after the queue is empty wastes capacity.

The Kubernetes Horizontal Pod Autoscaler is not limited to CPU and memory. It can consume resource, pod, object, custom, and external metrics. The hard part is often everything around those metrics: deploying an adapter, connecting it to each event source, handling authentication, defining activation behavior, and operating the integration. KEDA (GitHub) packages that work into a Kubernetes-native autoscaling layer.

KEDA connects workloads to event sources such as Kafka, RabbitMQ, Amazon SQS, Azure Service Bus, Prometheus, PostgreSQL, and dozens of others. It can activate a dormant workload when work arrives, let the HPA handle ordinary horizontal scaling, and return the workload to zero when the event source becomes inactive. KEDA 2.20 documents 77 available scalers.

This guide explains:

• how KEDA relates to the Kubernetes HPA;
• how its scaling loop works;
• how to install KEDA with Helm;
• how to scale a Kafka consumer from zero;
• which settings matter in production;
• where the HTTP add-on, external scalers, and managed offerings fit;
• what KEDA does not optimize;
• how DevZero can address the workload-sizing and node layers around KEDA.

What is KEDA?#

KEDA stands for Kubernetes Event-driven Autoscaling. It was created by Microsoft and Red Hat and is now a CNCF Graduated project. The project reached CNCF's highest maturity level on August 22, 2023, and its source code is available under the Apache 2.0 license.

As of August 13, 2026, the current stable patch release is KEDA 2.20.2, released on July 31, 2026. Because compatibility and configuration details can change between releases, production guides and manifests should pin a tested version rather than install an unversioned "latest" release.

KEDA scales Kubernetes workloads, not cluster nodes. A ScaledObject can target a Deployment, StatefulSet, or another resource that implements Kubernetes' /scale subresource. For batch processing, a ScaledJob creates and manages Kubernetes Job objects in response to events.

KEDA does not replace the HPA#

KEDA and the Horizontal Pod Autoscaler work together.

When you create a ScaledObject, KEDA normally creates and manages an HPA for the target workload. KEDA handles the activation and deactivation transitions around zero replicas, while the HPA calculates the desired replica count once the workload is active.

The responsibilities are:

• Zero to one and one to zero: handled by the KEDA operator.
• One to N and N to one: handled by the Kubernetes HPA.
• Event-source integration: handled by KEDA scalers and authentication resources.
• Replica creation: handled by the workload's normal Kubernetes controller, such as a Deployment and its ReplicaSet.

Native HPA configurations default to a minimum of one replica. Kubernetes does expose an alpha HPAScaleToZero feature for certain object and external metrics, but it requires a feature gate and at least one supported metric. KEDA provides a purpose-built activation path plus packaged integrations for event sources.

How KEDA works#

KEDA installs three main components in the cluster.

The KEDA scaling lifecycle#

A typical ScaledObject moves through the following loop:

Activation and scaling thresholds are different controls#

An activation threshold determines whether a workload at zero should wake up. A scaling threshold becomes the HPA target once the workload is active.

Activation uses a strict greater-than comparison: an activation threshold of 10 means the scaler activates when the measured value exceeds 10, not when it equals 10. For an integer signal like Kafka lag, that means 11 or more.

That separation lets you ignore insignificant activity without making active scaling too conservative. For example, you might keep a consumer asleep until Kafka lag exceeds 10, then target roughly 50 messages of lag per replica once it is running.

Key KEDA features#

Scale from application demand#

KEDA can scale on queue depth, Kafka consumer lag, database query results, Prometheus queries, cloud-service metrics, cron schedules, and other signals that are often closer to actual demand than infrastructure utilization.

Scale to zero#

A ScaledObject can use minReplicaCount: 0, allowing an inactive workload to run with no application pods. This is especially useful for background processors, asynchronous pipelines, development services, and other workloads that can tolerate startup latency.

Combine multiple signals#

A workload can use multiple triggers. You might combine Kafka lag with CPU utilization, for example, so that backlog drives event-based scaling while CPU protects against unexpectedly expensive message processing. The HPA uses the largest replica recommendation produced by the configured metrics.

Secure event-source access#

TriggerAuthentication and ClusterTriggerAuthentication separate credentials and identity configuration from the ScaledObject. KEDA supports Kubernetes Secret objects and a range of cloud-native identity and secret providers.

Extend KEDA with custom scalers#

When a built-in scaler does not cover an internal system, KEDA can connect to an externally operated scaler over gRPC. KEDA remains responsible for integrating the resulting metric with its scaling loop, while your team owns the external scaler service and its availability.

How to install KEDA#

Helm is the most common installation method. KEDA also publishes raw manifests and can be installed through OperatorHub or a supported managed-cluster add-on.

Prerequisites#

For the KEDA 2.20 release line, the project's tested Kubernetes compatibility matrix lists Kubernetes 1.33 through 1.35. Treat that as a release-specific tested window, not a permanent rule: check the compatibility matrix again when upgrading KEDA or Kubernetes.

You will also need:

• kubectl configured for the target cluster;
• Helm 3;
• permission to install CRDs, API services, webhooks, and cluster-level RBAC;
• network access from the KEDA operator to the event sources it will query;
• control-plane connectivity to the KEDA metrics service.

KEDA's cluster documentation identifies control-plane connectivity to port 443 as a general requirement and port 6443 for Google Cloud's documented networking path. Validate those paths when using private clusters, restrictive firewalls, or custom control-plane networking.

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
 
helm install keda kedacore/keda \
  --namespace keda \
  --create-namespace \
  --version 2.20.2

kubectl get pods -n keda

kubectl get apiservice v1beta1.external.metrics.k8s.io

kubectl get crd | grep keda.sh

Configure KEDA for a Kafka consumer#

The following ScaledObject scales a Kafka consumer named order-processor between zero and 50 replicas.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor
  namespace: orders
spec:
  scaleTargetRef:
    name: order-processor
 
  minReplicaCount: 0
  maxReplicaCount: 50
 
  pollingInterval: 30
  cooldownPeriod: 300
 
  fallback:
    failureThreshold: 3
    replicas: 3
 
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 300
 
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka.kafka.svc:9092
        consumerGroup: order-processor
        topic: orders
        lagThreshold: "50"
        activationLagThreshold: "10"
        allowIdleConsumers: "false"
        offsetResetPolicy: latest

The authentication configuration is omitted to keep the example focused. In a production cluster, add an authenticationRef linked to a TriggerAuthentication, workload identity, or another supported credential provider rather than placing sensitive values directly in trigger metadata.

What the Kafka settings mean#

Note that a maxReplicaCount of 50 does not guarantee that 50 consumers can become useful. If the topic has 12 partitions and idle consumers are disabled, the effective limit will normally be 12.

Be deliberate about Kafka offset behavior#

offsetResetPolicy should match how the consumer is expected to behave when no committed offset exists.

KEDA also documents an important discovery edge case: when topic is omitted, a consumer group with no commit history may not provide enough information for KEDA to rediscover the topic after the workload reaches zero. Supplying the topic explicitly, as the example does, avoids that particular problem.

Test the Kafka scaler#

kubectl apply -f order-processor-scaledobject.yaml

kubectl get scaledobject order-processor -n orders
kubectl describe scaledobject order-processor -n orders

kubectl get hpa -n orders
kubectl describe hpa -n orders

kubectl get deployment order-processor -n orders -w

Test the cold-start path, scale-out behavior, scale-down behavior, and failure path separately. A configuration that looks correct while the consumer is already running can behave differently when starting with no pods and no committed consumer state.

KEDA production best practices#

Tune activation and scaling independently#

Do not treat the activation threshold and HPA target as interchangeable.

A low activation threshold improves responsiveness but may wake the workload for trivial or transient activity. A high threshold avoids noisy activations but increases the amount of work waiting before the first pod starts.

The scaling target controls the number of active replicas. Set it from observed processing throughput rather than choosing a round number. If one consumer sustainably processes 20 messages per second and the acceptable queue-clearance time is known, you can work backward from those measurements to a useful target.

Decide whether zero is operationally honest#

Scale-to-zero is not free of latency. Detection may take up to one pollingInterval, followed by pod scheduling, image pulling, container startup, readiness checks, and connection establishment with the event source. Once the workload is active, HPA queries are generally more frequent than the default KEDA polling interval.

Use minReplicaCount: 1 when the first request or message cannot tolerate that delay. Zero is usually a better fit for asynchronous work, development environments, scheduled processing, and background pipelines than for latency-critical synchronous paths.

Configure fallback for unreliable metric sources#

A scaler can fail because of event-source downtime, DNS problems, certificate issues, expired credentials, throttling, or network partitions.

Use the fallback block where supported, and choose the replica count based on the safest degraded behavior. A fallback of one may preserve basic processing. A larger value may be appropriate when the cost of accumulating backlog is higher than the cost of temporary overprovisioning.

Fallback is not a substitute for monitoring. It is a controlled response while the scaling signal is unavailable.

Tune HPA behavior, not only KEDA cooldown#

cooldownPeriod is frequently used to solve the wrong problem. It affects the final transition to zero. It does not slow ordinary HPA scale-down while replicas remain active.

Use advanced.horizontalPodAutoscalerConfig.behavior to configure stabilization windows, rate limits, and scale policies for the one-to-N range.

Keep one autoscaling owner#

Do not run an independent HPA and a KEDA-managed HPA against the same workload. Two autoscalers writing the same replica count can produce conflicting changes and unstable behavior.

KEDA supports transferring ownership of an existing HPA when a migration requires it. Otherwise, let the ScaledObject own the HPA lifecycle.

The same principle applies to GitOps. Configure your delivery system so it does not continually restore a static replica count over the value owned by the autoscaler. Depending on the deployment workflow, that may mean omitting the replicas field or configuring an ignore rule for autoscaler-managed changes.

Model concurrency limits before setting maxReplicaCount#

A high maximum does not guarantee useful parallelism.

For Kafka, partition count normally limits productive consumers. For databases, connection limits may become the bottleneck. For external APIs, downstream rate limits may make additional replicas harmful. For Job workloads, storage or network throughput may become the shared constraint.

Set maxReplicaCount from the capacity of the full processing path, not merely the number of pods the cluster could schedule.

Protect long-running work#

When HPA scales a Deployment down, Kubernetes does not know which worker is closest to finishing its current task. A pod that has spent hours processing an item can be selected for termination.

Use application-level acknowledgements, idempotency, graceful shutdown, appropriate termination grace periods, and queue visibility or lease semantics. When each unit of work should have an isolated Kubernetes lifecycle, evaluate ScaledJob instead of a long-running shared worker Deployment.

Use identity rather than static credentials#

Prefer workload identity, IAM roles, or an equivalent cloud-native identity mechanism where the scaler supports it. Keep secret-backed authentication in TriggerAuthentication rather than embedding credentials in the ScaledObject.

Scope access carefully. The KEDA operator must be able to retrieve the credentials and query the event source, which makes its permissions part of the cluster's security boundary.

Monitor each KEDA component separately#

The operator and metrics API server have different responsibilities and different failure modes.

Monitor at least:

• operator reconciliation errors;
• scaler and authentication errors;
• external metrics API availability;
• ScaledObject readiness and activity conditions;
• HPA metric and scaling conditions;
• trigger latency and error rate;
• workloads stuck at zero despite queued work;
• workloads stuck at fallback replica counts;
• webhook availability during deployments.

Do not reduce every failure to "scaling freezes." A broken operator can affect activation and reconciliation, while an unavailable external metrics path prevents the HPA from calculating desired replicas from those metrics. KEDA 2.20.2 also added a dedicated HPAActive condition to make HPA metric health easier to distinguish from overall ScaledObject readiness.

KEDA limitations#

KEDA scales pods, not nodes#

When KEDA increases a replica count, the new pods may remain Pending if the cluster has no capacity. Karpenter, Cluster Autoscaler, or a managed node-provisioning service must provide the nodes underneath them.

KEDA and node autoscaling solve different control loops: KEDA decides how many application instances are needed, while the node layer decides what infrastructure is required to place them.

The standard loop is reactive#

Most KEDA scalers observe a current queue, metric, or event-source state and react to it. Cron triggers can pre-scale for known schedules, and KEDA 2.20 includes an experimental Elastic Forecast scaler as well as a PredictKube integration, but predictive behavior requires an explicitly forecast-based signal rather than appearing automatically in the normal scaling loop.

Scale-to-zero introduces cold starts#

KEDA can detect the work, but it cannot remove image-pull latency, application initialization, cache warming, dependency handshakes, or node-provisioning time.

KEDA does not rightsize each replica#

KEDA can use CPU and memory metrics to influence replica count. That is different from changing the CPU and memory requests assigned to each pod.

If a worker requests four vCPUs but normally needs one, every replica created during scale-out carries the same four-vCPU request until another system or an operator changes the workload specification — see automated pod rightsizing without restarts for how that layer works. CPU and memory triggers also cannot provide scale-from-zero by themselves because no pod exists at zero from which to collect those metrics; they require another non-CPU or non-memory activation signal.

Scaler maturity varies#

KEDA 2.20 lists 77 scalers, but they do not all have the same maintainer, history, availability status, or operational characteristics. Review the documentation and ownership information for the exact scaler you plan to use, then load-test it with your service, authentication method, failure modes, and expected number of ScaledObject resources.

High availability has upstream constraints#

KEDA documents partial rather than complete high availability. Multiple operator replicas can provide a standby, but only one is active. The external metrics API also has an upstream constraint around the active provider for external.metrics.k8s.io. Multiple replicas can reduce some downtime, but they do not turn every KEDA path into an active-active architecture.

Open-source KEDA support is community-provided on a best-effort basis. Organizations requiring contractual support should evaluate a commercially supported distribution or service.

HTTP scaling, external scalers, and managed KEDA#

KEDA HTTP Add-on#

HTTP request scaling is not part of core KEDA. The separate KEDA HTTP Add-on places an interceptor in the request path, counts request activity, and reports metrics that KEDA can use to scale an HTTP backend, including from zero.

The current 0.15 documentation uses two user-managed resources:

• an InterceptorRoute for routing, service targeting, and request metrics;
• a normal KEDA ScaledObject for scaling behavior.

The older HTTPScaledObject API is deprecated. New implementations should use InterceptorRoute rather than copying older examples based on HTTPScaledObject.

Because the interceptor sits between ingress and the application, the add-on changes the HTTP request path. Evaluate availability, latency, timeout behavior, observability, and cold-start buffering as application architecture concerns rather than autoscaler settings.

External scalers#

External scalers let a team expose an internal or proprietary scaling source through KEDA's gRPC contract. This is appropriate when the signal cannot be represented by a built-in scaler or an existing metrics adapter.

The external service becomes another production dependency. It needs authentication, TLS where appropriate, monitoring, capacity planning, and a failure strategy.

Managed KEDA on AKS#

AKS Automatic includes KEDA preconfigured. AKS Standard offers KEDA as a managed add-on. Microsoft also notes that the HTTP add-on is separate from the managed KEDA installation and that only one external metrics server should operate in the cluster.

KEDA 2.15 and later removed Azure pod identity support. Workloads crossing that version boundary should move to Microsoft Entra Workload ID rather than relying on the deprecated identity model.

Check the provider's KEDA-to-Kubernetes version mapping instead of assuming that a managed add-on always runs the newest upstream patch.

KEDA alternatives and complements#

Plain Kubernetes HPA#

Use HPA directly when an existing resource, custom-metrics, or external-metrics pipeline already provides the signal you need and you do not require KEDA's event-source integrations or activation lifecycle.

This is the lowest-component option, but your team owns the metric adapter and event-source integration.

Knative Serving#

Knative Serving (GitHub) is a stronger fit for teams adopting a complete request-driven application platform. It provides HTTP-oriented autoscaling, scale-to-zero, revisions, routing, traffic splitting, and an activator that can buffer requests while a revision starts. That is a broader platform decision than adding KEDA to an existing Deployment.

KEDA HTTP Add-on#

The HTTP add-on is the more incremental option when you want KEDA-based HTTP scale-to-zero without adopting the broader Knative Serving model. Its interceptor still becomes part of the request path, so the operational tradeoff must be evaluated explicitly.

Node autoscalers#

Karpenter, Cluster Autoscaler, and managed node-provisioning systems are complements rather than replacements. KEDA creates demand for pods; the node autoscaler supplies infrastructure for pods that cannot be placed.

How DevZero complements KEDA#

Commercial context: This section explains how DevZero fits around KEDA. KEDA is an independent CNCF open-source project. Statements about DevZero capabilities are based on DevZero's product documentation.

KEDA answers an important question: how many replicas should this workload have right now? It does not, by itself, answer two adjacent questions: how large should each replica be, and what nodes should run those replicas?

Those layers matter because horizontal scaling multiplies the workload's existing resource requests. A consumer that requests four vCPUs and uses one does not waste only three vCPUs at idle. If KEDA scales it to 40 replicas, the gap between requested and used CPU is multiplied across the burst.

In Datadog's 2024 sample of organizations using its Cloud Cost Management product to analyze AWS bills, 83% of containerized EC2 costs were associated with idle resources. Datadog divided that result into 54 percentage points of cluster idle and 29 percentage points of workload idle. The 29% is not 29% of the 83%; it is a separate portion of the total. Because the result comes from a particular customer and cloud sample, it should be treated as evidence of a widespread optimization problem rather than as a universal industry constant.

DevZero Workload Operator#

DevZero's Workload Operator, the write operator in the product architecture, generates and applies workload optimization recommendations that adjust both resource requests and limits.

According to DevZero's documentation, it can use Kubernetes in-place vertical scaling to change CPU and memory on running pods when the cluster version, container runtime, pod configuration, QoS implications, and available node capacity permit it. When an in-place resize is not feasible, the operator can fall back to a rolling restart. Memory-limit decreases — the most common rightsizing change — are applied in place on Kubernetes 1.34 and newer for workloads that explicitly opt in; without the opt-in they keep the predictable restart-based path.

The distinction matters:

• KEDA horizontal scaling changes the number of replicas.
• In-place vertical scaling changes CPU and memory assigned to an existing replica.
• Rolling replacement or migration recreates a pod with a new specification.

DevZero also documents a CRIU-based live-migration path. That process checkpoints application state, recreates the pod with its new resource specification, and restores the checkpoint. It is not the same as leaving the original pod untouched, and it has workload, node-agent, image, memory-footprint, and networking prerequisites. When migration cannot complete, the documented behavior is to fall back to a standard rolling restart.

In-place resizing is limited to CPU and memory. GPU, hugepage, and ephemeral-storage changes require pod recreation rather than an in-place resize.

DevZero Scheduler#

The DevZero Scheduler, dz-scheduler, runs alongside the default Kubernetes scheduler. Workloads opt in through schedulerName: dz-scheduler.

Its documented plugins score candidate nodes using factors including node cost, target allocated CPU and memory, and limit overprovisioning. It also filters checkpoint-restored pods so they land on compatible nodes. Standard Kubernetes constraints such as taints, affinity, and topology remain part of the scheduling decision.

This layer becomes relevant when KEDA creates a burst of replicas simultaneously. Better placement can reduce fragmentation and the amount of cleanup the node layer must perform afterward.

DevZero Node Operator#

The DevZero Node Operator handles node provisioning and consolidation beneath the pod-scaling loop.

DevZero's documentation describes it as provisioning nodes for Pending pods, selecting a fitting instance type based on cost and workload requirements, consolidating underutilized nodes, managing spot capacity, and respecting PodDisruptionBudgets and placement constraints during consolidation.

For existing Karpenter users, DevZero documents compatibility with NodePool, NodeClaim, and EC2NodeClass resources and provides a controller-migration procedure. Treat that as a migration that requires validation and rollback planning, not as a reason to replace a production controller without testing.

Taken together, the intended control loops are:

• KEDA: determines how many replicas demand requires.
• DevZero Workload Operator: adjusts the requests and limits assigned to those replicas.
• DevZero Scheduler: determines where opted-in pods should land.
• DevZero Node Operator: provisions and consolidates the node fleet beneath them.

What KEDA will not fix#

KEDA is effective when replica count is the variable that needs to change.

It will not automatically correct:

• oversized CPU or memory requests;
• inefficient application startup;
• a Kafka topic with insufficient partitions;
• unsafe message acknowledgement or retry behavior;
• downstream database or API limits;
• poor pod placement;
• missing cluster capacity;
• an overly aggressive or overly conservative threshold;
• a workload that cannot tolerate being interrupted during scale-down.

That is not a criticism of KEDA, just the boundary of its responsibility.

The most useful mental model is to separate the system into four questions:

1. Demand: What signal represents work?
2. Replica count: How many copies should run?
3. Replica size and placement: How large should each copy be, and where should it run?
4. Infrastructure: What nodes are needed to host the resulting pods?

KEDA provides a strong answer to the first two questions. Production efficiency depends on solving the other two as well.

See where horizontal scaling is multiplying oversized requests. Analyze your cluster in minutes:

npx devzero@latest analyze-cluster

Or get started with DevZero.

Frequently Asked Questions#

Is KEDA free?#

Yes. KEDA is open source under the Apache 2.0 license and does not charge a software license fee. You still pay for the cluster resources used by the KEDA control plane and the workloads it scales. Open-source support is community-based and best effort; commercial support options also exist.

Does KEDA replace the Horizontal Pod Autoscaler?#

No. For a ScaledObject, KEDA normally creates and manages an HPA. The KEDA operator handles activation around zero, while the HPA handles scaling between one and the configured maximum.

Can I use KEDA and a separate HPA on the same workload?#

Do not run two independent HPAs against the same workload. They can compete over the replica count. Put the required metrics into the KEDA-managed configuration or use KEDA's documented HPA ownership-transfer mechanism during a migration.

Can KEDA scale on CPU and memory?#

Yes. KEDA includes CPU and memory triggers, and those triggers use Kubernetes Metrics Server.

CPU or memory alone cannot activate a workload from zero because there is no running pod from which to collect utilization. To combine resource scaling with scale-to-zero, add a non-CPU or non-memory trigger such as Kafka or Prometheus.

Can KEDA scale Kubernetes nodes?#

No. KEDA changes workload replica counts. A node autoscaler or managed provisioning service must supply capacity for Pending pods.

How quickly does KEDA scale from zero?#

Trigger detection may take up to one pollingInterval, which defaults to 30 seconds. The total cold-start time also includes scheduling, node provisioning where needed, image pull, container startup, readiness checks, and application initialization.

Measure the complete path in your own cluster. Keep a warm replica when that latency is not acceptable.

Which Kubernetes versions does KEDA support?#

Compatibility is release-specific. KEDA 2.20 is documented as tested with Kubernetes 1.33 through 1.35. Check the compatibility matrix for the exact KEDA version before upgrading either side.

Can KEDA scale HTTP workloads?#

Core KEDA does not include request interception. The separate KEDA HTTP Add-on supports HTTP-based scaling and scale-to-zero.

For current deployments, use an InterceptorRoute together with a user-managed KEDA ScaledObject. HTTPScaledObject is deprecated.

What happens when KEDA cannot read the metric source?#

Without a usable metric, normal event-driven scaling decisions can be disrupted. For supported ScaledObject triggers, the fallback configuration can provide a defined replica target after a set number of consecutive failures.

Fallback does not support CPU and memory triggers and should be paired with alerts for scaler, authentication, network, and external metrics errors.

Does KEDA perform predictive scaling?#

Most KEDA configurations are reactive: they scale from the metric currently reported by an event source.

Predictive behavior is possible when the trigger itself provides a forecast. KEDA 2.20 includes an experimental Elastic Forecast scaler and also documents a PredictKube scaler, but those are explicit integrations with their own dependencies and maturity considerations.

Is Kafka replica count always equal to lag divided by lagThreshold?#

No. That is a useful approximation, not a complete model.

The final replica count is affected by the current HPA state, minimum and maximum replicas, stabilization behavior, metric availability, activation status, and, by default, the number of Kafka partitions.