Question 1

What is inference checkpoint-restore?

Accepted Answer

Inference checkpoint-restore captures the full state of a running inference workload, including GPU memory, model weights in VRAM, KV cache, and engine state, and saves it as a snapshot. New pods can restore from this snapshot in seconds instead of repeating the full cold-start initialization process.

Question 2

How fast is restoring from a checkpoint compared to a cold start?

Accepted Answer

A cold start for Llama 3.1 405B on 8×B200 takes roughly 11 minutes (pulling weights, loading to VRAM, warming the engine, initializing KV cache). Restoring from a DevZero checkpoint takes about 22 seconds. For smaller models like Llama 3.1 70B on 2×H100, restore is about 10 seconds.

Question 3

How does GPU checkpointing work under the hood?

Accepted Answer

DevZero extends CRIU (Checkpoint/Restore In Userspace) with GPU-aware capabilities that capture CUDA context, VRAM contents, and device state alongside the standard process state (memory, file descriptors, network connections). We are core contributors and maintainers of the CRIU project, and are actively upstreaming novel improvements from our GPU checkpoint work.

Question 4

Does checkpoint-restore work with multi-GPU workloads?

Accepted Answer

Yes. DevZero supports checkpointing workloads running across multiple GPUs, including tensor-parallel and pipeline-parallel configurations. We have tested with models up to 405B parameters across 8-GPU NVLink domains, with all GPU states captured and restored atomically.

Question 5

What happens during a spot preemption?

Accepted Answer

When a spot preemption signal is received, DevZero automatically checkpoints the inference workload to remote object storage (S3, GCS, or any S3-compatible store). Once a new spot node is available, the workload is restored from the checkpoint, typically in under 30 seconds.

Question 6

Where are checkpoints stored?

Accepted Answer

Checkpoints are stored in any S3-compatible object storage: Amazon S3, Google Cloud Storage, Azure Blob Storage, or MinIO. Checkpoints are compressed to minimize storage costs and transfer time.

Question 7

Do I need to modify my inference code?

Accepted Answer

No. DevZero operates at the Kubernetes and GPU level. You deploy your inference workloads as normal, and the DevZero operator handles checkpoint and restore operations transparently via Kubernetes CRDs.

Question 8

Which cloud providers and Kubernetes distributions are supported?

Accepted Answer

DevZero works on any Kubernetes cluster with NVIDIA GPUs, including Amazon EKS, Google GKE, Azure AKS, and bare-metal clusters. The operator is installed via a Helm chart and configured through Kubernetes custom resources.

Question 9

How does live migration differ from checkpoint-restore?

Accepted Answer

Live migration moves a running workload between nodes with near-zero downtime. The process state is transferred while the workload continues serving. Checkpoint-restore involves saving state to storage and restoring on a new node, which has a brief interruption (~22 seconds) but works across availability zones and cloud providers.

Question 10

What GPU types are supported?

Accepted Answer

DevZero supports NVIDIA datacenter GPUs including B200, B100, H200, H100, A100, L40S, and L4. Both PCIe and NVLink/NVSwitch configurations are supported, including multi-node NVLink domains.

Feature	DevZero	NVIDIA Dynamo	GKE Pod Snapshots	Raw CRIU
GPU checkpoint/restore	✅	✅	❌	❌
Engine agnostic	✅	❌	N/A	N/A
Multi-GPU support	✅	✅	❌	❌
Cloud portable	✅	❌	❌	✅
Remote storage (S3/GCS)	✅	❌	❌	❌
Live migration	✅	❌	❌	❌
Spot preemption recovery	✅	❌	✅	❌
Kubernetes CRD	✅	❌	✅	❌
Compressed checkpoints	✅	✅	❌	❌

Your GPUs are expensive. Don't waste them on cold starts.

Inference cold starts are the silent GPU killer

Kubernetes-native. Engine-agnostic. Cloud-portable.

Built on CRIU, extended for GPUs

Cloud portable

Operator-managed

Tested at production scale

How DevZero compares

Frequently asked questions

What our customers say

Stop wasting GPUs on cold starts