vLLM

NVIDIA Dynamo Snapshot cuts inference startup time from minutes to seconds on Kubernetes

NVIDIA introduced Dynamo Snapshot, a checkpoint/restore system that reduces cold-start latency for GPU inference workloads on Kubernetes by capturing both CUDA device state and host process state, then restoring them across cluster nodes. For commerce teams running auto-scaling inference deployments, this eliminates GPU idle time during traffic spikes and dramatically reduces SLA violation risk when demand suddenly increases.