KAI Scheduler

What it is

KAI Scheduler is a Kubernetes scheduler purpose-built for GPU-heavy AI/ML workloads — training jobs, batch inference, distributed jobs — in multi-tenant clusters. It replaces or augments the default kube-scheduler with GPU-aware bin-packing, fractional GPU sharing, gang scheduling (all-or-nothing pod placement for distributed jobs), hierarchical queue-based fair-share allocation, and preemption policies tuned for AI workloads rather than generic microservices. If you've hit the wall where kube-scheduler treats a GPU like any other resource and your cluster utilization is garbage because of fragmentation, this is the category of tool meant to fix that.

Who builds it and why

KAI Scheduler originated at Run:ai, which NVIDIA acquired in 2024, and was open-sourced and donated to CNCF as a sandbox project. The motivation is straightforward: NVIDIA wants Kubernetes to be a credible substrate for large GPU clusters instead of ceding that ground to Slurm or proprietary schedulers, and open-sourcing the scheduler logic removes a barrier for enterprises deciding whether to commit to Run:ai-style scheduling without buying the whole platform. 65 contributors and an active commit cadence (latest touch July 2026) suggest this isn't a dumped-and-abandoned corporate donation — it's still being iterated on, which matters for a component this deep in your control plane.

Production readiness signal

CNCF sandbox status means: early-stage, not yet vetted for graduation, no CNCF guarantee of stability. That's the honest baseline. Signals in its favor: Apache-2.0 licensing (no rug-pull risk), a real contributor base beyond a single company's payroll, and a release cadence that's reached v0.16.2 — past "0.1 experimental" but not yet a 1.0 API-stability commitment. Signals to weigh against: sandbox-stage CNCF projects have no requirement for security audits, no adoption bar, and API/CRD surfaces can still shift release to release. Star count (1356) is modest for a scheduler with this much backing, suggesting adoption is still concentrated among early GPU-cluster operators rather than broad community use. Treat it as "actively developed, pre-1.0 maturity" — pilot it, don't bet your only production GPU cluster on it without a fallback plan.

Who should use this

Platform teams running shared GPU clusters (50+ GPUs, multiple tenants/teams) for training or batch inference who are outgrowing default scheduling and hitting fragmentation, starvation, or unfair queue behavior. Teams already invested in Kubernetes as their orchestration layer who want GPU-aware scheduling without migrating to Slurm or buying a commercial platform. Teams comfortable running CNCF sandbox-stage software with their own testing/rollback discipline, since this touches the scheduler — a component where bugs mean stuck pods, not just a crashed sidecar.

Who should NOT use this

Small clusters or single-tenant GPU usage (a handful of nodes, one team) — the complexity of queues, fair-share, and gang scheduling isn't buying you anything over default scheduling or simple node selectors. Teams that need a CNCF-graduated or incubating project for compliance/procurement reasons — sandbox status will fail that bar. Anyone running HPC-style batch workloads without Kubernetes already in the picture — Slurm is still the more mature, boring answer for pure HPC. Teams that need vendor support contracts today — this is upstream open source with community support only; if you need an SLA, you're looking at Run:ai's commercial product (now NVIDIA) instead.

Alternatives

• Volcano — CNCF incubating batch scheduler with broader workload support (not GPU-exclusive), more mature and widely adopted.
• Apache YuniKorn — general-purpose batch scheduler for Kubernetes with queue-based resource management, less GPU-specialized.
• Slurm — the incumbent HPC scheduler; better fit if you're not committed to Kubernetes as the orchestration layer.

Pricing

Fully open source, Apache-2.0. No paid tier, no dual licensing. Commercial support is available indirectly through NVIDIA/Run:ai's enterprise platform, but the scheduler itself costs nothing to run.