November 20, 2026, (Inside AI) — NVIDIA has opened up its Run:ai orchestration stack, and the first practical consequence is landing on Amazon Elastic Kubernetes Service. A new validated walkthrough shows how the GPU Fractioning Operator can place two workloads on a single NVIDIA A10G GPU inside Amazon EKS, each capped by an enforced 8 GiB memory contract, without relying on Multi-Instance GPU partitioning.
The demonstration matters because it attacks a familiar waste problem. Teams running notebooks, model evaluations, and small inference services often queue for a whole GPU even when most of its memory sits idle. The default NVIDIA device plugin exposes a physical GPU as one indivisible nvidia.com/gpu resource. Two Pods that each request one GPU cannot share a node, even when their combined memory demand would fit comfortably.
The walkthrough, validated on October 22, 2026 and repeated on November 18, 2026, used a single g5.xlarge instance in the US East (Ohio) Region. The A10G carries 23,028 MiB of GPU memory. Two Pods each held an 8 GiB contract and allocated 6 GiB. A third test Pod then attempted a 9 GiB allocation and was rejected by the driver.
Memory Limits Are Not Isolation Guarantees
The distinction between a memory contract and a security boundary is the central caveat. GPU fractioning builds on Multi-Process Service, or MPS, which permits concurrent CUDA processes on one device. NVIDIA documents separate workload error domains in its default mode and a shared error domain in sm-sharing mode. Neither arrangement constitutes a security guarantee, and the walkthrough tested memory limits only, not fault containment or compute performance.
Read: AWS Brings NVIDIA Dynamo to SageMaker HyperPod for Multi-Node LLM Inference
A memory cap also does not reserve proportional compute. Two Pods sharing a device can still contend for streaming multiprocessors and memory bandwidth. The demonstrated density improvement does not establish a throughput gain or a universal cost reduction. That gap between access and performance is where most production evaluations will succeed or fail.
The practical implication is that fractioning widens access to existing capacity without changing the underlying economics of a GPU. Operators gain scheduling control and per-Pod accounting, but they inherit a new monitoring burden. The experiment recorded 6,394 MiB of usage per Pod against a 6,144 MiB allocation, a gap of roughly 250 MiB attributed to CUDA context and runtime overhead. That overhead must be budgeted before contracts are sized.
The operator requires NVIDIA driver branch R615 or newer. At validation time, the EKS-optimized Amazon Linux 2023 NVIDIA image shipped the R580 and R595 branches, so the team built a custom image with Packer. The workflow installed driver 615.71.10 alongside the NVIDIA Container Toolkit and containerd. GPU Operator 26.7.1 supplied the device plugin while preserving the image-owned driver.
NVIDIA announced the open source release of Run:ai on October 13, 2026. The walkthrough uses two of its four modular components: KAI Scheduler and the GPU Fractioning Operator. KAI Resource Management and Workload Map, which address organizational governance and additional workload types, sit outside the tested scope.
The baseline test established the constraint before any sharing was enabled. Two Pods each requested one whole GPU. One ran, the other stayed Pending. After installing KAI Scheduler 0.18.5, which bundles GPU Fractioning Operator 0.1.7, both Pods scheduled onto the same physical GPU. The script verified matching node names and GPU UUIDs, and each Pod reported a bounded device view of 8,192 MiB.
"Fractioning can improve access to existing capacity when workload memory fits," the walkthrough's authors note, adding that production adoption also requires isolation controls and performance testing.
Amazon EKS already documents several sharing routes, including MIG, device-plugin time-slicing, and time-slicing through the NVIDIA DRA driver with Karpenter. Fractioning occupies a specific niche: enforced per-Pod memory limits on GPUs that lack MIG support, paired with fixed-size GPU node groups. Dynamic Resource Allocation offers a newer Kubernetes API for device requests, but this walkthrough relies on NvFractions annotations instead.
Read: Amazon SageMaker AI Adds KV Cache-Aware Routing for Faster LLM Inference
Operational friction remains. The fractioning daemons run as privileged containers with host PID access and NRI and MPS host mounts. Administrators must restrict deployment to trusted operators. The custom image cannot run on EKS Auto Mode-managed nodes, because Auto Mode controls its own AMI selection. Autoscaling is also unvalidated. Fractional Pods request no nvidia.com/gpu resource and target a node group fixed at one node. Karpenter support for secondary schedulers remains an open feature request.
Teams already running Kueue face a governance question before adoption. KAI Scheduler acts as a second scheduler, so organizations must decide which component owns queueing for GPU workloads. The walkthrough also recommends pinning the base AMI, driver, toolkit, device plugin, operator, scheduler, and workload images as one tested combination, then testing rollback before rollout.
The validated configuration is deliberately narrow. Hardware support alone does not establish fractioning compatibility, and the authors confirm only the G5 and A10G pairing. Smaller devices need smaller contracts and adjusted probes. The sample scripts currently require R615 and image-build metadata, so moving to a stock EKS-optimized AMI would demand script changes.
For teams weighing the tradeoff, the decision hinges on workload shape. Interactive notebooks with idle periods, short evaluation jobs, and compatibility tests fit the model well. Regulated or multi-tenant workloads with strict isolation requirements should stay on whole GPUs or MIG where the hardware supports it. The full sample, including the Packer build and cleanup stages, is published in the AWS Samples repository for teams that want to reproduce the results.
If a research paper dropped this week, Mahesh has already read it. He covers AI tools and vibe coding for builders who need to know what is worth their time and what is not.