2 posts tagged with "kubernetes"

How Armada's GPU-as-a-Service Management Platform Orchestrates AI Token Factories at Scale

The Token Factory Challenge​

The phrase "token factory" has moved from marketing metaphor to operational reality. Neoclouds and sovereign cloud providers are assembling clusters of hundreds — or thousands — of GPU nodes, connecting them over high-speed interconnect fabrics, shared across dozens of tenants simultaneously: each with their own isolated network, dedicated storage quota, optional private NVLink or IB partition, and bespoke mix of training, fine-tuning, and inference workloads.

Relying solely on Kubernetes Namespaces or vClusters for multi-tenant isolation in GPU clouds is risky — especially when hosting untrusted or external workloads.

In September 2024, Wiz discovered a critical NVIDIA Container Toolkit vulnerability (CVE-2024-0132) that allowed GPU containers to escape soft isolation and gain root access to the host. This flaw impacted over one-third of GPU-enabled environments and exposed the limits of Kubernetes-based isolation.

Soft isolation is not secure isolation. For environments like Neoclouds, NVIDIA Cloud Partners (NCPs), or regulated industries, only hard or hybrid isolation strategies — such as dedicated Kubernetes clusters, MIG-based GPU partitioning, VPCs, VxLAN, VRFs, KVM virtualization, IB P-KEY, and NVLink partitioning — can protect against container escapes.