Skip to main content

NVIDIA NIM Cluster Template

Overview​

The NVIDIA NIM template provisions a Kubernetes cluster for GPU-accelerated AI/LLM inference using NVIDIA NIM microservices. Use this template when you need to deploy and serve inference with prebuilt, optimized containers and MLOps integration.

Accessing the cluster​

Bridge provides two ways to work with your cluster after it is created:

  • Download kubeconfig — Download the cluster kubeconfig from the cluster menu. Use it to access the cluster from your machine or external tools (e.g., kubectl, IDEs, or CI/CD) by setting KUBECONFIG or merging it into your default kubeconfig.

  • Kubectl Terminal — Use the Kubectl Terminal in the Bridge UI to run kubectl commands, manage resources, and operate the cluster without a separate command-line session.

This guide covers:

  • Configuring cluster name, version, and CNI
  • Selecting the NVIDIA NIM template and cluster nodes
  • Monitoring creation until the cluster is Running
  • Viewing cluster details, downloading kubeconfig, and using the Kubectl Terminal

Prerequisites​

  • Tenant Admin access — Log in as a Tenant Admin to create clusters.
  • Compute resources — Bare Metal or Virtual Machine resources allocated to your tenant.
  • Port-forward on Bridge node — For cluster creation to succeed, port-forward must be running on the Bridge-deployed node (see note below).
Port-forward for cluster creation

Run the port forwarding on the Bridge node so that cluster creation can complete.

kubectl -n amcop-system port-forward --address 0.0.0.0 svc/ingress-ingress-nginx-controller 443:443
  • Keep this command running during cluster creation.
  • If you do not have access to the Bridge node, contact your Bridge Super Administrator.

Create an NVIDIA NIM Cluster​

Step 1: Start Cluster Creation​

  1. Log in to Armada Bridge as a Tenant Admin.
  2. In the left sidebar, open Compute → Kubernetes.
  3. Click Create Cluster. Create Cluster Button

Step 2: Configure Cluster Details​

  1. Select type as Upstream
  2. Enter a name and description for the cluster.
  3. Select the Kubernetes version.
  4. Select the CNI plugin. Bridge supports Flannel and Cilium.
  5. (Optional) Enable Install NVIDIA GPU tools if you want GPU tooling on the cluster.
  6. Click Next. Cluster Basic Info Cluster Setup Details

Step 3: Select Cluster Template​

  1. Choose Deploy NVIDIA NIM for GPU-accelerated inference.
  2. Click Next. NVIDIA NIM Template

Step 4: Select Nodes and Create​

  1. Select the cluster node(s) (Bare Metal or Virtual Machine).
  2. Click Create to start cluster creation. Select Cluster Nodes

Step 5: Monitor Cluster Creation​

Wait until the cluster status is Running.

  1. Initializing Control Planes — Status shows Processing.
  2. Initializing Workers — Status remains Processing. Cluster Processing State
  3. When creation completes, Status shows Running. Cluster Running State

Step 6: View Cluster Details and Access Tools​

Click the cluster name to open the cluster detail view.

  • Overview — Cluster information, and options to scale the cluster, download kubeconfig, access the dashboard, and delete the cluster. The Nodes tab shows the nodes allocated to the cluster.

    Cluster Overview

  • Kubectl Terminal — Click the Kubectl Terminal (arrow) icon to run kubectl commands from the UI.

    Cluster Kubectl Terminal

note

Ensure all pods are in Running state before using the cluster for workloads. In the Kubectl Terminal, run:

kubectl get pods -A

  • GPUs — Click the GPUs tab to view GPU details for the cluster.

    Cluster GPU Details

Step 7: Download Kubeconfig​

  1. Click the menu (ellipsis) icon for the cluster.
  2. Select Kubeconfig to download the cluster kubeconfig file. Cluster Kubeconfig Download

Next Steps​

Last updated on