Skip to main content
Version: v5.0 Stable

GPU Quickstart

This guide walks through provisioning GPU bare metal servers as private nodes in a cluster, building on the NodeProvider and server registration from Get started. It covers GPU-specific prerequisites, node type configuration, and workload verification.

If you need to choose a GPU presentation mode, read GPU presentation modes first. That page covers bare metal PCIe devices, NVIDIA vGPU devices, MIG instances, AMD GPUs, and other accelerator types.

Prerequisites​

Complete Get started first. You need a working NodeProvider with Metal3 deployed and at least one BareMetalHost in available state.

You also need:

  • GPU servers that have passed Metal3 inspection and are in available state.
  • An OS image with GPU drivers pre-installed, cloud-init driver installation, or a vendor GPU Operator installed after provisioning.
  • For NVIDIA production fleets, the NVIDIA GPU Operator is the most common approach.
  • For AMD fleets, see the AMD GPU device plugin and AMD GPU Operator documentation.
  • SSH keys configured as SSHKey resources if you need post-provision server access.

1. Label your GPU servers​

Add labels to your BareMetalHost resources. The platform uses these labels to match servers to node types.

kubectl label baremetalhost server-01 -n metal3-system \
gpu-model=h100 \
gpu-count=8

These label keys are useful for GPU fleets:

  • gpu-model - GPU model, such as h100, a100, or l40s.
  • gpu-count - Number of GPUs per server.
  • rack or datacenter - Location for topology-aware scheduling.

2. Create an OSImage​

An OSImage resource stores the OS image URL and checksum. Multiple node types can reference the same OSImage by name.

apiVersion: management.loft.sh/v1
kind: OSImage
metadata:
name: ubuntu-noble-gpu
spec:
properties:
metal3.vcluster.com/image-url: https://your-registry.example.com/ubuntu-noble-gpu-amd64.img
metal3.vcluster.com/image-checksum: "<sha256-checksum>"
metal3.vcluster.com/image-checksum-type: sha256
kubectl apply -f os-image.yaml

If you don't have a GPU-ready image, use a standard cloud image and install the vendor driver through vcluster.com/user-data. See Configuration properties for user data options.

3. Configure a GPU node type​

Add a GPU node type to your NodeProvider. The resources field defines the capacity that Karpenter uses when selecting and provisioning this node type.

nodeTypes:
- name: "h100-8x"
displayName: "H100 8x GPU"
resources:
cpu: "64"
memory: 256Gi
nvidia.com/gpu: "8"
machines:
selector:
matchLabels:
gpu-model: h100
gpu-count: "8"
properties:
vcluster.com/os-image: ubuntu-noble-gpu
vcluster.com/ssh-keys: my-ssh-key-x7k2p
note

The platform doesn't check resources against the hardware inventory collected during Metal3 inspection. Make sure the CPU, memory, and nvidia.com/gpu values match the hosts the selector picks, and that the GPU resource name matches the vendor's. After the node joins the cluster, the vendor device plugin or GPU Operator advertises the actual GPU capacity to Kubernetes.

For AMD GPUs or another accelerator vendor, use the resource name published by that vendor's Kubernetes integration. For example, an AMD GPU node type may advertise amd.com/gpu instead of nvidia.com/gpu.

4. Create a cluster with GPU private nodes​

Steps 1 through 3 apply to any GPU server. This step and the next assume you want the server to join a cluster. To hand out GPU servers over SSH instead, use the GPU node type from step 3 and follow Independently provisioned instances from step 3 onward. You install the vendor driver through your OS image or user data rather than through a device plugin.

Configure a vCluster to use the GPU node type as a private node. This is the same privateNodes block covered in Cluster nodes, with a GPU node type selected.

privateNodes:
enabled: true
autoNodes:
- provider: metal3-provider
static:
- name: gpu-nodes
quantity: 1
nodeTypeSelector:
- property: vcluster.com/node-type
value: metal3-provider.h100-8x

The platform selects an available h100-8x server, installs the OS, and joins it to the cluster as a worker node.

5. Verify GPU access​

Once the node joins, verify the GPU is visible to the scheduler.

vcluster connect my-cluster
kubectl get nodes -o json | jq '.items[].status.capacity | select(."nvidia.com/gpu")'

The output should show "nvidia.com/gpu": "8" in the node capacity. If it's missing, check that the vendor device plugin or GPU Operator is installed on the node. For AMD or another accelerator, query the resource name that the vendor Kubernetes integration advertises.

What the platform handles​

  • Selecting a BareMetalHost that matches the node type label selector.
  • Allocating an IP and generating network configuration.
  • Generating cloud-init and joining the server to the cluster.
  • Releasing the IP and returning the server to the pool on deprovision.

What you manage​

  • Enrolling BareMetalHost resources and applying GPU labels.
  • Providing an OS image with GPU driver support, or installing drivers through cloud-init.
  • Deploying the vendor device plugin, DRA driver, or GPU Operator inside the cluster.
  • Setting the correct accelerator resource name and count in the node type to match the hardware and device plugin.

For fleet-scale label strategy, node type design, and day-2 lifecycle operations, see GPU Fleet Operations.

For how vCluster syncs GPU resources and DRA objects inside the cluster, see GPU and accelerator support in vCluster.