Metal3
| Enterprise | ||||
|---|---|---|---|---|
| Available in these plans | Free | Dev | Prod | Scale |
| Metal3 Node Provider | ||||
Free, Dev, Prod, and Scale are vCluster Platform license plans. Open source does not need a license or a Platform connection. See Compare open source and free tiers.
The Metal3 provider allows you to provision bare metal servers as instances using Metal3 and Ironic.
When an instance is requested, the platform claims an available BareMetalHost resource and configures it with the requested OS image and user data. Ironic then handles the PXE boot and OS installation on the physical server.
This enables you to offer different configurations of bare metal servers, all managed through BareMetalHost resources on a control plane clusterControl plane clusterThe Kubernetes cluster that hosts the virtualized control planes for clusters. The control plane cluster is operated by the platform provider and is completely invisible to tenants. There are no shared control plane nodes, no in-cluster agent pods, and no lateral path between tenant environments. With shared nodes, this cluster also runs tenant workloads alongside the control plane pods — the same node pool is used for both.. Instances can be used as private nodes for clusters or provisioned independently.
Overview​
The Metal3 provider works by selecting available BareMetalHost resources on a control plane cluster and provisioning them with an OS image and user data configuration. Node types let you organize bare metal servers based on type, location, or other criteria.
What happens once the operating system boots depends on what you requested the instance for. The server can join a cluster as a worker, carry that cluster's own control plane, or be a server you reach over SSH with no Kubernetes on it. Everything before that point is identical. See Two use cases, three outcomes.
When the instance is deleted, the platform powers the server off and clears its image, user data, and network data references. It waits for the BareMetalHost to return to its original reusable state, restores pre-existing managed DHCP annotations, and then releases any address allocated by platform IPAM. A later instance can reuse the host and address after that cleanup completes.
How it works: Provisioning​
After the platform selects a BareMetalHost, it prepares the server for provisioning:
- The platform merges provisioning properties from the NodeProvider, node type, network environment, and instance. The most specific level wins: instance-level properties override node type properties, which override NodeProvider properties.
- If
metal3.vcluster.com/network-cidris configured, the platform allocates an IP from that pool. See Network configuration. - The platform generates a user data configuration and, when an address was allocated, a network data configuration, and stores each in a Kubernetes Secret on the control plane cluster.
- The provider sets the BareMetalHost's
userDataandnetworkDatareferences to these Secrets, and the image information from the configured OSImage or direct image properties.
Ironic then provisions the server through power management, PXE boot, and an in-memory installer that writes the OS to disk. The server boots into the provisioned OS with the user data and network data configuration applied.
The platform always merges the resolved SSH keys into that user data. What it appends alongside them depends on the instance:
| Instance | The platform also appends | Result |
|---|---|---|
| Worker for a cluster | A registration script that joins the server to that cluster | A Kubernetes node in that cluster |
| Control plane for a cluster | The vCluster install script | A server running the cluster's control plane as a systemd service |
| Requested on its own | Nothing further | A provisioned server you reach over SSH |
See Independently provisioned instances, Control plane instances, and Cluster nodes for each path.
Infrastructure deployment​
The Metal3 provider can deploy the required infrastructure components on the control plane cluster. Each component can be individually enabled and customized with Helm values. You can also manage Metal3 and Ironic separately. When you enable the bundled DHCP server with external Ironic, configure proxy.ironicHttpUrl and either proxy.ironicApiUrl or proxy.apiAdvertiseUrl.
Tested versions​
vCluster Platform v5.0 deploys and tests the following versions when the provider deploys the components:
| Component | Version |
|---|---|
| vCluster Labs Metal3 chart | 0.7.3 |
| Ironic | ghcr.io/loft-sh/ironic:39.0.0-vcluster.1 |
| Bare Metal Operator | v0.14.1 |
| IPA images | v0.2.5 |
| MariaDB | 11.4 |
| DHCP server chart | 0.18.0 |
| Multus CNI | v4.2.4-thick |
The platform doesn't check versions. If you override a chart version or image, or manage Metal3 and Ironic yourself, the combination isn't tested, and you're responsible for its compatibility.
- Metal3 & Ironic
- DHCP Server
- Multus
Bare metal provisioning and lifecycle management. Deploys the Bare Metal Operator, Ironic, an IPA image server, and MariaDB as separate workloads.
Install cert-manager on the control plane cluster before enabling this component. The Bare Metal Operator uses it for admission webhooks, and internal components use it for mutual TLS (mTLS). The Platform UI checks this prerequisite and warns when it is missing.
The default Metal3 chart version is 0.7.3. Chart 0.6.0 replaced the legacy single-pod deployment with separate workloads and a MariaDB backend, and chart 0.7.0 switched Ironic to a vCluster Labs build. Review these architecture and database changes before upgrading an existing Metal3 deployment.
vClustervClusterAn open-source software product that creates and manages clusters within Kubernetes infrastructure. vCluster provides tenant isolation capabilities while reducing infrastructure costs. Platform 4.12 deploys chart 0.4.1, so upgrading the platform moves an unpinned provider across both changes. To retain the legacy topology temporarily, pin spec.metal3.deploy.metal3.version to the version that is already deployed.
The components use the following runtime topology:
- Ironic conductors run active/active. They shard host ownership over a consistent hash ring and coordinate through JSON-RPC.
- The managed MariaDB is a single-replica StatefulSet. For high availability, run an external MariaDB and set
mariadb.mode: external. - The IPA image server serves public static boot content outside the mTLS mesh.
Helm values:
| Value | Description | Default |
|---|---|---|
ironic.replicas | Number of Ironic conductors. Values greater than 1 run them active/active. | 2 |
ironic.service | Applied as the Ironic Service spec, so any field can be set (for example type: LoadBalancer, loadBalancerClass, and a nested metadata map for annotations and labels). name, namespace, and selector are managed by the chart. Ports use chart defaults but can be replaced with a custom ports list. Empty leaves a default ClusterIP Service. | {} |
ironic.image.registry | Ironic container image registry. | ghcr.io |
ironic.image.repository | Ironic container image repository. | loft-sh/ironic |
ironic.image.tag | Ironic image tag | 39.0.0-vcluster.1 |
ipaImages.replicas | Replicas of the IPA image server that serves the IPA kernel and initramfs. | 2 |
ipaImages.insecure | Skip TLS certificate verification when Ironic and the IPA fetch any OS image, so images uploaded to a platform with a self-signed certificate work. The image checksum still verifies the downloaded bytes. | true (set by the platform) |
mariadb.mode | Database backend for Ironic: statefulset (single-replica managed MariaDB StatefulSet with its own PVC and cert-manager TLS) or external (connect to your own MariaDB). | statefulset |
mariadb.externalHost | Host Ironic connects to in external mode. Ignored otherwise. | "" |
mariadb.database | Database name. | ironic |
mariadb.username | Database username Ironic uses. An empty value resolves to ironic. | "" |
mariadb.password | Database password Ironic uses. In statefulset mode, an empty value generates a password on first install and preserves it across upgrades. In external mode, an empty value resolves to ironic. | Mode-dependent |
mariadb.caCertSecretName / mariadb.caCertSecretKey | external mode only. Secret and key holding the CA certificate used to verify the external MariaDB over TLS. When set, Ironic connects over TLS. | "" / ca.crt |
mariadb.persistence.size | PVC size for the managed MariaDB in statefulset mode. | 10Gi |
bareMetalOperator.image.registry | Bare Metal Operator image registry. | quay.io |
bareMetalOperator.image.repository | Bare Metal Operator image repository. | metal3-io/baremetal-operator |
bareMetalOperator.image.tag | Bare Metal Operator tag | v0.14.1 |
ipaDownloader.image.registry | IPA image downloader registry. | ghcr.io |
ipaDownloader.image.repository | IPA image downloader repository. | loft-sh/vcluster-platform-ipa-images |
ipaDownloader.image.tag | IPA image downloader tag | v0.2.5 |
Bare metal hosts reach the Ironic agent API through the bundled DHCP proxy, which forwards it to the Ironic Service. The default ClusterIP Service is sufficient for provisioning, so you don't need a LoadBalancer for the agent. See Connectivity inside the control plane cluster.
For highly available MariaDB, run your own database and point Ironic at it:
deploy:
metal3:
enabled: true
helmValues: |
mariadb:
mode: external
externalHost: mariadb.example.internal
database: ironic
username: ironic
password: <MARIADB-PASSWORD>
The platform skips TLS verification for every OS image Ironic and the IPA fetch, including images referenced by an external URL. For an uploaded image, the IPA follows the platform's redirect to the object storage that holds it. It verifies both endpoints against the public CAs in its ramdisk, and there's no option to add a custom CA.
Turn verification back on only when the platform presents a certificate from a public CA. The object storage must do the same or serve plain HTTP. For the security trade-off, see TLS verification for OS images.
To turn verification back on, set ipaImages.insecure to false:
deploy:
metal3:
enabled: true
helmValues: |
ipaImages:
insecure: false
Handles PXE and HTTP Boot by acting as a proxy between bare metal servers and Ironic, which may reside in a different network. When the provider deploys Metal3, the platform configures the boot-image endpoint. Configure proxy.apiAdvertiseUrl to send IPA directly to the bundled Ironic agent endpoint.
The values below target the vcluster-platform-dhcp-server chart 0.18.0.
Deployment fields (NodeProvider metal3.deploy.dhcp):
| Field | Description | Default |
|---|---|---|
enabled | Deploy the DHCP server. | false |
chartRepo | Override the Helm chart repository. | oci://ghcr.io/loft-sh/charts |
chart | Override the Helm chart name. | vcluster-platform-dhcp-server |
version | Override the Helm chart version. | 0.18.0 |
helmValues | Raw YAML passed as values to the chart. | — |
Helm values:
| Value | Description | Default |
|---|---|---|
image.registry | Container image registry. | ghcr.io |
image.repository | Container image repository. | loft-sh/vcluster-platform-dhcp-server |
image.tag | Container image tag. | Chart app version |
image.pullPolicy | Image pull policy. | Always |
imagePullSecrets | Pull secrets for the image. | [] |
replicas | Number of DHCP server replicas. Only supported with a DHCP relay in front of the server, since without one every replica answers the same broadcast. Replies are stateless (addresses come from the BareMetalHost, not a lease database), so any replica can serve any client. Incompatible with networkAttachmentDefinition.enabled; run under hostNetwork or plain pod networking. | 1 |
hostNetwork | Run the pod on the host network. When true, the pod also runs as root and the Multus NetworkAttachmentDefinition isn't attached. | false |
podAnnotations / podLabels | Extra pod metadata. | {} |
resources | Container resource requests and limits. | {} |
nodeSelector / tolerations / affinity | Pod scheduling constraints. | {} / [] / {} |
securityContext | Container security context. Overridden to runAsUser: 0 when hostNetwork is true. | NET_BIND_SERVICE capability, runAsNonRoot: true, runAsUser: 1000 |
extraArgs | Extra arguments appended to the server binary. Use to set the IPA inspector and installer kernel parameters. | [] |
extraEnv | Extra environment variables on the server container. | [] |
networkAttachmentDefinition.enabled | Attach a Multus secondary interface so the pod sits on the provisioning L2 and can answer broadcast DHCP. Ignored under hostNetwork. Set to false for ordinary pod networking with no Multus dependency; this only works behind a DHCP relay, and you must set dhcp.serverIP and http.advertiseUrl to an address clients can reach (typically the provisioningService load balancer). | true |
networkAttachmentDefinition.vip | Virtual IP with prefix for the DHCP server. Injected into the NetworkAttachmentDefinition under ipam.static. Ignored when hostNetwork: true. | 192.168.100.3/24 |
networkAttachmentDefinition.resourceName | Device-plugin resource the NetworkAttachmentDefinition binds to, stamped as the k8s.v1.cni.cncf.io/resourceName annotation so Multus passes the allocated device ID to the CNI plugin (for SR-IOV or other device-plugin interfaces). Empty leaves the annotation off. | "" |
networkAttachmentDefinition.config | CNI configuration JSON for the NetworkAttachmentDefinition. Use bridge if control plane cluster nodes have a bridge attached to the provisioning network. Use macvlan if the bare metal servers are on the same network as the control plane cluster nodes. | bridge br0 |
dhcp.serverIP | IP the DHCP server advertises as itself. | $(SERVER_IP) (pod env) |
dhcp.port | DHCP listen port. | 67 |
dhcp.listenAddr | DHCP listen address. Set to $(SERVER_IP):{{ .Values.dhcp.port }} under hostNetwork to avoid answering on shared bridges. Rendered with tpl. | 0.0.0.0:67 |
tftp.port | TFTP listen port. | 69 |
tftp.listenAddr | TFTP listen address. Set to "" to disable TFTP entirely (HTTP Boot still works). Rendered with tpl. | $(SERVER_IP):69 |
http.port | HTTP boot file server port. | 8080 |
http.listenAddr | HTTP boot file server listen address. Rendered with tpl. | $(SERVER_IP):8080 |
http.advertiseUrl | Base URL clients use to fetch boot files. Also the default for proxy.apiAdvertiseUrl. Rendered with tpl. | http://$(SERVER_IP):8080 |
https.listenAddr | HTTPS boot file server listen address. Set it to enable HTTPS for Redfish virtual media. Bind to 0.0.0.0 when the BMC reaches the server through an interface other than SERVER_IP. Rendered with tpl. | "" |
https.port | HTTPS boot file server port. | 8443 |
https.extraSans | Additional IP addresses or DNS names for the proxy TLS certificate. Include the address that BMCs use for Redfish virtual media. | [] |
provisioningService.type | Service type exposing the DHCP, TFTP, and HTTP ports so a DHCP relay can unicast to one address instead of relying on broadcast. Usually LoadBalancer. Empty leaves the Service out entirely. Needed for replicas > 1. | "" |
provisioningService (spec) | Interpreted as the Service spec, so any field can be set (loadBalancerClass, externalTrafficPolicy, and so on). A nested metadata map is lifted out and applied to the object's metadata. ports defaults to the DHCP, TFTP, and HTTP ports and may be overridden, but an override replaces the defaults wholesale, so it must repeat every port; name, namespace, and selector are always the chart's. | {} |
proxy.callbackPort | IPA callback proxy port. | 8081 |
proxy.callbackListenAddr | IPA callback proxy listen address. Rendered with tpl. | $(POD_IP):8081 |
proxy.callbackAdvertiseUrl | URL Ironic uses to reach the IPA callback proxy. Rendered with tpl. | https://ipa-callback.<release-namespace>.svc.cluster.local |
proxy.apiAdvertiseUrl | Ironic API base URL written into the IPA kernel parameters. Empty defaults to http.advertiseUrl, which routes IPA through this proxy. Keep this default for bundled Metal3. Its operator API requires mutual TLS and isn't an IPA endpoint. For externally managed Ironic, use an API root reachable from the provisioning network. Setting this value makes proxy.ironicApiUrl unnecessary. Rendered with tpl. | "" |
proxy.ironicHttpUrl | Ironic HTTP endpoint the proxy forwards image and boot-file requests to. Always required. | Set to the bundled IPA image service when Metal3 is enabled |
proxy.ironicApiUrl | Ironic API endpoint the proxy forwards to. Optional when proxy.apiAdvertiseUrl points to an external API. For bundled Metal3, the platform sets http://ironic.<namespace>.svc.cluster.local:80, the agent API Service port. | Generated when Metal3 is enabled |
Server flags (via extraArgs):
| Flag | Description |
|---|---|
--inspector-collectors | Comma-separated list overriding the ipa-inspection-collectors kernel parameter in the inspector iPXE script. |
--inspector-extra-kernel-param | Extra kernel parameter appended to the inspector iPXE script. May be specified multiple times. |
--installer-extra-kernel-param | Extra kernel parameter appended to the installer iPXE script. May be specified multiple times. |
--callback-insecure-skip-verify | Skip TLS verification when the callback proxy forwards to IPA. |
--api-advertise-url | Ironic API base URL written into the IPA kernel parameters. Corresponds to proxy.apiAdvertiseUrl; defaults to the HTTP advertise URL. |
--leader-election-id | Name of the Lease electing which replica manages the proxy certificate Secret. The chart sets this flag automatically when replicas is greater than 1; don't add it through extraArgs. The DHCP, TFTP, HTTP, and callback servers keep serving in every replica. |
--tftp-single-port | EXPERIMENTAL. Serve every TFTP transfer from the listen port instead of a per-transfer ephemeral port, so replies survive Service NAT. Required when TFTP traffic is reached through a Service. Some client firmware may reject replies from port 69, so it is off by default. |
Per-BareMetalHost DHCP annotations:
The DHCP server reads these annotations from the matched BareMetalHost to build the DHCP reply.
| Annotation | Description |
|---|---|
metal3.vcluster.com/ip-address | IPv4 address with prefix (CIDR) to lease. Required. |
metal3.vcluster.com/gateway | Default gateway. If unset, the first host address in the ip-address CIDR is used. |
metal3.vcluster.com/dns-servers | Comma-separated DNS servers. |
metal3.vcluster.com/ntp-servers | Comma-separated NTP servers. |
When an instance's network properties supply an address, gateway, or DNS servers, such as from metal3.vcluster.com/network-cidr, the platform writes those annotations itself. When the instance is released, the platform restores the values you set and removes the ones it added.
When the platform-managed DHCP server is enabled, neutronEnabled is off, and no network property supplies an address, the platform validates the host's annotations. metal3.vcluster.com/ip-address must be an IPv4 address with a prefix length. Any gateway or DNS server annotation must be IPv4. Otherwise the instance fails with the reason InvalidConfiguration before the host boots.
Example with a bridge:
deploy:
dhcp:
enabled: true
helmValues: |
networkAttachmentDefinition:
vip: 192.168.100.2/24
config: |
{
"cniVersion": "0.3.1",
"type": "bridge",
"bridge": "br0",
"isDefaultGateway": false
}
Example with macvlan (bare metal servers on the same network as the control plane cluster nodes):
deploy:
dhcp:
enabled: true
helmValues: |
networkAttachmentDefinition:
vip: 10.0.0.2/24
config: |
{
"cniVersion": "0.3.1",
"type": "macvlan",
"master": "eth0",
"mode": "bridge"
}
Example with hostNetwork and custom IPA kernel parameters:
deploy:
dhcp:
enabled: true
helmValues: |
hostNetwork: true
dhcp:
listenAddr: "$(SERVER_IP):67"
extraArgs:
- --inspector-extra-kernel-param=console=ttyS0,115200n8
- --installer-extra-kernel-param=console=ttyS0,115200n8
High availability and DHCP relay​
By default the DHCP server runs a single replica attached to the provisioning L2 through Multus, where it answers broadcast DHCP directly. This mode doesn't scale past one active answerer.
To run more than one replica, place a DHCP relay in front of the server and expose it through a Service:
- Set
replicasgreater than1. - Set
provisioningService.type: LoadBalancerso the relay has a single unicast target that also serves boot files. - Point the relay at the
provisioningServiceaddress, and setdhcp.serverIPandhttp.advertiseUrlto that same address so clients boot through the Service instead of from whichever replica answered. - Add the experimental
--tftp-single-portso TFTP replies survive the Service NAT. The chart configures leader election automatically. - Disable Multus with
networkAttachmentDefinition.enabled: false, since its static IPAM would hand the same VIP to every pod.
deploy:
dhcp:
enabled: true
helmValues: |
replicas: 2
networkAttachmentDefinition:
enabled: false
dhcp:
serverIP: 192.168.100.20
http:
advertiseUrl: http://192.168.100.20:8080
provisioningService:
type: LoadBalancer
metadata:
annotations:
metallb.universe.tf/loadBalancerIPs: 192.168.100.20
extraArgs:
- --tftp-single-port
CNI plugin that enables attaching the DHCP server to a separate provisioning network.
Helm values:
| Value | Description | Default |
|---|---|---|
image.registry | Container image registry | ghcr.io |
image.repository | Container image repository | k8snetworkplumbingwg/multus-cni |
image.tag | Container image tag | v4.2.4-thick |
resources | CPU and memory requests and limits for the main Multus container. The installer init container has its own fixed resources and isn't affected by this value. | 100m CPU and 50Mi memory |
The platform installs Multus into metal3.clusterRef.namespace.
deploy:
multus:
enabled: true
The DHCP server reads its lease data from BareMetalHost annotations. When network properties enable platform IPAM, the provider writes the allocated address and related network settings to those annotations.
Configuration​
A Metal3 NodeProvider configuration consists of a cluster reference, optional infrastructure deployment settings, and a list of node types.
Cluster reference​
The clusterRef specifies the control plane cluster where the platform installs Metal3 components and where the BareMetalHost resources live. This cluster must be connected to vCluster Platform.
| Field | Description | Required |
|---|---|---|
clusterRef.cluster | Name of the connected control plane cluster | Yes |
clusterRef.namespace | Namespace on the control plane cluster for Metal3 components and BareMetalHost resources | Yes |
The cluster and namespace references are immutable after you create the provider. A connected control plane cluster can be referenced by only one Metal3 NodeProvider.
Ironic must have network access to the BMC addresses of the bare metal servers. Ensure the control plane cluster where Ironic is deployed can reach the BMC network (Redfish/IPMI endpoints).
Example​
This configuration deploys Metal3, Ironic, the DHCP server, and Multus, and defines a single node type that selects BareMetalHosts with the label role: compute.
apiVersion: management.loft.sh/v1
kind: NodeProvider
metadata:
name: metal3-provider
spec:
displayName: "Metal3 Bare Metal Provider"
metal3:
clusterRef:
cluster: bare-metal-cluster
namespace: metal3-system
deploy:
metal3:
enabled: true
dhcp:
enabled: true
helmValues: |
networkAttachmentDefinition:
# Attach the DHCP server to the provisioning network.
vip: 192.168.100.2/24
multus:
enabled: true
nodeTypes:
- name: "compute-node"
displayName: "Compute Node"
resources:
cpu: "32"
memory: 128Gi
machines:
selector:
matchLabels:
role: compute
properties:
vcluster.com/os-image: ubuntu-noble
# Claim addresses stay separate from the initial host reservation.
metal3.vcluster.com/network-cidr: 192.168.100.1/24
metal3.vcluster.com/network-ip-range: 192.168.100.100-192.168.100.199
metal3.vcluster.com/dns-servers: 192.168.100.53
Get started​
This walkthrough covers the essential steps to go from a connected control plane cluster to a provisioned bare metal server. Steps 1 through 4 apply to every use of vMetal. Step 5 is a branch point.
The walkthrough spans two clusters, so every command names its context. NodeProvider, OSImage, SSHKey, and NodeClaim are management cluster resources. BareMetalHost and BMC Secrets live on the control plane cluster named by clusterRef.cluster. Replace each placeholder with a name from kubectl config get-contexts.
Before you begin:
- Confirm your hardware, BMCs, and network meet the driver's requirements. The BMC has to be reachable from the control plane cluster, and each server needs a NIC that can PXE boot on the provisioning network.
- Configure BMC certificate trust. The example keeps certificate verification enabled.
- Install cert-manager on the control plane cluster.
- Connect each control plane cluster node to the provisioning network through a
br0bridge, or replace the DHCP NetworkAttachmentDefinition configuration. - Replace all example network addresses with addresses from your provisioning network. The bundled setup routes IPA through the DHCP proxy and keeps the Ironic Services internal.
- Create an OSImage named
ubuntu-noble, or change the node type to reference an existing Metal3-compatible OSImage. - Check your license. vMetal requires a license that includes it, and Metal3 additionally requires the
machine-managementfeature, because it provisions from a Machine inventory. That applies to every instance on this provider, not only ones requested without a cluster. In the platform, navigate to Platform > Platform Config > License and Billing.
SSH keys are injected into the server's cloud-init at provision time. A server that's already provisioned doesn't pick up a key you add afterwards, so you'd have to release and reprovision it.
Add vcluster.com/ssh-keys to the node type in step 1 if you'll want shell access to these servers, even when they're destined to become Kubernetes nodes. It's how you inspect a node whose kubelet never came up. See Register an SSH key.
Apply the NodeProvider.
Create a NodeProvider that references your control plane cluster and defines at least one node type. The node type's
machines.selectordetermines which servers it can claim.apiVersion: management.loft.sh/v1kind: NodeProvidermetadata:name: metal3-providerspec:displayName: "Metal3 Bare Metal Provider"metal3:clusterRef:cluster: bare-metal-clusternamespace: metal3-systemdeploy:metal3:enabled: truedhcp:enabled: truehelmValues: |networkAttachmentDefinition:# Attach the DHCP server to the provisioning network.vip: 192.168.100.2/24multus:enabled: truenodeTypes:- name: "compute-node"displayName: "Compute Node"resources:cpu: "32"memory: 128Gimachines:selector:matchLabels:role: computeproperties:vcluster.com/os-image: ubuntu-noble# Claim addresses stay separate from the initial host reservation.metal3.vcluster.com/network-cidr: 192.168.100.1/24metal3.vcluster.com/network-ip-range: 192.168.100.100-192.168.100.199metal3.vcluster.com/dns-servers: 192.168.100.53# Optional, but add it now if you'll ever want shell access.# vcluster.com/ssh-keys: ops-key-x7k2pkubectl --context <management-context> apply -f metal3-provider.yamlWait for Metal3 and Ironic to be running on the control plane cluster before creating BareMetalHost resources. The Metal3 webhook must be ready to validate them.
Create BMC credentials.
Create a Secret with the BMC username and password for your server. This Secret is referenced by the BareMetalHost resource.
apiVersion: v1kind: Secretmetadata:name: server-01-bmcnamespace: metal3-systemtype: OpaquestringData:username: adminpassword: <BMC-PASSWORD>kubectl --context <control-plane-context> apply -f server-01-bmc-secret.yamlCreate a BareMetalHost.
Register the physical server by creating a BareMetalHost resource. The
bmc.addressscheme selects the BMC driver. ThebootMACAddressidentifies the provisioning NIC.Set a host address for initial inspection. The example uses
192.168.100.20/24. Reserve it outside the claim allocation pool and replace the gateway and DNS address for your network.apiVersion: metal3.io/v1alpha1kind: BareMetalHostmetadata:name: server-01namespace: metal3-systemannotations:# Initial inspection address, outside the claim allocation pool.metal3.vcluster.com/ip-address: 192.168.100.20/24metal3.vcluster.com/gateway: 192.168.100.1metal3.vcluster.com/dns-servers: 192.168.100.53labels:role: computespec:bmc:address: redfish://192.168.1.100/redfish/v1/Systems/1credentialsName: server-01-bmcdisableCertificateVerification: falsebootMACAddress: "aa:bb:cc:dd:ee:01"kubectl --context <control-plane-context> apply -f server-01-bmh.yamlThe server moves through
registeringandinspectingstates as Metal3 verifies BMC access and collects hardware inventory.Verify the server reaches available state.
Once the BareMetalHost passes inspection, it transitions to
available. This means the server is registered, its hardware inventory is collected, and it is ready for provisioning.kubectl --context <control-plane-context> get baremetalhost -n metal3-systemNAME STATE CONSUMER ONLINE ERROR AGEserver-01 available false 5mClaim the server.
Everything to this point is the same whatever you do with the capacity. From here, what you claim the server for decides which guide to follow. All three create a
NodeClaim, use the same node types and the same server pool, and differ by a field or two on the request.You want Follow A server you reach over SSH, with no Kubernetes on it Independently provisioned instances A cluster whose control plane runs on your own hardware Control plane instances Dedicated worker capacity for an existing cluster Cluster nodes Underneath, the three differ only in what the platform sets on the instance and in the cloud-init it generates:
Independently provisioned Worker Control plane spec.vClusterRefEmpty Names the cluster Names the cluster spec.controlPlanefalsefalsetrueCloud-init appends Nothing beyond SSH keys A join command The vCluster install script Result A server you reach over SSH A worker node in the cluster The cluster's control plane In every case the platform selects an eligible server, renders cloud-init, and hands the server to Ironic for provisioning. When a network CIDR is configured, it also allocates an IP. Everything in steps 1 through 4 is identical across all three. See Instance and server concepts for the model behind them.
Define node types​
Each node type specifies which servers it can claim and what properties to apply during provisioning.
Select machines by label​
Use machines.selector to match the provider's Machines by label. Every BareMetalHost has a Machine in the platform's inventory, and the platform copies the host's labels onto it. A selector written against BareMetalHost labels therefore matches the same servers. An unset or empty selector matches every Machine of the provider.
Labels are the only thing that selects a host. The provider doesn't compare a node type's resources with the inspected hardware. Instead, label each host with its hardware profile, for example its CPU, memory, or GPU model. Then point the selector at those labels.
nodeTypes:
- name: "gpu-server"
displayName: "GPU Server"
resources:
cpu: "64"
memory: 256Gi
machines:
selector:
matchLabels:
role: gpu
datacenter: us-east
- name: "general-compute"
displayName: "General Compute"
resources:
cpu: "32"
memory: 128Gi
machines:
selector:
matchLabels:
role: compute
How host labels reach Machines​
The platform keeps each Machine's labels in step with its BareMetalHost:
- It copies every host label except keys under the
vcluster.comandloft.shdomains and their subdomains, which are reserved for the platform. Keys with no domain prefix are always copied. - When the Machine and its host set the same key, the host's value wins.
- Removing a label from the host removes it from the Machine.
- Labels you set on the Machine itself, under keys the host doesn't set, are kept.
The platform refuses an edit on the Machine to a label copied from the host. Change the label on the BareMetalHost instead.
Selection reads the host's current labels, so a server matches as soon as you label its BareMetalHost. A selector on a vcluster.com or loft.sh key matches only labels set on the Machine itself, and saving such a node type returns a warning that says so.
Label hosts with keys under a domain you own, such as example.com/role, or with keys that have no domain prefix.
Migrate from bareMetalHosts.selector​
machines.selector replaces bareMetalHosts.selector, which is deprecated. The platform still reads the old field while machines.selector is unset, and saving a node type that uses it returns a deprecation warning. A node type can't set both. The platform refuses the NodeProvider with deprecated; set only machines.selector.
To migrate, move the selector under machines and keep its contents:
nodeTypes:
- name: "general-compute"
# Before (deprecated):
# bareMetalHosts:
# selector:
# matchLabels:
# role: compute
machines:
selector:
matchLabels:
role: compute
Editing a node type in the platform UI migrates it for you. The form reads and writes only machines.selector, and it removes bareMetalHosts when you save.
For how the platform determines each node type's capacity and cost, see Capacity and Cost.
Configuration properties​
Properties are key-value pairs on node types or instances that control provisioning behavior.
Image configuration​
vcluster.com/os-image​
Type: string
Required: Yes, except on instances that an external platform provider creates, which carry the image properties directly
References an OSImage resource by name. When an instance references an OSImage, the provider reads the image URL, checksum, and checksum type only from it, through the metal3.vcluster.com/image-url, metal3.vcluster.com/image-checksum, and metal3.vcluster.com/image-checksum-type properties. See Metal3 image properties.
Set these image properties on the OSImage. The provider ignores them on a node provider, node type, or network environment. Set vcluster.com/os-image on every instance.
Network configuration​
metal3.vcluster.com/network-cidr​
Type: string (standard CIDR notation)
Specifies the gateway IP and subnet for IP allocation. Use standard CIDR notation where the host portion is the gateway address. The platform allocates IPs from the resulting subnet using its built-in IPAM.
Example: 10.0.0.1/24
metal3.vcluster.com/network-ip-range​
Type: string (comma-separated IP ranges)
Restricts allocation to explicit IP ranges. The metal3.vcluster.com/network-cidr property remains required because it supplies the gateway and prefix. Keep every range within that subnet. Format: IP1-IP2,IP3-IP4
Example: 10.0.0.20-10.0.0.30,10.0.0.40-10.0.0.50
metal3.vcluster.com/dns-servers​
Type: string (comma-separated)
DNS servers to configure on the provisioned server. Example: 8.8.8.8,8.8.4.4
metal3.vcluster.com/network-data​
Type: string
Complete custom network-data configuration for the installed operating system. When set, it replaces the network-data document that the provider would generate. CIDR and IP range properties still control address allocation and provisioning DHCP.
vcluster.com/network-data-template-config​
Type: string
References a MachineConfigTemplate by name. The provider renders the template's networkDataTemplate into a cloud-init network-config document when metal3.vcluster.com/network-data isn't set. See Machine Config Templates for the template syntax and available variables.
Network environment​
vcluster.com/network-environment​
Type: string
Names the network environment, a cluster-scoped NetworkEnvironment resource whose properties are merged into the NodeClaim's effective property set. The platform resolves this from the merged properties of NodeProvider, NodeType, and NodeClaim. When set, this property takes precedence over the typed NodeClaim.spec.environmentRef field.
A network environment can also select the Netris or Aviz network integration. See Network integration for configuration requirements.
Server pinning​
metal3.vcluster.com/server-name​
Type: string
Pins an instance to a specific BareMetalHost by name. This is useful when creating an instance directly and you want to target a particular server. Set this property on the instance, not on the node type.
SSH and user data​
vcluster.com/ssh-keys​
Type: string (comma-separated)
References SSHKey resources by their generated names. The public keys are included in the user data during provisioning. Every key must be visible to the instance's tenant, or the platform refuses the instance. See Share SSH keys with tenants.
vcluster.com/user-data​
Type: string
Custom cloud-init configuration merged with the generated user data. Accepts any valid cloud-config directives, including packages, write_files, and runcmd. The platform appends to this document rather than replacing it. It always adds the resolved SSH keys, and adds a cluster registration command only when the instance has a cluster to register with. Anything the platform appends runs after your own runcmd entries.
Takes precedence over vcluster.com/user-data-template-config.
vcluster.com/user-data-template-config​
Type: string
References a MachineConfigTemplate by name. The provider renders the template's cloudInitTemplate into a cloud-config document when vcluster.com/user-data isn't set. See Machine Config Templates for the template syntax and available variables.
The platform always merges resolved SSH keys into the cloud-config. For a cluster worker, it also appends the join command. For an instance that carries a cluster's control plane, it appends the control plane installation commands. Independently provisioned instances receive no cluster registration command.
vcluster.com/user-data-claim-policy​
Type: string (Replace or Merge)
Default: Replace
Decides how an instance's own user data and SSH keys combine with the ones the node provider and node type set. Only a node provider or a node type can set this property. The platform refuses it on an instance, a network environment, or an OSImage.
Replace: the instance's user data replaces the operator's, and its SSH key list replaces the operator's list.Merge: the instance's user data is added to the operator's, and the SSH keys of both apply. Top-level lists, such asruncmd, get the instance's entries after the operator's, so the operator's commands run first. The operator keeps every value it sets. Awrite_filesentry from the instance is dropped when the operator writes the same path. The operator's user data template renders with the operator's properties only, so an instance can't change what it produces.
Any other value fails the instance with the reason InvalidConfiguration.
Try it yourself​
The community and experimental vCluster Bare Metal with KubeVirt guide runs the Metal3 provisioning flow locally with KubeVirt VMs as fake bare metal servers. It sets up a vind (vCluster in Docker) cluster, a Metal3 NodeProvider, and simulated BareMetalHosts with Redfish BMC endpoints. No physical hardware is required.