Control plane instances
A cluster's control plane usually runs as a workload on an existing Kubernetes cluster. It can instead run directly on machines that vMetal provisions. vCluster calls this a standalone control plane, and the platform installs it on the server as a systemd service rather than as pods. For what a standalone control plane is and how vCluster runs on the server it lands on, see vCluster Standalone.
This is the path for an operator whose hardware is the starting point. You don't need a Kubernetes cluster to host tenant control planes, only the control plane cluster that runs Metal3 itself.
Control plane instances are still instances. They use the same node types, the same server pool, and the same lifecycle as any other. See Instance and server concepts.
Before you begin​
Work through the Metal3 get started walkthrough steps 1 through 4 first, so you have a NodeProvider and servers in available state. Step 5 there is a branch point that routes to one of three guides. This is the one for servers that carry a cluster's control plane, and it picks up where step 4 ends.
The NodeProvider, cluster configuration, and instance commands in this guide use the management cluster context. SSH and systemd checks run on the provisioned server itself.
You also need a node type whose servers are suitable for a control plane. Control planes want reliable, modestly sized machines rather than your largest GPU hosts. Select them by label the same way you would any other node type. See Define node types.
controlPlane is immutable on an instance. You can't convert a worker instance into a control plane instance, or the reverse. The platform creates control plane instances from the cluster's own configuration, so the decision is made when you create the cluster.
1. Define a control plane node type​
Add a node type for the servers you want control planes to land on.
nodeTypes:
- name: "control-plane-node"
displayName: "Control Plane Node"
resources:
cpu: "8"
memory: 32Gi
machines:
selector:
matchLabels:
role: control-plane
properties:
vcluster.com/os-image: ubuntu-noble
vcluster.com/ssh-keys: ops-key-x7k2p
The install script needs a Linux distribution with systemd, plus curl, iptables, and root access. The node also needs raised inotify watcher limits. Use an OS image that covers this, and see Node requirements for the authoritative list. The script detects ARM and x86 hosts on its own.
Include SSH keys. A control plane instance has no kubelet joining it to anything, so if the install fails you reach the server over SSH to read the logs. See Register an SSH key.
2. Create the cluster in standalone mode​
Point the cluster's control plane at your NodeProvider. The platform creates one instance per requested control plane node and installs vCluster on each.
controlPlane:
standalone:
enabled: true
autoNodes:
provider: metal3-provider
quantity: 3
nodeTypeSelector:
- property: vcluster.com/node-type
value: metal3-provider.control-plane-node
The nesting is a fixed path the platform reads. Four values sit under it:
enabledselects standalone control plane mode. Without it, the platform doesn't create control plane instances fromautoNodes.providernames the NodeProvider to claim servers from. The platform reportsNoControlPlaneNodeProvideron the cluster and provisions nothing if you leave it out.quantityis how many control plane instances to claim. It must be at least one. A quantity of zero is rejected withNoControlPlaneNodeQuantity.nodeTypeSelectornarrows which node type the instances come from. Without it, the platform picks from anything the provider offers. Match on the node type's fully qualified name,<provider>.<node type>. See Request nodes from the cluster.
Set quantity to three or more for a highly available control plane. The platform keeps the peer set and its PKI in sync across every control plane instance it creates, so members find each other without manual configuration. For the etcd and datastore implications of a multi-node control plane, see High availability.
3. Watch the control plane come up​
The platform names these instances after the cluster, with a -cp- infix, and labels them loft.sh/vcluster-control-plane with the cluster name.
kubectl --context <management-context> get nodeclaims -n loft-p-my-project \
-l loft.sh/vcluster-control-plane=my-cluster
Control plane instances provision before the cluster is reachable, which is the reverse of a worker. A worker instance waits for the cluster to come online before it provisions, because it needs a live API server to join. A control plane instance doesn't wait, because it's what brings the cluster online.
For the same reason, a control plane instance only needs its network environment's infrastructure to be ready. It doesn't wait for Kubernetes to be provisioned there, the way a worker does.
The cluster controller reports VirtualClusterSynced and VirtualClusterReady after it has created the requested control plane instance objects. Those conditions don't mean the servers have finished provisioning, so neither condition is proof the cluster is up. Keep watching the instances until they reach Available.
Then confirm the cluster itself answers, which is the only check that proves the control plane started:
vcluster connect my-cluster
kubectl get --raw /readyz
A response of ok means the API server on the provisioned instance is serving. If the instances are Available but this fails, the server booted but the vCluster service didn't start. Follow Control plane instance is available but the cluster is offline.
4. Add workers​
A standalone control plane serves workers the same way any cluster does. Add them from the same fleet or a different node type, through the same privateNodes configuration. See Cluster nodes.
Workers wait for the control plane to come online before they provision, so expect them to follow rather than arrive together.
Lifecycle​
Control plane instances are owned by the cluster that created them. Deleting the cluster deletes its control plane instances, and the driver cleans those servers and returns them to the pool.
Control plane instances aren't drained on teardown the way workers are. The platform's graceful node drain applies to joined worker nodes, and a control plane instance isn't one. Quiesce the cluster yourself before removing a control plane member.
Reducing quantity doesn't remove existing control plane instances. The platform creates instances to reach the requested count and doesn't delete them to shrink it. Removing a control plane member is a deliberate operation on the cluster, not a value change. See Manage a standalone control plane for the node-level operations.
The underlying server still follows the normal BareMetalHost lifecycle. See Server lifecycle.
Troubleshoot a control plane instance​
The install runs from cloud-init at first boot, so failures show up on the server rather than in the platform.
- The instance provisions but the cluster never comes online. Follow Control plane instance is available but the cluster is offline. The script checks for
curl, systemd, and root access up front and exits with a clear message when one is missing. It doesn't check the rest, so a missingiptablesor an inotify limit left at the distribution default fails later and less obviously. - The cluster reports
NoControlPlaneNodeProviderorNoControlPlaneNodeQuantity. ThecontrolPlane.standalone.autoNodessection is missing a required field. Nothing is provisioned until you fix it. - The instance never leaves
Pending. No server matched the node type selector. Check that servers carrying the right labels are inavailablestate. See Troubleshooting.