Private Nodes

Limited vCluster Tenancy Configuration Support

This feature is only available when using the following worker node types:

• Private Nodes

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Using private nodes is a tenancy model for vCluster where, instead of sharing the host cluster’s worker nodes, you assign individual worker nodes to a vCluster instance. These private nodes act as the vCluster’s worker nodes and are treated as worker nodes for the vCluster.

Because these nodes are real Kubernetes nodes, vCluster does not sync any resources to the host cluster as no host cluster worker nodes are used. All workloads run directly on the attached nodes as if they were native to the virtual cluster.

This approach can be understood as follows: the control plane creates a virtual Kubernetes context, while the workloads can run on separate physical nodes that exist entirely within that virtual context.

One key benefit of using private nodes is that vCluster can automatically manage the lifecycle of the worker nodes.

To allow nodes to join the virtual cluster, the cluster must be exposed and accessible. One option is to expose a LoadBalancer service and use its endpoint for the worker nodes. vCluster does not need direct access to the nodes; instead, it uses Konnectivity to tunnel requests—such as kubectl logs—from the control plane to the nodes.

vcluster.yaml configuration

To create a vCluster with private nodes, you'll need to enable the feature as well as set some other configuration options in preparation for worker nodes to join.

Enable private nodes

# Enable private nodes
privateNodes:
  enabled: true

# vCluster control plane options
controlPlane:
  distro:
    k8s:
      image:
        tag: v1.31.2 # Kubernetes version you want to use
  service:
    spec:
      type: LoadBalancer # If you want to expose vCluster using LoadBalancer (Recommended option)

# Networking configuration
networking:
  # Specify the pod CIDR
  podCIDR: 10.64.0.0/16
  # Specify the service CIDR
  serviceCIDR: 10.128.0.0/16

Scheduling enabled by default

When private nodes are enabled, scheduling is handled by the virtual cluster and the virtual scheduler is automatically enabled. You do not need to manually configure this in your vcluster.yaml.

Virtual scheduler automatically enabled with private nodes

# Enabling private nodes enables the virtual scheduler
# Disabling the virtual scheduler is not possible
controlPlane:
  advanced:
    virtualScheduler:
      enabled: true

Other vCluster feature limitations

When private nodes are enabled, certain vCluster features are automatically disabled or unavailable. If you include these options in your vcluster.yaml, they are ignored or might cause configuration errors.

The following features are not available when using private nodes:

• sync.* - No resource syncing between virtual and host clusters
• integrations.* - Integrations depend on syncing functionality
• networking.replicateServices - Services are not replicated to host
• controlPlane.distro.k3s - Only standard Kubernetes (k8s) is supported
• controlPlane.coredns.embedded: true - Embedded CoreDNS conflicts with custom CNI
• controlPlane.advanced.virtualScheduler.enabled: false - Virtual scheduler cannot be disabled
• sleepMode.* - No ability to sleep workloads or control plane

Unsupported vcluster.yaml options with private nodes

# These configurations are NOT supported with private nodes

# Resource syncing between virtual and host clusters is disabled
sync:
  services:
    enabled: false  # Services cannot be synced to host cluster
  secrets:
    enabled: false  # Secrets cannot be synced to host cluster
                    # All other sync options (pods, configmaps, etc.) are also disabled

# Platform integrations require syncing functionality
integrations:
  metricsServer:
    enabled: false  # Metrics server integration not supported
                    # All other integrations are disabled due to sync dependency

# Service replication to host cluster is not available
networking:
  replicateServices:
    enabled: false  # Services run entirely within virtual cluster

# Distribution restrictions
controlPlane:
  distro:
    k3s:
      enabled: false  # k3s distribution not supported
    k8s:
      enabled: true   # Only standard Kubernetes works

  # DNS configuration limitations
  coredns:
    embedded: false   # Embedded CoreDNS conflicts with custom CNI options
  advanced:
    # Virtual scheduler is required for workload placement
    virtualScheduler:
      enabled: true   # Always enabled (cannot be disabled)
  # Host Path Mapper is not supported
  hostPathMapper:
    enabled: false

# Sleep mode is not available
sleepMode:
  enabled: false

# Isolated Control Plane mode is not supported
experimental:
  isolatedControlPlane:
    enabled: false

Configure addons

vCluster supports addons that extend the functionality of your virtual cluster. You can configure these addons during deployment to adjust networking, observability, and other features for your environment and requirements.

CNI

vCluster installs Flannel by default. You can install your own CNI with the following.

Disable default Flannel CNI

privateNodes:
  cni:
    flannel:
      enabled: false

CoreDNS

vCluster installs CoreDNS by default to allow cluster DNS resolving of services and pods.

Konnectivity

vCluster uses Konnectivity to establish a connection between the control plane and worker nodes for commands such as kubectl logs, kubectl exec, kubectl port-forward. The Konnectivity server runs as part of the vCluster control plane, while an agent is deployed inside the vCluster. If it is not needed or wanted, you can disable Konnectivity in the vCluster config.

Kube proxy

vCluster installs Kube Proxy by default to ensure that services are configured on the nodes. Some CNI implement their own kube-proxy functionality. Optionally, you can disable kube-proxy deployment in the vCluster config.

Local Path Provisioner

vCluster installs the Local Path Provisioner by default to allow stateful workloads within the vCluster.

Deploy vCluster with private nodes

For step-by-step instructions on deploying vCluster with private nodes and joining worker nodes, see the Deploy private nodes guide.

Config reference

privateNodes required object

PrivateNodes holds configuration for vCluster private nodes mode.

enabled required boolean false

Enabled defines if dedicated nodes should be enabled.

kubelet required object

Kubelet holds kubelet configuration that is used for all nodes.

config required object {}

Config is the config for the kubelet that will be merged into the default kubelet config. More information can be found here: https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration

autoUpgrade required object

AutoUpgrade holds configuration for auto upgrade.

enabled required boolean true

Enabled defines if auto upgrade should be enabled.

image required string

Image is the image for the auto upgrade pod started by vCluster. If empty defaults to the controlPlane.statefulSet.image.

imagePullPolicy required string

ImagePullPolicy is the policy how to pull the image.

nodeSelector required object

NodeSelector is the node selector for the auto upgrade. If empty will select all worker nodes.

binariesPath required string

BinariesPath is the base path for the kubeadm binaries. Defaults to /usr/local/bin

cniBinariesPath required string

CNIBinariesPath is the base path for the CNI binaries. Defaults to /opt/cni/bin

concurrency required integer 1

Concurrency is the number of nodes that can be upgraded at the same time.

joinNode required object

JoinNode holds configuration specifically used during joining the node (see "kubeadm join").

preInstallCommands required string[]

PreInstallCommands are commands that will be executed before containerd, kubelet etc. is installed.

preJoinCommands required string[]

PreJoinCommands are commands that will be executed before kubeadm join is executed.

postJoinCommands required string[]

PostJoinCommands are commands that will be executed after kubeadm join is executed.

containerd required object

Containerd holds configuration for the containerd join process.

enabled required boolean true

Enabled defines if containerd should be installed and configured by vCluster.

registry required object

Registry holds configuration for how containerd should be configured to use a registries.

configPath required string

ConfigPath is the path to the containerd registry config.

mirrors required {key: object}

Mirrors holds configuration for the containerd registry mirrors. E.g. myregistry.io:5000 or docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

server required string

Server is the fallback server to use for the containerd registry mirror. E.g. https://registry-1.docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

caCert required string[]

CACert are paths to CA certificates to use for the containerd registry mirror.

skipVerify required boolean

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

capabilities required string[]

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

overridePath required boolean

OverridePath is a boolean to override the path for the containerd registry mirror.

hosts required object[]

Hosts holds configuration for the containerd registry mirror hosts. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

server required string

Server is the server to use for the containerd registry mirror host. E.g. http://192.168.31.250:5000.

caCert required string[]

CACert are paths to CA certificates to use for the containerd registry mirror host.

skipVerify required boolean

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

capabilities required string[]

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

overridePath required boolean

OverridePath is a boolean to override the path for the containerd registry mirror.

auth required {key: object}

Auth holds configuration for the containerd registry auth. See https://github.com/containerd/containerd/blob/main/docs/cri/registry.md#configure-registry-credentials for more details.

username required string

Username is the username for the containerd registry.

password required string

Password is the password for the containerd registry.

identityToken required string

IdentityToken is the token for the containerd registry.

auth required string

Auth is the auth config for the containerd registry.

pauseImage required string

PauseImage is the image for the pause container.

caCertPath required string

CACertPath is the path to the SSL certificate authority used to secure communications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

skipPhases required string[]

SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm join --help" command.

nodeRegistration required object

NodeRegistration holds configuration for the node registration similar to the kubeadm node registration.

criSocket required string

CRI socket is the socket for the CRI.

kubeletExtraArgs required object[]

KubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

name required string

Name is the name of the argument.

value required string

Value is the value of the argument.

taints required object[]

Taints are additional taints to set for the kubelet.

key required string

Required. The taint key to be applied to a node.

value required string

The taint value corresponding to the taint key.

effect required string

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

ignorePreflightErrors required string[]

IgnorePreflightErrors provides a slice of pre-flight errors to be ignored when the current node is registered, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

imagePullPolicy required string

ImagePullPolicy specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.