Skip to main content
Version: main 🚧

StatefulSet

The vCluster control plane is deployed as a StatefulSet, since vCluster requires a PersistentVolume resource to store data across restarts. When you configure statefulSet.persistence.volumeClaim.enabled: false or you don't configure statefulSet.persistence.volumeClaimTemplates, vCluster is deployed as a Deployment.

High availability​

  • The highAvailability settings let you run more than one pod for the vCluster control plane, with one running as the leader.
  • If the leader crashes, is unhealthy, or restarts, more pods take over leadership, depending on the number of replicas.
  • Adjusting the leaseDuration / renewDeadline / retryPeriod changes the leader election behavior (how often a leader is renewed & retries when it cannot be renewed)

Scheduling​

scheduling configures various scheduler behavior for different purposes. Examples of each follow:

  • nodeSelector: Matches labels on nodes to make the scheduler "prefer" scheduling the virtual cluster pods on certain nodes. Do this to:
    • Target nodes in a region.
    • Target a specific architecture or machine class.
  • affinity: Value can be affinity (attract the pod) or anti-affinity (repel the pod).
    • Use anti-affinity to spread pods away from each other. A common technique is to make virtual cluster pods repel each other so that they are not scheduled on the same nodes. This increases resiliency in the event a node is scaled down or replaced by a cloud provider.
    • Use affinity to group certain pods together to reduce network latency for critical services
  • tolerations: Another method of influencing where the scheduler places pods. A common use is to "taint" nodes for non-virtual-cluster workloads and make virtual cluster workloads "tolerate" the taint. This results in separating critical apps from the more ephemeral vCluster instances.
  • priorityClassName: See Pod Priority and Preemption.
  • podManagementPolicy: See Pod Management Policies.
  • topologySpreadConstraints: See Pod Topology Spread Constraints.

Re-using existing PersistenceVolumeClaim​

It is possible to pre-provision a PersistenceVolumeClaim, and then configure vCluster to use it. To do that, set existing claim name in the chart values controlPlane.statefulSet.persistence.dataVolume under persistenceVolumeClaim.claimName.

controlPlane:
  statefulSet:
    persistence:
      dataVolume:
        - name: data
          persistentVolumeClaim:
            claimName: my-existing-pvc

vCluster image options​

There are currently 3 vCluster image builds you can use in statefulSet.image.repository:

  • loft-sh/vcluster-pro: The default image for the Helm chart, this image works for all use-cases.
  • loft-sh/vcluster-oss: A purely open-source build of vCluster
  • loft-sh/vcluster: Deprecated. Use loft-sh/vcluster-oss as a replacement.

Config reference​

statefulSet required object pro​

StatefulSet defines options for vCluster statefulSet deployed by Helm.

highAvailability required object pro​

HighAvailability holds options related to high availability.

replicas required integer 1 pro​

Replicas is the amount of replicas to use for the statefulSet.

leaseDuration required integer 60 pro​

LeaseDuration is the time to lease for the leader.

renewDeadline required integer 40 pro​

RenewDeadline is the deadline to renew a lease for the leader.

retryPeriod required integer 15 pro​

RetryPeriod is the time until a replica will retry to get a lease.

resources required object pro​

Resources are the resource requests and limits for the statefulSet container.

limits required object map[ephemeral-storage:8Gi memory:2Gi] pro​

Limits are resource limits for the container

requests required object map[cpu:200m ephemeral-storage:400Mi memory:256Mi] pro​

Requests are minimal resources that will be consumed by the container

scheduling required object pro​

Scheduling holds options related to scheduling.

nodeSelector required object {} pro​

NodeSelector is the node selector to apply to the pod.

affinity required object {} pro​

Affinity is the affinity to apply to the pod.

tolerations required object[] [] pro​

Tolerations are the tolerations to apply to the pod.

priorityClassName required string pro​

PriorityClassName is the priority class name for the the pod.

podManagementPolicy required string Parallel pro​

PodManagementPolicy is the statefulSet pod management policy.

topologySpreadConstraints required object[] [] pro​

TopologySpreadConstraints are the topology spread constraints for the pod.

security required object pro​

Security defines pod or container security context.

podSecurityContext required object {} pro​

PodSecurityContext specifies security context options on the pod level.

containerSecurityContext required object map[allowPrivilegeEscalation:false runAsGroup:0 runAsUser:0] pro​

ContainerSecurityContext specifies security context options on the container level.

probes required object pro​

Probes enables or disables the main container probes.

livenessProbe required object pro​

LivenessProbe specifies if the liveness probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

readinessProbe required object pro​

ReadinessProbe specifies if the readiness probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

startupProbe required object pro​

StartupProbe specifies if the startup probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

persistence required object pro​

Persistence defines options around persistence for the statefulSet.

volumeClaim required object pro​

VolumeClaim can be used to configure the persistent volume claim.

enabled required string|boolean auto pro​

Enabled enables deploying a persistent volume claim. If auto, vCluster will automatically determine based on the chosen distro and other options if this is required.

accessModes required string[] [ReadWriteOnce] pro​

AccessModes are the persistent volume claim access modes.

retentionPolicy required string Retain pro​

RetentionPolicy is the persistent volume claim retention policy.

size required string 5Gi pro​

Size is the persistent volume claim storage size.

storageClass required string pro​

StorageClass is the persistent volume claim storage class.

volumeClaimTemplates required object[] [] pro​

VolumeClaimTemplates defines the volumeClaimTemplates for the statefulSet

dataVolume required object[] [] pro​

Allows you to override the dataVolume. Only works correctly if volumeClaim.enabled=false.

binariesVolume required object[] [map[emptyDir:map[] name:binaries]] pro​

BinariesVolume defines a binaries volume that is used to retrieve distro specific executables to be run by the syncer controller. This volume doesn't need to be persistent.

addVolumes required object[] [] pro​

AddVolumes defines extra volumes for the pod

addVolumeMounts required object[] pro​

AddVolumeMounts defines extra volume mounts for the container

name required string pro​

This must match the Name of a Volume.

readOnly required boolean pro​

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

mountPath required string pro​

Path within the container at which the volume should be mounted. Must not contain ':'.

subPath required string pro​

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).

mountPropagation required string pro​

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.

subPathExpr required string pro​

Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.

EnableServiceLinks for the StatefulSet pod

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

pods required object pro​

Additional labels or annotations for the statefulSet pods.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

image required object pro​

Image is the image for the controlPlane statefulSet container

registry required string ghcr.io pro​

Configure the registry of the container image, e.g. my-registry.com or ghcr.io It defaults to ghcr.io and can be overriding either by using this field or controlPlane.advanced.defaultImageRegistry

repository required string loft-sh/vcluster-pro pro​

Configure the repository of the container image, e.g. my-repo/my-image. It defaults to the vCluster pro repository that includes the optional pro modules that are turned off by default. If you still want to use the pure OSS build, use 'loft-sh/vcluster-oss' instead.

tag required string pro​

Tag is the tag of the container image, e.g. latest

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

workingDir required string pro​

WorkingDir specifies in what folder the main process should get started.

command required string[] [] pro​

Command allows you to override the main command.

args required string[] [] pro​

Args allows you to override the main arguments.

env required object[] [] pro​

Env are additional environment variables for the statefulSet container.

dnsPolicy required string pro​

Set DNS policy for the pod.

dnsConfig required object pro​

Specifies the DNS parameters of a pod.

nameservers required string[] pro​

A list of DNS name server IP addresses. This will be appended to the base nameservers generated from DNSPolicy. Duplicated nameservers will be removed.

searches required string[] pro​

A list of DNS search domains for host-name lookup. This will be appended to the base search paths generated from DNSPolicy. Duplicated search paths will be removed.

options required object[] pro​

A list of DNS resolver options. This will be merged with the base options generated from DNSPolicy. Duplicated entries will be removed. Resolution options given in Options will override those that appear in the base DNSPolicy.

name required string pro​

Required.

value required string pro​