Storage

vcluster - Persistent Volume Provisioning

Since the vCluster's syncer synchronizes pods to the underlying host cluster to schedule them, vCluster users can use the storage classes of the underlying host cluster to create persistent volume claims and to mount persistent volumes. By default, the host's storage classes can be used without the need to create it in the vCluster, but this can be configured by enabling sync of "storageclasses" or "hoststorageclasses".

vCluster provides helm values to adjust this behavior during vCluster installation or upgrade. Find out more below.

Sync Persistent Volumes

By default, creating persistent volumes in the vCluster will have no effect, as vCluster runs without any cluster scoped access in the host cluster. However, if you enable persistentvolumes sync via helm values, the appropriate ClusterRole will be created in the host cluster and the syncer will be started with a flag that enables persistent volume synchronization from vCluster down to the underlying host cluster.

Create a vCluster with persistent volume sync

To enable the synchronization of the PersistentVolume and StorageClass resources add the following to your values.yaml:

sync:
  persistentvolumes:
    enabled: true
  # If you want to create custom storage classes
  # inside the vCluster.
  storageclasses:
    enabled: true

then create or upgrade the vCluster with:

vcluster create my-vcluster --upgrade -f values.yaml

How does it work?

When you enable persistent volume sync, vCluster will create persistent volumes that are created in vCluster itself in the host cluster in the form of vcluster-PERSISTENT_VOLUME_NAME-x-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME to avoid any conflicts with already existing persistent volumes or other vClusters that sync persistent volumes. vCluster will then rewrite persistent volume claims with those new names so that it seems that the virtual name was bound.

This means that when you create a PVC in the form of:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: persistent-volume-claim
spec:
  storageClassName: my-storage-class
  volumeName: my-persistent-volume
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

vCluster will rewrite this PVC into the following in the host cluster to prevent any conflicts with already existing storage classes or persistent volumes:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: persistent-volume-claim-x-default-x-VCLUSTER_NAME
spec:
  storageClassName: vcluster-my-storage-class-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME
  volumeName: vcluster-my-persistent-volume-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

This only happens if persistent volume sync is enabled in the vCluster. There might be cases where you want to disable this automatic rewriting of PVCs (for example if you want to mount an already existing PV of the host cluster to a PVC in the vCluster), for that case you can set the annotation called vcluster.loft.sh/skip-translate to true, which will tell vCluster to not rewrite the PVC volumeName, storageClass, selectors or dataSource.

Sync Volume Snapshots

Kubernetes VolumeSnapshot resource represents a snapshot of a volume on a storage system. You can read more about volume snapshots on the official Kubernetes documentation page of this feature.

By default, VolumeSnapshot syncing is disabled, and creating a VolumeSnapshot custom resource in the vCluster will have no effect. Following chapters describe how to enable this feature in the vCluster.

Host prerequisites

Vcluster relies fully on the volume snapshot capabilities of the host cluster, which has to fulfill certain criteria.

Host cluster must have all relevant snapshot CRDs installed, without which the vCluster will fail to start when volume snapshots sync is enabled.

Host cluster should have a common snapshot controller installed, as well as a compatible CSI driver. Without these the volume snapshots will not be created in the storage backend.

It is also recommended for the host cluster to have the volume snapshots validating webhook installed.

Create a vCluster with volume snapshots sync

To enable synchronization of all resources relevant for the volume snapshotting, and automatically create the necessary RBAC permissions, add the following to your values.yaml:

sync:
  volumesnapshots:
    enabled: true

then create or upgrade the vCluster with:

vcluster create my-vcluster --upgrade -f values.yaml

info

It is recommend to install the volume snapshots validating webhook in your vCluster instance.

How does it work?

When you enable volume snapshot sync, vCluster will start watching VolumeSnapshot, VolumeSnapshotContent, and VolumeSnapshotClass CRs and syncing them between vCluster and host cluster. These resource types are synced in the following ways:

VolumeSnapshot resources created in the vCluster will be synced to the host cluster with the name in form of vcluster-VOLUME_SNAPSHOT_NAME-x-VOLUME_SNAPSHOT_NAMESPACE-x-VCLUSTER_NAME. The status and finalizers of this resource will be synced back into vCluster. The .spec.source field of the VolumeSnapshot resource in the host cluster will be rewritten to reference the expected PersistentVolumeClaim or VolumeSnapshotContent resource.

VolumeSnapshotContent resources created in the vCluster will be synced to the host cluster with the name in form of vcluster-VOLUME_SNAPSHOT_NAME-x-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME. VolumeSnapshotContent resources created in the host cluster and referencing VolumeSnapshot from the vCluster will be synced into vCluster. The status and finalizers of the resource in host cluster will be synced into its vCluster representation. The .spec.volumeSnapshotRef field of the VolumeSnapshotContent resource will be rewritten to reference the expected VolumeSnapshot resource.

VolumeSnapshotClass resources will be synced from the host cluster into vCluster only.