Resource synchronization
The sync section of vcluster.yaml controls which resources sync between the cluster and the control plane cluster, in each direction, and how they're transformed on the way. For the concepts behind syncing, see Resource synchronization in Overview.
- Sync to the control plane cluster covers resources created in the cluster, such as Pods, Services, and Ingresses.
- Sync from the control plane cluster covers resources the cluster reads from the control plane cluster, such as nodes and storage classes.
- Patching transforms fields while resources sync.
- Sync custom resources helps you choose a sync direction and resource type for custom resources.
How does syncing work?β
Since a cluster lacks actual worker nodes and its own network, syncing is the process by which vCluster replicates resources between the cluster and the control plane cluster. This capability is handled by a component called the syncer, deployed by vCluster. The syncer is the core mechanism that enables vCluster to emulate a fully functional Kubernetes cluster. When the syncer fails or malfunctions, it can lead to resource inconsistencies, application failures, state drift, and operational errors.
By default, vCluster only syncs low-level resources, such as pods, secrets, configmaps, or services, but a user can enable many other resources for all use cases. vCluster control plane runs with a minimal set of RBAC permissions to allow execution in restricted environments. Certain resources require extra permissions to sync, which are automatically given to the vCluster ServiceAccount if you enable a resource that requires additional permissions.
All fields of a resource are synced either from the cluster to the control plane cluster or the inverse, from the control plane cluster to the cluster. The direction on how
the sync of the resources is based on what section the resource is part of in the vcluster.yaml. There
are a few exceptions for each resource where specific fields are kept in sync in both directions (that is, bi-directional sync).
If the resource is being asked to sync from the cluster to the control plane cluster, vCluster copies the resources from the cluster and sends it to the control plane cluster to be created. There are a couple of additional labels added to the control plane cluster resource object for vCluster to keep it in sync. The syncer starts watching for changes to the resource on either side. When the resource is updated in the cluster, the syncer copies those changes in the control plane cluster.
When you enable a sync, the syncer will usually copy changes in one direction. For example, if you are syncing pods from the cluster to the control plane cluster, when changes are made to one of the pods in the control plane cluster, those fields are overwritten when the sync occurs. There are a couple of fields that are exceptions that are supported due to bi-directional sync.
If the resource is being asked to sync from the control plane cluster to the cluster, as soon as that resource is created in the control plane cluster, vCluster copies it to the cluster. Any changes to the resource in the control plane cluster are automatically applied to the resource in the cluster.
When syncing resources, vCluster stores the mapping of the VIRTUAL NAME <-> HOST NAME for the actual resource, and the references of the other resources used by this resource.
Tracking references helps the vCluster to understand what resources are still needed in the control plane cluster. This is important for secrets and configmaps because by default only the secrets and configmaps used by pods are actually synced to the control plane cluster.
Assume you've started your vCluster with ingress syncing to the host enabled. The sync field will automatically deploy syncer to sync resources between the cluster and the control plane cluster.
sync:
toHost:
ingresses:
enabled: true
After the cluster is up, the user creates an ingress in the cluster.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
spec:
tls:
- hosts:
- https-example.foo.com
secretName: testsecret-tls
rules:
- http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: nginx
port:
number: 80
When vCluster copies the ingress to the control plane cluster, the vCluster creates an ingress in the namespace of the cluster with the following changes:
- Name Rewritten: vCluster rewrites the name of the resource to avoid conflicts in the control plane cluster. The name is rewritten in the form of
NAME-x-NAMESPACE-x-VCLUSTER_NAME - Namespace Rewritten: vCluster rewrites the namespace of the resource to sync everything into the namespace of the deployed cluster control plane.
- Annotations / Labels Added: vCluster adds annotations and labels to identify the resource in the control plane cluster, which are pre-fixed with
vcluster.loft.sh/.
- References Rewritten: vCluster rewrites the
spec.rules[*].http.paths[*].backend.service.nameandspec.tls[*].secretNameto match the rewritten names of the resources in the control plane cluster.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
vcluster.loft.sh/object-host-name: nginx-x-default-x-vcluster
vcluster.loft.sh/object-host-namespace: vcluster
vcluster.loft.sh/object-kind: networking.k8s.io/v1, Kind=Ingress
vcluster.loft.sh/object-name: nginx
vcluster.loft.sh/object-namespace: default
vcluster.loft.sh/object-uid: bb8bea39-8c0c-4295-8616-6a20b3b1900a
labels:
vcluster.loft.sh/managed-by: vcluster
vcluster.loft.sh/namespace: default
name: nginx-x-default-x-vcluster
namespace: vcluster
spec:
rules:
- http:
paths:
- backend:
service:
name: nginx-x-default-x-vcluster
port:
number: 80
path: /
pathType: ImplementationSpecific
tls:
- hosts:
- https-example.foo.com
secretName: testsecret-tls-x-default-x-vcluster
Resource name length and truncationβ
This rule applies to resources that sync into the vCluster control plane's namespace, which is the default. If you enable namespace syncing and a namespace matches one of your mappings, vCluster keeps the object name unchanged and only rewrites its namespace. Nothing below applies to that object. Namespaces that match no mapping still fall back to the rewriting described here.
vCluster caps every rewritten name at 63 characters. When NAME-x-NAMESPACE-x-VCLUSTER_NAME is longer than that, vCluster shortens it instead of failing the sync.
The cap is vCluster's own and applies to every resource type, so don't read it as a Kubernetes constraint. Kubernetes is stricter for some kinds and far looser for others. Service names are limited to 63 characters, but ServiceAccount, Secret, and ConfigMap names are DNS subdomains and may be up to 253. vCluster rewrites past 63 regardless of what the resource type would allow.
The shortened name keeps the first 52 characters of the full rewritten name. It then appends - followed by the first 10 characters of the hex-encoded SHA-256 digest. The digest covers the whole untruncated name, so two objects that share their first 52 characters still get distinct names.
For example, a ServiceAccount named payments-service-account-with-a-very-long-name in the production-workloads namespace of a cluster named team-alpha-vcluster has this full rewritten name:
payments-service-account-with-a-very-long-name-x-production-workloads-x-team-alpha-vcluster
That name is 91 characters, so vCluster stores the object on the control plane cluster under this name instead:
payments-service-account-with-a-very-long-name-x-pro-d0698fbbcd
Truncated names are 63 characters. There is one exception. When the 52-character prefix ends with a dot, vCluster collapses the resulting .- sequence into a single -, so the name is 62 characters.
Names of 63 characters or fewer are never truncated, and no hash is appended.
To map a synced object on the control plane cluster back to its original name in the cluster, read its vcluster.loft.sh/object-name annotation. vCluster sets it on every synced object, and it always carries the untruncated name.
For namespaced objects, vcluster.loft.sh/object-namespace carries the original namespace in the cluster. Cluster-scoped objects have no namespace, so vCluster removes that annotation from them. Reverse-mapping automation must tolerate its absence rather than assume both annotations are present.
External systems that key off synced object names see the truncated name, not the name inside the cluster. ServiceAccount-dependent integrations are the most common place this surfaces, including EKS Pod Identity, IRSA, GKE Workload Identity, and AKS Workload Identity. An IAM trust policy that references the untruncated name silently fails to match.
Rather than reimplementing this rule, resolve the name through the vCluster Platform API. For a worked example, see Integrate EKS Pod Identity with vCluster.
The API always returns the single-namespace form shown above. It doesn't read your namespace mappings, so for an object in a mapped namespace it returns a name that doesn't exist on the control plane cluster. Use it only for objects that sync into the vCluster control plane's namespace.
Sync labels to the control plane clusterβ
When syncing labels to the control plane cluster, vCluster rewrites the following labels to avoid conflicts:
release- This label must be rewritten when pods are synced to the single namespace where the vCluster control plane pod is running, but not rewritten when the pod is synced in a namespace that is synced from the cluster to the control plane cluster.vcluster.loft.sh/namespacevcluster.loft.sh/managed-byvcluster.loft.sh/controlled-by
Resources available to sync from cluster to control plane clusterβ
vCluster syncs specific Kubernetes resources from the cluster to the control plane cluster. These resources are expected to be created and updated within the cluster. In addition to standard Kubernetes resources, custom resources can also be synced if configured.
Enabled by defaultβ
The following resources are synced by default.
Disabled by defaultβ
The following resources are not synced by default. You can explicitly enable them in your vcluster.yaml configuration.
- CustomResources
- Ingresses
- Gateway API
- NetworkPolicies
- PersistentVolumes
- StorageClasses
- ServiceAccounts
- PodDisruptionBudgets
- PriorityClasses
- Namespaces
Added by the clusterβ
When syncing objects from the cluster to the control plane cluster, the cluster adds management labels to the control plane cluster-side objects.
In the default (single-namespace) mode, the following labels are added:
releasevcluster.loft.sh/namespacevcluster.loft.sh/managed-byvcluster.loft.sh/controlled-by
If sync.toHost.namespaces.enabled is true (multi-namespace sync), the cluster does not add the release label. Only the vcluster.loft.sh/* labels are applied.
Bidirectional syncingβ
For all resources synced from the cluster to the control plane cluster, metadata.labels and metadata.annotations are synchronized in both directions.
This means changes made in the cluster are reflected in the control plane cluster, and changes made in the control plane cluster are also reflected in the cluster.
Any labels prefixed with vcluster.loft.sh on the control plane cluster resources should not be updated by the user. These labels are managed by the vCluster control plane to track resources and prevent conflicts.
Additional fields that support bidirectional syncβ
All metadata.labels and metadata.annotations are bidirectionally synced for resources synced from the cluster to the control plane cluster.
In addition to these fields, vCluster also supports bidirectional syncing for the following resource-specific fields:
| Resource | Fields |
|---|---|
| Pods |
|
| Secrets |
|
| ConfigMaps |
|
| Services |
|
| Ingresses |
|
| Gateway API |
|
| PersistentVolumes |
|
| StorageClass |
|
Resources available to sync from control plane cluster to clusterβ
vCluster can sync certain resources from the control plane cluster to make them available inside the cluster, but when these resources are synced, they are only synced in read-only mode. No changes to the resource in the cluster syncs back to the control plane cluster as the resources are shared across the control plane cluster.
A good example would be nodes, which are nice to view inside the cluster and can be also used to enabled certain features such as scheduling inside the vCluster, but you wouldn't want your cluster to change the node itself. Another benefit of only syncing from the control plane cluster is that vCluster itself only requires read-only RBAC permissions.
vCluster also lets you sync custom resources with the custom resource syncer
There are a couple of labels that are created on the control plane cluster by vCluster that never get synced to the cluster resource. These labels are:
release(Label is needed to avoid conflicts with the vCluster pods themselves).vcluster.loft.sh/namespacevcluster.loft.sh/managed-byvcluster.loft.sh/controlled-byvcluster.loft.sh/ns-label-*These labels are added as management labels when syncing from the cluster to the control plane cluster, but skipped when syncing from the control plane cluster to the cluster to prevent sync loops.
Resource creation behaviorβ
Cluster-scoped resourcesβ
The following resources are not created during sync if they donβt already exist in the cluster:
The following resources are created in the cluster as read-only if they don't already exist during the sync:
- Nodes
- IngressClasses
- Gateways and GatewayClasses
- StorageClasses
- Cluster-scoped CustomResources
Namespaced resourcesβ
Namespaced resources are automatically created in the cluster if they do not already exist. Before synchronization occurs, the mapping between the control plane cluster source namespace name and the corresponding cluster target namespace name must be defined. This applies to:
- Events
- ConfigMaps
- Secrets
- Namespace-scoped CustomResources
For CSIStorageCapacities, synchronization occurs only when a corresponding mapping exists between the physical StorageClass name in the control plane cluster and the StorageClass name in the cluster and the capacity aligns with the associated physical node:
Enabled by defaultβ
Disabled by defaultβ
- CustomResources
- Nodes
- IngressClasses
- Gateways and GatewayClasses
- StorageClasses
- CSINodes
- CSIDrivers
- CSIStorageCapacities
- ConfigMaps
- Secrets
No bi-directional syncingβ
Since syncing resources from the control plane cluster is in read only mode and changes in the cluster do not get applied to the resource in the control plane cluster, bi-directional syncing does not exist across these resources.
Patch synced resourcesβ
Use patching to transform fields, references, and custom resource labels while resources sync between the cluster and the control plane cluster.
Config referenceβ
For every field under sync, see the sync reference.