Skip to main content

Experimental Generic Sync (Deprecated)

Deprecated

This experimental feature is now deprecated. There are currently no plans to remove it, but further enhancements are not expected to be added. For more advanced capabilities, please consider using the custom resource sync (fromHost, toHost) and any available integrations, or consider writing a custom plugin.

Make host cluster CRDs and resources available to the virtual cluster for use by workloads.

Configuration syntax​

Virtual to host cluster​

experimental:
  genericSync:
    export:
      - apiVersion:
        kind:
        patches:
          - op:
            path:
        reversePatches:
          - op:
            path:
        selector:
          labelSelector:

Use the top-level export field to declare which virtual resources you want to sync to the host cluster. Each item in the export array defines the resource via apiVersion and kind strings. Each apiVersion and kind pair can have only one entry in the export array.

Use patches to define how to modify synced resources before the resource's creation or update in the host cluster.

Use reversePatches to declare how changes to synced host cluster resource propagate back to the original resource in the virtual cluster.

Use selector to limit which resources to sync from the virtual cluster. The virtual resource is synced when it matches one or more selectors, or when the selector field is empty.

See Certificate manager for an example.

Host to virtual cluster​

experimental:
  multiNamespaceMode:
    enabled: true
  genericSync:
    import:
      - apiVersion:
        kind:
        patches:
          - op:
            path:
        reversePatches:
          - op:
            path:
        selector:
          labelSelector:

Requires multi-namespace mode.

Use the top-level import field to declare which host resources to sync to the virtual cluster. Each item in the import array defines the resource via apiVersion and kind strings. Each apiVersion and kind pair can have only one entry in the import array.

Use patches to define how to modify certain synced resource fields before the resource's creation or update in the virtual cluster. See [patches reference] for details.

Use reversePatches to declare how changes to synced virtual cluster resource propagate back to the original resource in the host cluster.

Use selector to limit which resources to sync from the host cluster. The host resource is synced when it matches, or when the selector is empty.

Hooks (Pro)​


Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

Use Hooks to:

  • Patch any Kubernetes resource during synchronization and before resources are applied to the Kubernetes API server.
  • Remove sensitive information before being written to the host or virtual cluster.

You can also add additional information to a resource, such as labels and annotations on all resources belonging to a tenant or add imagePullSecrets to virtual cluster pods as they are synced to the host cluster.

Hooks configuration specifies Kubernetes resources to patch. You can specify the resource group, API version, and resource kind and then the verbs for which API calls to patch. Configure patches using operations like add, remove, replace, and copyFromObject. See Node info remover and add imagePullSecrets to Pods for examples.

Examples​

Find additional configuration examples in the vCluster repository's generic-sync-examples folder.

Certificate manager export​

Create TLS certificates for websites and manage your own cert-manager Issuer. The RBAC-namespaced role and cluster-scoped clusterRole configure permissions.

The vCluster instance must run in multi-namespace mode so that it can:

  • Utilize cert-manager's Issuer resource.
  • Sync cert-manager's namespace-scoped Secret to the virtual cluster's namespace, so the virtual cluster's Ingress can use the Secret.

Follow these steps to use cert-manager:

  1. Create an Issuer in the virtual cluster.
  2. Create an Ingress in the virtual cluster that uses the Issuer through annotation: cert-manager.io/issuer: ${ISSUER_NAME}.
  3. Verify that a Certificate resource has been created for the ingress.
  4. Verify that a Secret with the name from spec.tls[0].secretName has been created and synced back to the vCluster with the generated certificate.
  5. Apply the cert-manager configuration.
cert-manager configuration
experimental:
  multiNamespaceMode:
    enabled: true

sync:
  toHost:
    # sync all secrets
    secrets:
      all: true

    # sync ingresses and allow "cert-manager.io/issuer" annotation
    ingresses:
      enabled: true

    customResources:
      # sync cert-manager certificates
      certificates.cert-manager.io:
        enabled: true
      # sync cert-manager issuers
      issuers.cert-manager.io:
        enabled: true

  fromHost:
    customResources:
      # sync cert-manager cluster issuers
      clusterissuers.cert-manager.io:
        enabled: true
        scope: Cluster

Istio export​

Use Istio VirtualService and Gateway resources to expose vCluster services.

Follow these steps to use Istio:

  1. Install Istio on your host cluster.
  2. Deploy an Istio app in the virtual cluster. You can use Istio's BookInfo app if you don't have one.
  3. Create the VirtualService and Gateway in the virtual cluster by applying the genericSync Istio configuration.
  4. Verify that you can browse to the app's website using the host-installed Istio.
Istio configuration
# Setting these podLabels and coredns.podLabels should ensure that vcluster
# control plane pods won't have the sidecar injected, as this is unnecessary
# for most use cases
podLabels:
  sidecar.istio.io/inject: "false"
coredns:
  podLabels:
    sidecar.istio.io/inject: "false"


# Adding the --sync-labels argument to the syncer will propagate sidecar.istio.io/inject
# label from the workloads(pods) created in the vcluster to their representation in the
# host cluster, which would allow vcluster users to control the sidecar injection
# Comment out 3 lines below to enable this, and merge this with your other syncer values
# if you have some.
# syncer:
#   extraArgs:
#     - "--sync-labels=sidecar.istio.io/inject"

#
# Warning: current configuration supports VirtualService traffic only within the same namespace.
# Traffic to service.namespace.svc(.cluster.local) URLs will not be routed by Istio.
#
---
experimental:
  multiNamespaceMode:
    enabled: true

  genericSync:
    role:
      extraRules:
        - apiGroups: ["networking.istio.io"]
          resources: ["virtualservices", "destinationrules", "serviceentries", "gateways"]
          verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
    # extra cluster scoper permissions required for the plugin
    clusterRole:
      extraRules:
        - apiGroups: ["apiextensions.k8s.io"]
          resources: ["customresourcedefinitions"]
          verbs: ["get", "list", "watch"]
    export:
      - apiVersion: networking.istio.io/v1beta1
        kind: Gateway

      - apiVersion: networking.istio.io/v1beta1
        kind: VirtualService
        patches:
          - op: rewriteName
            path: .spec.gateways[*]
            regex: "($NAMESPACE/)?$NAME"
            conditions:
              - notEqual: "mesh"

      - apiVersion: networking.istio.io/v1beta1
        kind: DestinationRule
        patches:
          - op: rewriteName
            path: .spec.exportTo
            regex: $NAMESPACE
            conditions:
              - notEqual: "."
    import:
      - kind: Secret
        apiVersion: v1

Node info remover hooks (Pro)​


Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

Configure hooks to hide Node images from virtual cluster users. After applying the configuration, you can test by running kubectl get nodes in the virtual cluster.

Node info remover configuration
genericSync:
  hooks:
    hostToVirtual:
    - apiVersion: v1
      kind: Node
      patches:
      - op: remove
        path: status.images
      - op: replace
        path: metadata.annotations
        value: {}
      - op: replace
        path: metadata.labels
        value: {}
      - fromPath: metadata.annotations["node.alpha.kubernetes.io/ttl"]
        op: copyFromObject
        path: metadata.annotations["node.alpha.kubernetes.io/ttl"]
      - fromPath: metadata.labels["kubernetes.io/arch"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/arch"]
      - fromPath: metadata.labels["kubernetes.io/hostname"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/hostname"]
      - fromPath: metadata.labels["kubernetes.io/os"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/os"]
      - fromPath: metadata.labels["beta.kubernetes.io/arch"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/arch"]
      - fromPath: metadata.labels["beta.kubernetes.io/hostname"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/hostname"]
      - fromPath: metadata.labels["beta.kubernetes.io/os"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/os"]
      verbs: ["update", "patch", "create", "get", "list"]
    virtualToHost: []

Pods add imagePullSecrets hook (Pro)​


Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

Configure hooks to add imagePullSecrets to all pods in all namespaces created in the virtual cluster. The secrets specified must be available in the host cluster namespace that the pod is synced. After applying the configuration, you can test by getting the spec of any synced pod and ensuring that the spec.imagePullSecrets has been added.

Pods add imagePullSecrets configuration
experimental:
  genericSync:
    hooks:
      virtualToHost:
      - apiVersion: v1
        kind: Pod
        patches:
        - op: add
          path: spec.imagePullSecrets[0]
          value: 
            name: hook-secret

patches reference​

Bidirectional

opDescription
addAdd contents of the value into the path field. The value can be either scalar or a complex object.
copyFromObjectCopy value of the field referenced in the fromPath from the originating object to the path field of the destination object. The fromPath can be omitted, in such case, it will default to the same field path as referenced in the path.
removeRemove the contents of the path field
replaceReplace the contents of the path field with the contents of the value. The value can be either scalar or a complex object.

Virtual to host

opDescription
rewriteLabelKeyThe keys of the .metadata.labels of the synced resources are rewritten by vCluster and plugins. This patch type allows you to rewrite the key references in the same way, so the fields that are referencing labels will still reference correct labels in their rewritten form. For example, the label key-value pair "app: curl" is rewritten to "vcluster.loft.sh/label-vcluster-x-a172cedcae: curl", so with this patch operation you can rewrite a field that contains "app" to "vcluster.loft.sh/label-vcluster-x-a172cedcae, and the controllers operating on the synced resources will work with this label just as expected.
This is not necessary when using the ["Multi-namespace mode"].(./multi_namespace_mode.mdx).
rewriteLabelSelectorThis operation exists for the same reasons as described for the rewriteLabelKey operation. It is intended to be used for the key-value map fields that represent a label selector. This patch operation will rewrite all keys in the field referenced by path to the expected format for the label keys, and it will also add additional key-value pairs(with virtual namespace and vCluster name) to avoid naming conflicts.
This is not necessary when using multi-namespace mode.
rewriteLabelExpressionsSelectorSimilar to the rewriteLabelSelector, but expects path reference a field with the matchLabels and matchExpressions sub-fields, which will have the label keys rewritten just as described for rewriteLabelKey.
This is not necessary when using multi-namespace mode.
rewriteNameReplaces the contents of the path field with transformed content based on the namespace of the synced resource. This is typically done on the fields that refer to a resource name, and on the .metadata.name as well(implicit). This is done to avoid naming collisions when syncing resources to the host cluster, but it is not necessary when using multi-namespace mode.
As an example, the "logstash" value of a resource in the "logging" namespace of the vCluster named "vc" is rewritten to "logstash-x-logging-x-vc". If the resulting length of the value would be over 63 characters, the last 10 characters will be replaced with a hash of the full value.
rewriteName + namePath + namespacePathSimilar to rewriteName, but with an addition of the namePath and/or namespacePath. This is used when a field of the synced resource is referencing a different resource via namespace and name via two separate fields. When using this option you would set the path to reference a field that is a common parent of both namePath and namespacePath, and these two fields would then contain just the relative path. For example, path: spec.includes + namePath: name + namespacePath: namespace for a resource that contains name in spec.includes.name and namespace in spec.includes.namespace.
rewriteName + regexSimilar to rewriteName, but with an addition of the regex option for the patch. This is used when a string contains not just the resource name, but optionally a namespace, and other characters. For example, a string containing "namespace/name" can be correctly rewritten with the addition of this configuration option - regex: "$NAMESPACE/$NAME". The vCluster uses Go regular expressions to recognize the name part with the "NAME" capture group (can be written as $NAME), and the namespace with the "NAMESPACE" capture group (can be written as $NAMESPACE).

Config reference​

genericSync required object pro​

GenericSync holds options to generically sync resources from virtual cluster to host.

version required string pro​

Version is the config version

export required object[] pro​

Exports syncs a resource from the virtual cluster to the host

apiVersion required string pro​

APIVersion of the object to sync

kind required string pro​

Kind of the object to sync

optional required boolean pro​

replaceOnConflict required boolean pro​

ReplaceWhenInvalid determines if the controller should try to recreate the object if there is a problem applying

patches required object[] pro​

Patches are the patches to apply on the virtual cluster objects when syncing them from the host cluster

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

reversePatches required object[] pro​

ReversePatches are the patches to apply to host cluster objects after it has been synced to the virtual cluster

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

selector required object pro​

Selector is a label selector to select the synced objects in the virtual cluster. If empty, all objects will be synced.

labelSelector required object pro​

LabelSelector are the labels to select the object from

import required object[] pro​

Imports syncs a resource from the host cluster to virtual cluster

apiVersion required string pro​

APIVersion of the object to sync

kind required string pro​

Kind of the object to sync

optional required boolean pro​

replaceOnConflict required boolean pro​

ReplaceWhenInvalid determines if the controller should try to recreate the object if there is a problem applying

patches required object[] pro​

Patches are the patches to apply on the virtual cluster objects when syncing them from the host cluster

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

reversePatches required object[] pro​

ReversePatches are the patches to apply to host cluster objects after it has been synced to the virtual cluster

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

hooks required object pro​

Hooks are hooks that can be used to inject custom patches before syncing

hostToVirtual required object[] pro​

HostToVirtual is a hook that is executed before syncing from the host to the virtual cluster

apiVersion required string pro​

APIVersion of the object to sync

kind required string pro​

Kind of the object to sync

verbs required string[] pro​

Verbs are the verbs that the hook should mutate

patches required object[] pro​

Patches are the patches to apply on the object to be synced

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

virtualToHost required object[] pro​

VirtualToHost is a hook that is executed before syncing from the virtual to the host cluster

apiVersion required string pro​

APIVersion of the object to sync

kind required string pro​

Kind of the object to sync

verbs required string[] pro​

Verbs are the verbs that the hook should mutate

patches required object[] pro​

Patches are the patches to apply on the object to be synced

op required string pro​

Operation is the type of the patch

fromPath required string pro​

FromPath is the path from the other object

path required string pro​

Path is the path of the patch

namePath required string pro​

NamePath is the path to the name of a child resource within Path

namespacePath required string pro​

NamespacePath is path to the namespace of a child resource within Path

value required object pro​

Value is the new value to be set to the path

regex required string pro​

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro​

Conditions are conditions that must be true for the patch to get executed

path required string pro​

Path is the path within the object to select

subPath required string pro​

SubPath is the path below the selected object to select

equal required object pro​

Equal is the value the path should be equal to

notEqual required object pro​

NotEqual is the value the path should not be equal to

empty required boolean pro​

Empty means that the path value should be empty or unset

ignore required boolean pro​

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro​

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro​
configmap required boolean pro​

clusterRole required object pro​

extraRules required object[] [] pro​

role required object pro​

extraRules required object[] [] pro​