Experimental generic sync

Make host cluster CRDs and resources available to the virtual cluster for use by workloads.

Configuration syntax

Virtual to host cluster

experimental:
  genericSync:
    export:
      - apiVersion:
        kind:
        patches:
          - op:
            path:
        reversePatches:
          - op:
            path:
        selector:
          labelSelector:

Use the top-level export field to declare which virtual resources you want to sync to the host cluster. Each item in the export array defines the resource via apiVersion and kind strings. Each apiVersion and kind pair can have only one entry in the export array.

Use patches to define how to modify synced resources before the resource's creation or update in the host cluster.

Use reversePatches to declare how changes to synced host cluster resource propogate back to the original resource in the virtual cluster.

Use selector to limit which resources to sync from the virtual cluster. The virtual resource is synced when it matches one or more selectors, or when the selector field is empty.

See Certificate manager for an example.

Host to virtual cluster

experimental:
  multiNamespaceMode:
    enabled: true
  genericSync:
    import:
      - apiVersion:
        kind:
        patches:
          - op:
            path:
        reversePatches:
          - op:
            path:
        selector:
          labelSelector:

Requires multi-namespace mode.

Use the top-level import field to declare which host resources to sync to the virtual cluster. Each item in the import array defines the resource via apiVersion and kind strings. Each apiVersion and kind pair can have only one entry in the import array.

Use patches to define how to modify certain synced resource fields before the resource's creation or update in the virtual cluster. See [patches reference] for details.

Use reversePatches to declare how changes to synced virtual cluster resource propogate back to the original resource in the host cluster.

Use selector to limit which resources to sync from the host cluster. The host resource is synced when it matches, or when the selector is empty.

Hooks (Pro)

Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

Use Hooks to:

• Patch any Kubernetes resource during synchronization and before resources are applied to the Kubernetes API server.
• Remove sensitive information before being written to the host or virtual cluster.

You can also add additional information to a resource, such as labels and annotations on all resources belonging to a tenant.

Hooks configuration specifies Kubernetes resources to patch. You can specify the resource group, API version, and resource kind and then the verbs for which API calls to patch. Configure patches using operations like add, remove, replace, and copyFromObject. See Node info remover for an example.

Examples

Find additional configuration examples in the vCluster repository's generic-sync-examples folder.

Certificate manager export

Create TLS certificates for websites and manage your own cert-manager Issuer. The RBAC-namespaced role and cluster-scoped clusterRole configure permissions.

The vCluster instance must run in multi-namespace mode so that it can:

• Utilize cert-manager's Issuer resource.
• Sync cert-manager's namespace-scoped Secret to the virtual cluster's namespace, so the virtual cluster's Ingress can use the Secret.

Follow these steps to use cert-manager:

1. Create an Issuer in the virtual cluster.
2. Create an Ingress in the virtual cluster that uses the Issuer through annotation: cert-manager.io/issuer: ${ISSUER_NAME}.
3. Verify that a Certificate resource has been created for the ingress.
4. Verify that a Secret with the name from spec.tls[0].secretName has been created and synced back to the vCluster with the generated certificate.
5. Apply the cert-manager configuration.

cert-manager configuration

experimental:
  multiNamespaceMode:
    enabled: true

  genericSync:
    role:
      extraRules:
        - apiGroups: ["cert-manager.io"]
          resources: ["issuers", "certificates"]
          verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
    clusterRole:
      extraRules:
        - apiGroups: ["apiextensions.k8s.io"]
          resources: ["customresourcedefinitions"]
          verbs: ["get", "list", "watch"]
    export:
      - apiVersion: cert-manager.io/v1
        kind: Issuer
      - apiVersion: cert-manager.io/v1
        kind: Certificate
    import:
      - kind: Secret
        apiVersion: v1

Istio export

Use Istio VirtualService and Gateway resources to expose vCluster services.

Follow these steps to use Istio:

1. Install Istio on your host cluster.
2. Deploy an Istio app in the virtual cluster. You can use Istio's BookInfo app if you don't have one.
3. Create the VirtualService and Gateway in the virtual cluster by applying the genericSync Istio configuration.
4. Verify that you can browse to the app's website using the host-installed Istio.

Istio configuration

# Setting these podLabels and coredns.podLabels should ensure that vcluster
# control plane pods won't have the sidecar injected, as this is unnecessary
# for most use cases
podLabels:
  sidecar.istio.io/inject: "false"
coredns:
  podLabels:
    sidecar.istio.io/inject: "false"


# Adding the --sync-labels argument to the syncer will propagate sidecar.istio.io/inject
# label from the workloads(pods) created in the vcluster to their representation in the
# host cluster, which would allow vcluster users to control the sidecar injection
# Comment out 3 lines below to enable this, and merge this with your other syncer values
# if you have some.
# syncer:
#   extraArgs:
#     - "--sync-labels=sidecar.istio.io/inject"

#
# Warning: current configuration supports VirtualService traffic only within the same namespace.
# Traffic to service.namespace.svc(.cluster.local) URLs will not be routed by Istio.
#
---
experimental:
  multiNamespaceMode:
    enabled: true

  genericSync:
    role:
      extraRules:
        - apiGroups: ["networking.istio.io"]
          resources: ["virtualservices", "destinationrules", "serviceentries", "gateways"]
          verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
    # extra cluster scoper permissions required for the plugin
    clusterRole:
      extraRules:
        - apiGroups: ["apiextensions.k8s.io"]
          resources: ["customresourcedefinitions"]
          verbs: ["get", "list", "watch"]
    export:
      - apiVersion: networking.istio.io/v1beta1
        kind: Gateway

      - apiVersion: networking.istio.io/v1beta1
        kind: VirtualService
        patches:
          - op: rewriteName
            path: .spec.gateways[*]
            regex: "($NAMESPACE/)?$NAME"
            conditions:
              - notEqual: "mesh"

      - apiVersion: networking.istio.io/v1beta1
        kind: DestinationRule
        patches:
          - op: rewriteName
            path: .spec.exportTo
            regex: $NAMESPACE
            conditions:
              - notEqual: "."
    import:
      - kind: Secret
        apiVersion: v1

Node info remover hooks (Pro)

Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

Configure hooks to hide Node images from virtual cluster users. After applying the configuration, you can test by running kubectl get nodes in the virtual cluster.

Node info remover configuration

genericSync:
  hooks:
    hostToVirtual:
    - apiVersion: v1
      kind: Node
      patches:
      - op: remove
        path: status.images
      - op: replace
        path: metadata.annotations
        value: {}
      - op: replace
        path: metadata.labels
        value: {}
      - fromPath: metadata.annotations["node.alpha.kubernetes.io/ttl"]
        op: copyFromObject
        path: metadata.annotations["node.alpha.kubernetes.io/ttl"]
      - fromPath: metadata.labels["kubernetes.io/arch"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/arch"]
      - fromPath: metadata.labels["kubernetes.io/hostname"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/hostname"]
      - fromPath: metadata.labels["kubernetes.io/os"]
        op: copyFromObject
        path: metadata.labels["kubernetes.io/os"]
      - fromPath: metadata.labels["beta.kubernetes.io/arch"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/arch"]
      - fromPath: metadata.labels["beta.kubernetes.io/hostname"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/hostname"]
      - fromPath: metadata.labels["beta.kubernetes.io/os"]
        op: copyFromObject
        path: metadata.labels["beta.kubernetes.io/os"]
      verbs: ["update", "patch", "create", "get", "list"]
    virtualToHost: []

patches reference

Bidirectional

op	Description
add	Add contents of the value into the path field. The value can be either scalar or a complex object.
copyFromObject	Copy value of the field referenced in the fromPath from the originating object to the path field of the destination object. The fromPath can be omitted, in such case, it will default to the same field path as referenced in the path.
remove	Remove the contents of the path field
replace	Replace the contents of the path field with the contents of the value. The value can be either scalar or a complex object.

Virtual to host

op	Description
rewriteLabelKey	The keys of the .metadata.labels of the synced resources are rewritten by vCluster and plugins. This patch type allows you to rewrite the key references in the same way, so the fields that are referencing labels will still reference correct labels in their rewritten form. For example, the label key-value pair "app: curl" is rewritten to "vcluster.loft.sh/label-vcluster-x-a172cedcae: curl", so with this patch operation you can rewrite a field that contains "app" to "vcluster.loft.sh/label-vcluster-x-a172cedcae, and the controllers operating on the synced resources will work with this label just as expected. This is not necessary when using the ["Multi-namespace mode"].(./multi_namespace_mode.mdx).
rewriteLabelSelector	This operation exists for the same reasons as described for the rewriteLabelKey operation. It is intended to be used for the key-value map fields that represent a label selector. This patch operation will rewrite all keys in the field referenced by path to the expected format for the label keys, and it will also add additional key-value pairs(with virtual namespace and vCluster name) to avoid naming conflicts. This is not necessary when using multi-namespace mode.
rewriteLabelExpressionsSelector	Similar to the rewriteLabelSelector, but expects path reference a field with the matchLabels and matchExpressions sub-fields, which will have the label keys rewritten just as described for rewriteLabelKey. This is not necessary when using multi-namespace mode.
rewriteName	Replaces the contents of the path field with transformed content based on the namespace of the synced resource. This is typically done on the fields that refer to a resource name, and on the .metadata.name as well(implicit). This is done to avoid naming collisions when syncing resources to the host cluster, but it is not necessary when using multi-namespace mode. As an example, the "logstash" value of a resource in the "logging" namespace of the vCluster named "vc" is rewritten to "logstash-x-logging-x-vc". If the resulting length of the value would be over 63 characters, the last 10 characters will be replaced with a hash of the full value.
rewriteName + namePath + namespacePath	Similar to rewriteName, but with an addition of the namePath and/or namespacePath. This is used when a field of the synced resource is referencing a different resource via namespace and name via two separate fields. When using this option you would set the path to reference a field that is a common parent of both namePath and namespacePath, and these two fields would then contain just the relative path. For example, path: spec.includes + namePath: name + namespacePath: namespace for a resource that contains name in spec.includes.name and namespace in spec.includes.namespace.
rewriteName + regex	Similar to rewriteName, but with an addition of the regex option for the patch. This is used when a string contains not just the resource name, but optionally a namespace, and other characters. For example, a string containing "namespace/name" can be correctly rewritten with the addition of this configuration option - regex: "$NAMESPACE/$NAME". The vCluster uses Go regular expressions to recognize the name part with the "NAME" capture group (can be written as $NAME), and the namespace with the "NAMESPACE" capture group (can be written as $NAMESPACE).

Config reference

genericSync required object pro

GenericSync holds options to generically sync resources from virtual cluster to host.

version required string pro

Version is the config version

export required object[] pro

Exports syncs a resource from the virtual cluster to the host

apiVersion required string pro

APIVersion of the object to sync

kind required string pro

Kind of the object to sync

optional required boolean pro

replaceOnConflict required boolean pro

ReplaceWhenInvalid determines if the controller should try to recreate the object if there is a problem applying

patches required object[] pro

Patches are the patches to apply on the virtual cluster objects when syncing them from the host cluster

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

reversePatches required object[] pro

ReversePatches are the patches to apply to host cluster objects after it has been synced to the virtual cluster

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

selector required object pro

Selector is a label selector to select the synced objects in the virtual cluster. If empty, all objects will be synced.

labelSelector required object pro

LabelSelector are the labels to select the object from

import required object[] pro

Imports syncs a resource from the host cluster to virtual cluster

apiVersion required string pro

APIVersion of the object to sync

kind required string pro

Kind of the object to sync

optional required boolean pro

replaceOnConflict required boolean pro

ReplaceWhenInvalid determines if the controller should try to recreate the object if there is a problem applying

patches required object[] pro

Patches are the patches to apply on the virtual cluster objects when syncing them from the host cluster

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

reversePatches required object[] pro

ReversePatches are the patches to apply to host cluster objects after it has been synced to the virtual cluster

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

hooks required object pro

Hooks are hooks that can be used to inject custom patches before syncing

hostToVirtual required object[] pro

HostToVirtual is a hook that is executed before syncing from the host to the virtual cluster

apiVersion required string pro

APIVersion of the object to sync

kind required string pro

Kind of the object to sync

verbs required string[] pro

Verbs are the verbs that the hook should mutate

patches required object[] pro

Patches are the patches to apply on the object to be synced

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

virtualToHost required object[] pro

VirtualToHost is a hook that is executed before syncing from the virtual to the host cluster

apiVersion required string pro

APIVersion of the object to sync

kind required string pro

Kind of the object to sync

verbs required string[] pro

Verbs are the verbs that the hook should mutate

patches required object[] pro

Patches are the patches to apply on the object to be synced

op required string pro

Operation is the type of the patch

fromPath required string pro

FromPath is the path from the other object

path required string pro

Path is the path of the patch

namePath required string pro

NamePath is the path to the name of a child resource within Path

namespacePath required string pro

NamespacePath is path to the namespace of a child resource within Path

value required object pro

Value is the new value to be set to the path

regex required string pro

Regex - is regular expresion used to identify the Name, and optionally Namespace, parts of the field value that will be replaced with the rewritten Name and/or Namespace

conditions required object[] pro

Conditions are conditions that must be true for the patch to get executed

path required string pro

Path is the path within the object to select

subPath required string pro

SubPath is the path below the selected object to select

equal required object pro

Equal is the value the path should be equal to

notEqual required object pro

NotEqual is the value the path should not be equal to

empty required boolean pro

Empty means that the path value should be empty or unset

ignore required boolean pro

Ignore determines if the path should be ignored if handled as a reverse patch

sync required object pro

Sync defines if a specialized syncer should be initialized using values from the rewriteName operation as Secret/Configmap names to be synced

secret required boolean pro

configmap required boolean pro

clusterRole required object pro

extraRules required object[] [] pro

role required object pro

extraRules required object[] [] pro