Skip to main content
Version: main 🚧

Custom Resource Definitions


Pro Feature

This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.

vCluster allows you to sync custom resources from the vCluster to the host cluster. This allows you to sync arbitrary resources that are by default not synced by vCluster. This only works for resources that have a custom resource definition in the host cluster.

If those custom resources will create other resources inside the host cluster, vCluster will try to find them and sync them back to the host cluster as well. E.g. a cert-manager certificate creates a secret which will be synced back automatically into the vCluster.

vCluster will automatically add the required cluster and namespace RBAC permissions for retrieving the custom resource definition and syncing the resources from the vCluster to the host cluster.


Only Namespace-Scoped Resource

This feature currently only works for namespace-scoped resources only.

Multi-Namespace-Mode

If you want to sync many custom resources, consider using multi-namespace-mode.

Enable Custom Resource Syncing​

To enable custom resource syncing, figure out what CRDs you want to sync via kubectl get crds. Copy the name, e.g. certificates.cert-manager.io and then configure the vcluster.yaml like this:

sync:
  toHost:
    customResourceDefinitions:
      certificates.cert-manager.io:
        enabled: true

Translate Patches​

You can modify the sync behaviour with translate patches that target specific paths. Currently there is 3 different kinds of patches supported.

Wildcard patches

You can use * in paths to select all entries of an array or object, e.g. spec.containers[*].name or spec.containers[*].volumeMounts[*]. vCluster will then just call the patch multiple times.

Reference patches​

Tell vCluster that this specific field points to a different resource that should get rewritten. vCluster will also automatically import the referenced resource if it can find it in the host cluster. E.g.:

sync:
  toHost:
    customResourceDefinitions:
      certificates.cert-manager.io:
        enabled: true
        translate:
        - path: spec.secretName
          reference:
            apiVersion: v1
            kind: Secret

This will tell vCluster to translate the path spec.secretName as it points to a secret. If the secret is created in the host cluster, vCluster will automatically import it.

Multi-Namespace-Mode

With multi-namespace-mode you only need to rewrite references that include a namespace. You can use the namespacePath option to specify the path of the namespace of the reference.

Labels patches​

Tell vCluster that this specific field is a label selector. This is needed to avoid conflicts between different namespaces and vClusters, so vCluster will add a label called vcluster.loft.sh/namespace to only select resources that are in the specific virtual namespace. To translate a labels field, you can add:

sync:
  toHost:
    customResourceDefinitions:
      my-resource.my-group.io:
        enabled: true
        translate:
        - path: spec.labelSelector
          labels: {}
Multi-Namespace-Mode

This is not needed in multi-namespace-mode as labels are not translated, so you can omit this.

JavaScript Expressions​

These are powerful JavaScript ES6 compatible expressions to change a field while syncing. You can define how it should changed when syncing from the vCluster into the host cluster or when syncing from the host cluster into the virtual cluster. To add a suffix to certificate dns names you can do:

sync:
  toHost:
    customResourceDefinitions:
      certificates.cert-manager.io:
        enabled: true
        translate:
        - path: spec.dnsNames[*]
          expression:
            toHost: '"my-prefix"+value'
            # optional fromHost, if omitted patches fromHost will be discarded
            # fromHost: 'value.slice("my-prefix".length)'

There is also a variable called context besides value that can be used to access vCluster specific data:

  • context.vcluster.name: Name of the vCluster
  • context.vcluster.namespace: Namespace of the vCluster
  • context.vcluster.config: Config of the vCluster, basically vcluster.yaml merged with the defaults
  • context.hostObject: Host object (can be null if not available)
  • context.virtualObject: Virtual object (can be null if not available)
  • context.path: The matched path on the object, useful when using wildcard path selectors (*)

Full Example: Using cert-manager custom resource in Single-Namespace-Mode (default)​

1. Install cert-manager on your host cluster​

Ensure that you have cert-manager installed and running on your host cluster. Use your preferred method of installing cert-manager.

2. Install vCluster​

Use following vcluster.yaml to create virtual cluster on your host. Save this file as vcluster.yaml

sync:
  toHost:
    # we want to rewrite ingress cert-manager.io/issuer annotations
    ingresses:
      enabled: true
      translate:
        - path: metadata.annotations["cert-manager.io/issuer"]
          reference:
            apiVersion: cert-manager.io/v1
            kind: Issuer

    customResourceDefinitions:
      # sync certificates
      certificates.cert-manager.io:
        enabled: true
        translate:
          - path: spec.secretName
            reference:
              apiVersion: v1
              kind: Secret
          - path: spec.issuerRef
            reference:
              apiVersion: cert-manager.io/v1
              kind: Issuer   # defaults to Issuer
              kindPath: kind # to also allow ClusterIssuer
              namePath: name
          - path: status.nextPrivateKeySecretName
            reference:
              apiVersion: v1
              kind: Secret
      # sync issuers
      issuers.cert-manager.io:
        enabled: true
        translate:
          - path: spec.acme.privateKeySecretRef.name
            reference:
              apiVersion: v1
              kind: Secret

  fromHost:
    customResourceDefinitions:
      # sync cluster issuers in read-only mode
      clusterissuers.cert-manager.io:
        enabled: true

And run:

vcluster create my-vcluster -f vcluster.yaml

After you started the vCluster with the above configuration you should see that the custom resource definitions have synced:

$ kubectl get customresourcedefinitions
NAME                             CREATED AT
certificates.cert-manager.io     2024-08-21T14:36:07Z
clusterissuers.cert-manager.io   2024-08-21T14:36:09Z
issuers.cert-manager.io          2024-08-21T14:36:08Z

3. Create Issuer and Certificate inside your virtual cluster​

We'll use a simple self signed certificate just to demonstrate vCluster capabilities. First, you'll need to create an Issuer resource:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test-selfsigned
spec:
  selfSigned: {}
kubectl apply -f issuer.yaml

And then Certificate that uses our test-selfsigned Issuer:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: test-cert
spec:
  secretName: test-cert-tls
  subject:
    organizations:
    - example.com
  commonName: example.com
  issuerRef:
    name: test-selfsigned
    kind: Issuer
kubectl apply -f cert.yaml

4. Validate that Secret was created inside your virtual cluster​

Thats it! You should have now test-cert-tls Secret available inside your virtual cluster! Just run:

kubectl get secret test-cert-tls

And you should see

NAME            TYPE                DATA   AGE
test-cert-tls   kubernetes.io/tls   3      2s

Full Example: Using cert-manager custom resource in Multi-Namespace-Mode​

1. Install cert-manager on your host cluster​

Ensure that you have cert-manager installed and running on your host cluster. Use your preferred method of installing cert-manager.

2. Install vCluster​

Use following vcluster.yaml to create virtual cluster on your host. Save this file as vcluster.yaml

experimental:
  multiNamespaceMode:
    enabled: true

sync:
  toHost:
    # sync all secrets
    secrets:
      all: true

    # sync ingresses and allow "cert-manager.io/issuer" annotation
    ingresses:
      enabled: true

    customResourceDefinitions:
      # sync cert-manager certificates
      certificates.cert-manager.io:
        enabled: true
      # sync cert-manager issuers
      issuers.cert-manager.io:
        enabled: true

  fromHost:
    customResourceDefinitions:
      # sync cert-manager cluster issuers
      clusterissuers.cert-manager.io:
        enabled: true

And run:

vcluster create my-vcluster -f vcluster.yaml

After you started the vCluster with the above configuration you should see that the custom resource definitions have synced:

$ kubectl get customresourcedefinitions
NAME                             CREATED AT
certificates.cert-manager.io     2024-08-21T14:36:07Z
clusterissuers.cert-manager.io   2024-08-21T14:36:09Z
issuers.cert-manager.io          2024-08-21T14:36:08Z

3. Create Issuer and Certificate inside your virtual cluster​

We'll use a simple self signed certificate just to demonstrate vCluster capabilities. First, you'll need to create an Issuer resource:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test-selfsigned
spec:
  selfSigned: {}
kubectl apply -f issuer.yaml

And then Certificate that uses our test-selfsigned Issuer:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: test-cert
spec:
  secretName: test-cert-tls
  subject:
    organizations:
    - example.com
  commonName: example.com
  issuerRef:
    name: test-selfsigned
    kind: Issuer
kubectl apply -f cert.yaml

4. Validate that Secret was created inside your virtual cluster​

Thats it! You should have now test-cert-tls Secret available inside your virtual cluster! Just run:

kubectl get secret test-cert-tls

And you should see

NAME            TYPE                DATA   AGE
test-cert-tls   kubernetes.io/tls   3      2s

Config reference​

customResourceDefinitions required {key: object} pro​

CustomResourceDefinitions defines what custom resource definitions should get synced from the virtual cluster to the host cluster.

enabled required boolean pro​

Enabled defines if this option should be enabled.

translate required object[] pro​

Translate the patch according to the given patches.

path required string pro​

Path is the path within the patch to target. If the path is not found within the patch, the patch is not applied.

expression required object pro​

Expression transforms the value according to the given JavaScript expression.

toHost required string pro​

ToHost is the expression to apply when retrieving a change from virtual to host.

fromHost required string pro​

FromHost is the patch to apply when retrieving a change from host to virtual.

reference required object pro​

Reference treats the path value as a reference to another object and will rewrite it based on the chosen mode automatically. In single-namespace mode this will translate the name to "vxxxxxxxxx" to avoid conflicts with other names, in multi-namespace mode this will not translate the name.

apiVersion required string pro​

APIVersion is the apiVersion of the referenced object.

apiVersionPath required string pro​

APIVersionPath is optional relative path to use to determine the kind. If APIVersionPath is not found, will fallback to apiVersion.

kind required string pro​

Kind is the kind of the referenced object.

kindPath required string pro​

KindPath is the optional relative path to use to determine the kind. If KindPath is not found, will fallback to kind.

namePath required string pro​

NamePath is the optional relative path to the reference name within the object.

namespacePath required string pro​

NamespacePath is the optional relative path to the reference namespace within the object. If omitted or not found, namespacePath equals to the metadata.namespace path of the object.

labels required object pro​

Labels treats the path value as a labels selector.