Custom Resources
This feature is available in the vCluster Pro tier. Contact us for more details and to start a trial.
vCluster allows you to sync custom resources from the virtual cluster to the host cluster. This allows you to sync arbitrary resources that are by default not synced by vCluster. This only works for resources that have a custom resource definition in the host cluster.
If those custom resources will create other resources inside the host cluster, vCluster will try to find them and sync them back to the host cluster as well. E.g. a cert-manager certificate creates a secret which will be synced back automatically into the virtual cluster.
vCluster will automatically add the required cluster and namespace RBAC permissions for retrieving the custom resource definition and syncing the resources from the virtual cluster to the host cluster.
This feature currently only works for namespace-scoped resources only.
If you want to sync many custom resources, consider using multi-namespace-mode.
Enable Custom Resource Syncing​
To enable custom resource syncing from the virtual cluster to the host cluster, figure out what CRDs you want to sync via kubectl get crds
. Add the name into the customResources
section in the sync section. Even though vCluster syncs custom resources from the virtual cluster to the host cluster, the CRDs are also copied from the host cluster to the virtual cluster.
sync:
toHost:
customResources:
certificates.cert-manager.io:
enabled: true
Patches​
You can modify the sync behaviour with patches that target specific paths. Currently there is 2 different kinds of patches supported.
You can use *
in paths to select all entries of an array or object, e.g. spec.containers[*].name
or spec.containers[*].volumeMounts[*]
. vCluster calls the patch multiple times.
Reference patches​
A reference patch can be used to have a specific field of one resource point to a different resource that should get rewritten. vCluster automatically imports the referenced resource to the virtual cluster if it can find it in the host cluster. For example:
sync:
toHost:
customResources:
certificates.cert-manager.io:
enabled: true
patches:
- path: spec.secretName
reference:
apiVersion: v1
kind: Secret
vCluster translates the path spec.secretName
as it points to a secret. If the secret is created in the host cluster, vCluster automatically imports it into the virtual cluster.
With multi-namespace-mode you only need to rewrite references that include a namespace. You can use the namespacePath
option to specify the path of the namespace of the reference.
JavaScript Expression Patches​
These are powerful JavaScript ES6 compatible expression patches that can be used to change a field while syncing. You define how it changes when syncing from the virtual cluster into the host cluster or when syncing from the host cluster into the virtual cluster. To add a suffix to certificate DNS names you can do:
sync:
toHost:
customResources:
certificates.cert-manager.io:
enabled: true
patches:
- path: spec.dnsNames[*]
# specifies the sync direction here again, because you can also react on change for fromHost with an expression
expression: '"www."+value'
# optional reverseExpression, if omitted patches from host will be discarded for that path
# reverseExpression: 'value.slice("my-prefix".length)'
There is also a variable called context
besides value
that can be used to access vCluster specific data:
context.vcluster.name
: Name of the virtual clustercontext.vcluster.namespace
: Namespace of the virtual clustercontext.vcluster.config
: Config of the virtual cluster, basicallyvcluster.yaml
merged with the defaultscontext.hostObject
: Host object (can be null if not available)context.virtualObject
: Virtual object (can be null if not available)context.path
: The matched path on the object, useful when using wildcard path selectors (*)
For example, let's assume you want to add www.
to every DNS name specified in a cert-manager certificate in the path spec.dnsNames
, you could use the following patch:
sync:
toHost:
customResources:
certificates.cert-manager.io:
enabled: true
patches:
- path: spec.dnsNames[*]
# specifies the sync direction here again, because you can also react on change for fromHost
expression: "value.startsWith('www.') ? value : `www.${value}`"
# specifies how to sync back changes to the virtual cluster. If omitted will not sync back changes.
reverseExpression: "value.startsWith('www.') ? value.slice(4) : value"
With that patch, creating a new certificate within the vCluster like this:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: name-within-vcluster
spec:
dnsNames:
- example.com
Would create the following certificate in the host cluster:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: synced-name
spec:
dnsNames:
- www.example.com # the patch added www. to this field
When you change now the certificate in the host cluster like this:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: synced-name
spec:
dnsNames:
- www.other-domain.com # changed from www.example.com
vCluster would sync back the certificate like this:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: name-within-vcluster
spec:
dnsNames:
- other-domain.com # the patch removed the www. from www.other-domain.com
Full Example: Using cert-manager custom resource in Single-Namespace-Mode (default)​
1. Install cert-manager on your host cluster​
Ensure that you have cert-manager installed and running on your host cluster. Use your preferred method of installing cert-manager.
2. Install vCluster​
Use following vcluster.yaml
to create virtual cluster on your host. Save this file as vcluster.yaml
sync:
toHost:
# we want to rewrite ingress cert-manager.io/issuer annotations
ingresses:
enabled: true
patches:
- path: metadata.annotations["cert-manager.io/issuer"]
reference:
apiVersion: cert-manager.io/v1
kind: Issuer
customResources:
# sync certificates
certificates.cert-manager.io:
enabled: true
patches:
- path: spec.secretName
reference:
apiVersion: v1
kind: Secret
- path: spec.issuerRef
reference:
apiVersion: cert-manager.io/v1
kind: Issuer # defaults to Issuer
kindPath: kind # to also allow ClusterIssuer
namePath: name
- path: status.nextPrivateKeySecretName
reference:
apiVersion: v1
kind: Secret
# sync issuers
issuers.cert-manager.io:
enabled: true
patches:
- path: spec.acme.privateKeySecretRef.name
reference:
apiVersion: v1
kind: Secret
fromHost:
customResources:
# sync cluster issuers in read-only mode
clusterissuers.cert-manager.io:
enabled: true
And run:
vcluster create my-vcluster -f vcluster.yaml
After you started the vCluster with the above configuration you should see that the custom resource definitions have synced:
$ kubectl get customresourcedefinitions
NAME CREATED AT
certificates.cert-manager.io 2024-08-21T14:36:07Z
clusterissuers.cert-manager.io 2024-08-21T14:36:09Z
issuers.cert-manager.io 2024-08-21T14:36:08Z
3. Create Issuer
and Certificate
inside your virtual cluster​
We'll use a simple self signed certificate just to demonstrate vCluster capabilities. First, you'll need to create an Issuer
resource:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: test-selfsigned
spec:
selfSigned: {}
kubectl apply -f issuer.yaml
And then Certificate
that uses our test-selfsigned
Issuer
:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: test-cert
spec:
secretName: test-cert-tls
subject:
organizations:
- example.com
commonName: example.com
issuerRef:
name: test-selfsigned
kind: Issuer
kubectl apply -f cert.yaml
4. Validate that Secret
was created inside your virtual cluster​
Thats it! You should have now test-cert-tls
Secret available inside your virtual cluster! Just run:
kubectl get secret test-cert-tls
And you should see
NAME TYPE DATA AGE
test-cert-tls kubernetes.io/tls 3 2s
Full Example: Using cert-manager custom resource in Multi-Namespace-Mode​
1. Install cert-manager on your host cluster​
Ensure that you have cert-manager installed and running on your host cluster. Use your preferred method of installing cert-manager.
2. Install vCluster​
Use following vcluster.yaml
to create virtual cluster on your host. Save this file as vcluster.yaml
experimental:
multiNamespaceMode:
enabled: true
sync:
toHost:
# sync all secrets
secrets:
all: true
# sync ingresses and allow "cert-manager.io/issuer" annotation
ingresses:
enabled: true
customResources:
# sync cert-manager certificates
certificates.cert-manager.io:
enabled: true
# sync cert-manager issuers
issuers.cert-manager.io:
enabled: true
fromHost:
customResources:
# sync cert-manager cluster issuers
clusterissuers.cert-manager.io:
enabled: true
And run:
vcluster create my-vcluster -f vcluster.yaml
After you started the vCluster with the above configuration you should see that the custom resource definitions have synced:
$ kubectl get customresourcedefinitions
NAME CREATED AT
certificates.cert-manager.io 2024-08-21T14:36:07Z
clusterissuers.cert-manager.io 2024-08-21T14:36:09Z
issuers.cert-manager.io 2024-08-21T14:36:08Z
3. Create Issuer
and Certificate
inside your virtual cluster​
We'll use a simple self signed certificate just to demonstrate vCluster capabilities. First, you'll need to create an Issuer
resource:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: test-selfsigned
spec:
selfSigned: {}
kubectl apply -f issuer.yaml
And then Certificate
that uses our test-selfsigned
Issuer
:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: test-cert
spec:
secretName: test-cert-tls
subject:
organizations:
- example.com
commonName: example.com
issuerRef:
name: test-selfsigned
kind: Issuer
kubectl apply -f cert.yaml
4. Validate that Secret
was created inside your virtual cluster​
Thats it! You should have now test-cert-tls
Secret available inside your virtual cluster! Just run:
kubectl get secret test-cert-tls
And you should see
NAME TYPE DATA AGE
test-cert-tls kubernetes.io/tls 3 2s
Config reference​
customResources
required {key: object} pro​
CustomResources defines what custom resources should get synced from the virtual cluster to the host cluster. vCluster will copy the definition automatically from host cluster to virtual cluster on startup.
vCluster will also automatically add any required RBAC permissions to the vCluster role for this to work.
customResources
required {key: object} pro​enabled
required boolean pro​
Enabled defines if this option should be enabled.
enabled
required boolean pro​patches
required object[] pro​
Patches patch the resource according to the provided specification.
patches
required object[] pro​path
required string pro​
Path is the path within the patch to target. If the path is not found within the patch, the patch is not applied.
path
required string pro​expression
required string pro​
Expression transforms the value according to the given JavaScript expression.
expression
required string pro​reverseExpression
required string pro​
ReverseExpression transforms the value according to the given JavaScript expression.
reverseExpression
required string pro​reference
required object pro​
Reference treats the path value as a reference to another object and will rewrite it based on the chosen mode
automatically. In single-namespace mode this will translate the name to "vxxxxxxxxx" to avoid conflicts with
other names, in multi-namespace mode this will not translate the name.
reference
required object pro​apiVersion
required string pro​
APIVersion is the apiVersion of the referenced object.
apiVersion
required string pro​apiVersionPath
required string pro​
APIVersionPath is optional relative path to use to determine the kind. If APIVersionPath is not found, will fallback to apiVersion.
apiVersionPath
required string pro​kind
required string pro​
Kind is the kind of the referenced object.
kind
required string pro​kindPath
required string pro​
KindPath is the optional relative path to use to determine the kind. If KindPath is not found, will fallback to kind.
kindPath
required string pro​namePath
required string pro​
NamePath is the optional relative path to the reference name within the object.
namePath
required string pro​namespacePath
required string pro​
NamespacePath is the optional relative path to the reference namespace within the object. If omitted or not found, namespacePath equals to the
metadata.namespace path of the object.
namespacePath
required string pro​labels
required object pro​
Labels treats the path value as a labels selector.
labels
required object pro​