Plugins

Don't install untrusted plugins

A plugin runs with the same permissions as vCluster itself does in the Kubernetes cluster. Plugins can also define additional permissions, so make sure you only install plugins you trust.

Plugins overview

Plugins extend the capabilities of vCluster. They let you to add custom functionality, such as:

1. Syncing specific resources from or to the virtual clusters, including cluster scoped resources like cluster roles
2. Syncing custom resources from or to the virtual cluster
3. Deploying resources on virtual cluster startup, such as CRDs, applications, etc.
4. Manage resources and applications inside the host or virtual cluster
5. Enforcing certain restrictions on synced resources or extending the existing syncers of vCluster
6. Any other operator use case that could benefit from having access to the virtual cluster and the host cluster simultaneously.

A plugin in its purest form is a Kubernetes operator that has access to both the virtual cluster and the host cluster simultaneously. This is the main difference between a vCluster plugin and a regular Kubernetes operator that you would install inside the virtual cluster itself. Given this dual access, the plugin is able to translate resources between both clusters, which is the basic building block of how vCluster works.

Recommended Reads

In order to better understand how vCluster plugins work, read about Kubernetes operators and controllers.

Architecture

Each plugin runs as a binary inside the vCluster syncer container. It is by default copied over from an init container. This is done to allow easier communication between vCluster and the plugins as well as provide certain capabilities such as high-availability out of the box. The plugin itself uses go-plugin, which is used by many popular open-source projects to extend their functionality. The vCluster syncer calls the plugin binary during startup and passes relevant information to the plugin. The plugin controllers are then started with these credentials, similar to how vCluster itself starts its resource syncers.

Plugin controllers

Resource syncing enables the virtual cluster to behave like an actual Kubernetes cluster. A Kubernetes controller that is responsible for resource syncing in vCluster is called a syncer. This controller reacts on changes to objects within the virtual cluster and on changes to objects within the host cluster. The syncer tries to map each virtual object to a physical object in the host cluster and then compares those. After it discovers a change, the syncer ensures that the virtual cluster object and the physical cluster object are aligned in the desired state, and if not, the syncer changes either one of those objects to reflect the desired state.

Each plugin can define several of those resource syncers that would work exactly like the built-in syncers of vCluster. However, you don't need to sync every Kubernetes resource to the host cluster, as some can stay purely virtual. Only resources that influence the workloads need to be synced, for example, pods, services, and endpoints, while others such as deployments, replicasets, namespaces etc. are only relevant to the Kubernetes control plane and hence are not needed in the host cluster.

There are sometimes also cases where you want to manage specific core resources yourself without interfering with what vCluster is syncing, for example special secrets or configmaps that were created from the host cluster or a different resource inside the host cluster. For this use case you can label resources vCluster should ignore either on the physical or virtual side with a label vcluster.loft.sh/controlled-by and a custom value of your choosing. This tells vCluster to ignore the resource in its syncers.

Plugin hooks

Plugin hooks are a great feature to adjust current syncing behaviour of vCluster without the need to override an already existing syncer in vCluster completely. They allow you to change outgoing objects of vCluster similar to an mutating admission controller in Kubernetes. Requirement for an hook to work correctly is that vCluster itself would sync the resource, so hooks only work for the core resources that are synced by vCluster such as pods, services, secrets etc.

If a plugin registers a hook to a specific resource, vCluster forwards all requests that match the plugin's defined hooks to the plugin and the plugin can then adjust or even deny the request completely. This opens up a wide variety of adjustment possibilities for plugins, where you for example only want to add a custom label or annotation.

Plugin SDK

Recommended Reads

If you want to start developing your own vCluster plugins, read about controller-runtime and kube builder, which uses the controller runtime internally.

vCluster provides an SDK for writing plugin controllers that abstracts a lot of the syncer complexity away from the user, but still gives you access to the underlying structures if you need it. Internally, the vCluster SDK uses the controller-runtime project, which is used by vCluster itself to create the controllers. The vCluster SDK lets you write custom plugin controllers with just a few lines of code.

Since the plugin SDK interfaces are mostly compatible with the vCluster syncers, you can also take a look at how those are implemented in the vCluster itself, which work in most cases the same way as if those would be implemented in a plugin. It would be even possible to reimplement all vCluster syncers in a separate plugin.

Most of the settings are for the init container from which vCluster copies the plugin. Adjust controlPlane.statefulSet.resources configuration to ensure the vCluster deployment has enough CPU and memory to also run the plugin.

note

• plugins[*].resources does not change the syncer's resource requests.
• plugins[*].config content ends up in an environment variable PLUGIN_CONFIG that is read by the plugin. This configuration has no specific format.

Config reference

plugins required {key: object} pro

Define which vCluster plugins to load.

name required string pro

Name is the name of the init-container and NOT the plugin name

image required string pro

Image is the container image that should be used for the plugin

imagePullPolicy required string pro

ImagePullPolicy is the pull policy to use for the container image

config required object pro

Config is the plugin config to use. This can be arbitrary config used for the plugin.

rbac required object pro

RBAC holds additional rbac configuration for the plugin

role required object pro

Role holds extra virtual cluster role permissions for the plugin

extraRules required object[] pro

ExtraRules are extra rbac permissions roles that will be added to role or cluster role

verbs required string[] pro

Verbs is a list of Verbs that apply to ALL the ResourceKinds contained in this rule. '*' represents all verbs.

apiGroups required string[] pro

APIGroups is the name of the APIGroup that contains the resources. If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed. "" represents the core API group and "*" represents all API groups.

resources required string[] pro

Resources is a list of resources this rule applies to. '*' represents all resources.

resourceNames required string[] pro

ResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.

nonResourceURLs required string[] pro

NonResourceURLs is a set of partial urls that a user should have access to. *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"), but not both.

clusterRole required object pro

ClusterRole holds extra virtual cluster cluster role permissions required for the plugin

extraRules required object[] pro

ExtraRules are extra rbac permissions roles that will be added to role or cluster role

verbs required string[] pro

Verbs is a list of Verbs that apply to ALL the ResourceKinds contained in this rule. '*' represents all verbs.

apiGroups required string[] pro

APIGroups is the name of the APIGroup that contains the resources. If multiple API groups are specified, any action requested against one of the enumerated resources in any API group will be allowed. "" represents the core API group and "*" represents all API groups.

resources required string[] pro

Resources is a list of resources this rule applies to. '*' represents all resources.

resourceNames required string[] pro

ResourceNames is an optional white list of names that the rule applies to. An empty set means that everything is allowed.

nonResourceURLs required string[] pro

NonResourceURLs is a set of partial urls that a user should have access to. *s are allowed, but only as the full, final step in the path Since non-resource URLs are not namespaced, this field is only applicable for ClusterRoles referenced from a ClusterRoleBinding. Rules can either apply to API resources (such as "pods" or "secrets") or non-resource URL paths (such as "/api"), but not both.

command required string[] pro

Command is the command that should be used for the init container

args required string[] pro

Args are the arguments that should be used for the init container

securityContext required object pro

SecurityContext is the container security context used for the init container

resources required object pro

Resources are the container resources used for the init container

volumeMounts required object[] pro

VolumeMounts are extra volume mounts for the init container