Skip to main content

Overview

Plugins are a feature to extend the capabilities of vCluster. They allow you to add custom functionality, such as:

  1. Syncing specific resources from or to the virtual clusters, including cluster scoped resources like cluster roles
  2. Syncing custom resources from or to the virtual cluster
  3. Deploying resources on virtual cluster startup, such as CRDs, applications, etc.
  4. Manage resources and applications inside the host or virtual cluster
  5. Enforcing certain restrictions on synced resources or extending the existing syncers of vCluster
  6. Any other operator use case that could benefit from having access to the virtual cluster and the host cluster simultaneously.

A plugin in its purest form is a Kubernetes operator that will have access to both the virtual cluster and the host cluster simultaneously. This is the main difference between a vCluster plugin and a regular Kubernetes operator that you would just install inside the vCluster itself. Given this dual access, the plugin is able to translate resources between both clusters, which is the basic building block of how vCluster works.

Recommended Reads

In order to better understand how vCluster plugins work, it is recommended to read about Kubernetes operators as well as controllers.

Architecture​

Each plugin will run as a binary inside the vCluster syncer container. It is by default copied over from an init container. This is done to allow easier communication between vCluster and the plugins as well as provide certain capabilities such as high-availability out of the box. The plugin itself uses go-plugin, which is used by many popular open-source projects to extend their functionality. The vCluster syncer container will call the plugin binary during startup and pass all relevant information to the plugin. The plugin controllers are then started with these credentials, similar to how vCluster itself starts its resource syncers.

Plugin Controllers​

Resource syncing is the heart of vCluster which enables the virtual cluster to behave like an actual Kubernetes cluster. A Kubernetes controller that is responsible for resource syncing in vCluster is called a syncer. This controller reacts on changes to objects within the virtual cluster and on changes to objects within the host cluster. The syncer tries to map each virtual object to a physical object in the host cluster and then compares those. After it discovers a change, the syncer ensures that the virtual cluster object and the physical cluster object are aligned in the desired state, and if not, the syncer changes either one of those objects to reflect the desired state.

Each plugin can define several of those resource syncers that would work exactly like the built-in syncers of vCluster. However, you'll not need to sync every Kubernetes resource to the host cluster, as some can stay purely virtual. Only resources that influence the workloads need to be synced, for example, pods, services, and endpoints, while others such as deployments, replicasets, namespaces etc. are only relevant to the Kubernetes control plane and hence are not needed in the host cluster.

There are sometimes also cases where you want to manage specific core resources yourself without interfering with what vCluster is syncing, for example special secrets or configmaps that were created from the host cluster or a different resource inside the host cluster. For this use case you can label resources vCluster should ignore either on the physical or virtual side with a label vcluster.loft.sh/controlled-by and a custom value of your choosing. This will tell vCluster to ignore the resource in its syncers.

Plugin Hooks​

Plugin hooks are a great feature to adjust current syncing behaviour of vCluster without the need to override an already existing syncer in vCluster completely. They allow you to change outgoing objects of vCluster similar to an mutating admission controller in Kubernetes. Requirement for an hook to work correctly is that vCluster itself would sync the resource, so hooks only work for the core resources that are synced by vCluster such as pods, services, secrets etc.

If a plugin registers a hook to a specific resource, vCluster will forward all requests that match the plugin's defined hooks to the plugin and the plugin can then adjust or even deny the request completely. This opens up a wide variety of adjustment possibilities for plugins, where you for example only want to add a custom label or annotation.

Plugin SDK​

Recommended Reads

If you want to start developing your own vCluster plugins, it is recommended that you read about controller-runtime as well as kube builder that uses the controller runtime internally.

vCluster provides an SDK for writing plugin controllers that abstracts a lot of the syncer complexity away from the user, but still gives you access to the underlying structures if you need it. Internally, the vCluster SDK uses the popular controller-runtime project, that is used by vCluster itself to create the controllers. The vCluster SDK lets you write custom plugin controllers with just a few lines of code.

Since the plugin SDK interfaces are mostly compatible with the vCluster syncers, you can also take a look at how those are implemented in the vCluster itself, which work in most cases the same way as if those would be implemented in a plugin. It would be even possible to reimplement all vCluster syncers in a separate plugin.

Loading and Installing Plugins to vCluster​

Since the most common distribution method of vCluster is helm, plugins are also configured via helm values. If you develop a plugin of your own, we recommend creating a plugin.yaml (the name has no special functionality, you could also name it my-plugin.yaml or extra-values.yaml) in the following format:

# Plugin Definition below. This is essentially a valid helm values file that will be merged
# with the other vCluster values during vCluster create or helm install.
plugin:
  myPlugin:
    # Version v2 is required to use the new plugin api
    version: v2
    image: plugin-image
    # Specify config that can be parsed within the plugin
    #config:
    # myKey: myValue
    # myConfigObj:
    #   otherKey: otherValue
    # Configure Extra RBAC Rules like this
    #rbac:
    #  role:
    #    extraRules:
    #      - apiGroups: ["example.loft.sh"]
    #        ...
    #  clusterRole:
    #    extraRules:
    #      - apiGroups: ["apiextensions.k8s.io"]
    #        ...

The plugin.yaml is a valid helm values file used to define the plugin's init container configuration and additional RBAC rules needed to function properly. If you want to distribute that plugin for others, it's also possible to install a plugin through an URL:

# Install a plugin with a local plugin.yaml
vcluster create my-vcluster -n my-vcluster -f plugin.yaml -f other-values.yaml

# Install a plugin with a remote URL
vcluster create my-vcluster -n my-vcluster -f https://github.com/my-org/my-plugin/plugin.yaml -f other-values.yaml

# Install a plugin with helm with a remote URL
helm install my-vcluster vcluster -n my-vcluster --repo https://charts.loft.sh -f https://github.com/my-org/my-plugin/plugin.yaml -f other-values.yaml
Examples

You can take a look at the vcluster-sdk repo for some working examples.

Don't install untrusted plugins

A plugin runs with the same permissions as vCluster itself does in the Kubernetes cluster and can also define additional permissions through its plugin.yaml, so make sure you only install plugins you trust.