GitOps Deployment Installation

Learning objectives

note

Though this guide refers to ArgoCD, any GitOps tooling can be used to deploy vCluster Platform, and the general approach remains the same!

If you are already using GitOps practices with your applications deployed into your Kubernetes cluster(s), then you'll likely want to do the same with vCluster Platform! This guide will give you a quick rundown of the basics to get vCluster Platform deployed via GitOps. There is nothing inherently "special" or different about vCluster Platform compared to any other application, so general GitOps practices apply nicely to vCluster Platform!

Basic deployment

note

We recommend using Helm for GitOps deployments, so before beginning, it is worth a few minutes to take a look at the installation with helm section of the docs.

The most basic GitOps vCluster Platform deployment consists of a helm based Application with the vCluster Platform chart and your desired values.

A simple ArgoCD example, but you'll need to update $PLATFORM_VERSION with a valid vCluster Platform version.

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: vcluster-platform
  namespace: argocd
spec:
  destination:
    name: ""
    namespace: vcluster-platform
    server: "https://kubernetes.default.svc"
  source:
    path: ""
    repoURL: "https://charts.loft.sh"
    targetRevision: $PLATFORM_VERSION
    chart: vcluster-platform
    helm:
      parameters:
        # admin
        - name: admin.create
          value: "true"
        - name: admin.username
          value: admin
        - name: admin.password
          value: password

        # ingress
        - name: ingress.enabled
          value: "true"
        - name: ingress.host
          value: "vcluster-platform.example"
        - name: ingress.name
          value: "vcluster-platform-ingress"

        # audit
        - name: audit.enableSideCar
          value: "true"
        - name: config.audit.level
          value: "1"

        # config
        - name: config.loftHost
          value: "https://vcluster-platform.example"

  project: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

The above ArgoCD Application would deploy vCluster Platform into the local cluster, in the "vcluster-platform" namespace. In this example there are values being passed to set some basic configurations including:

• An admin user/password
• Ingress configuration
• Some basic audit configuration

The vCluster Platform Deployment itself will deploy the vCluster Platform agent into the local cluster for you. If you would like to manage the agent deployment yourself, see the agent section.

ArgoCD and Helm Releases

If you are administering your vCluster Platform instance with ArgoCD you will be unable to update the vCluster Platform config in the vCluster Platform UI. This should not be an issue as if you are managing vCluster Platform in a GitOps fashion you are (hopefully) avoiding manual updates like this, but it is important to note! This is due to ArgoCD not deploying the Helm release secret.

Connected Clusters

One of the great benefits of vCluster Platform is that you can easily manage resources located in many physical clusters by adding them to vCluster Platform and using vCluster Platform as your central point of management. When you add a "connected" cluster to vCluster Platform, a Cluster resource is created and a vCluster Platform Agent is installed in the cluster to handle local reconciliation tasks. If you are managing vCluster Platform via GitOps, you may also wish to manage these connected clusters in a similar fashion, rather than letting vCluster Platform install and manage the Agent.

Cluster Resources

If you would like to manage vCluster Platform and its agents via your GitOps tooling, you likely also want to manage the connected cluster configurations that live inside vCluster Platform.

These configuration elements inform vCluster Platform of

• The remotely connected clusters.
• How to connect to those clusters to validate the agent installation.
• How to proxy kubernetes commands from the central vCluster Platform instance to the remote cluster(s).

Each cluster requires two resources, a Cluster object that simply defines the cluster name and the owner of the cluster inside vCluster Platform, and an associated Secret that contains relevant configuration information for vCluster Platform to connect to the cluster.

A common strategy for bundling cluster data with your primary vCluster Platform Application (as in an ArgoCD Application object) is to write a simple "parent" Helm chart that includes vCluster Platform as a dependency. This parent chart can contain anything you'd like of course, but in this case would be used to include the Cluster objects and their associated Secret objects.

Cluster Secrets

The secret associated with a Cluster object necessarily includes authentication data to access the remote cluster so take care to ensure that this information is handled appropriately!

A simple Chart.yaml that includes the base vCluster Platform chart as a dependency may look similar to this, but you'll need to update $PLATFORM_VERSION with a valid vCluster Platform version.

apiVersion: v2
name: vcluster-platform-manager
description: A parent Helm chart for vCluster Platform
type: application
version: 1.0.0
dependencies:
  - name: vcluster-platform
    version: $PLATFORM_VERSION
    repository: https://charts.loft.sh

Values that need to be passed to the dependent vCluster Platform chart can be passed by referring to the dependency name, in this case, "vcluster-platform". For example, if you wanted to set the replicaCount value on the vCluster Platform chart, you could do as follows in a values.yaml file:

vcluster-platform:
  replicaCount: 3

Who is managing the agent?!

Remember, if you want to manage the vCluster Platform agent via your GitOps workflow, make sure you set the DISABLE_AGENT environment variable to "true" for your vCluster Platform deployment!

The "parent" Helm chart can now include any additional resources that you may want to deploy with your vCluster Platform instance. In this case, the chart should include both the Cluster and Secret resources for any connected clusters. You can accomplish this by having a simple template that iterates over an array of clusters that users can provide via values, something like the following:

{{ range .Values.clusters }}
---
apiVersion: management.loft.sh/v1
kind: Cluster
metadata:
  name: {{ .name }}
  annotations:
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  access:
  - subresources:
    - '*'
    users:
    - admin
    verbs:
    - '*'
  config:
    secretName: loft-cluster-config-{{ .name }}
    secretNamespace: vcluster-platform
  displayName: {{ .name }}
  owner:
    user: admin
{{ end }}

{{ range .Values.clusters }}
---
apiVersion: v1
data:
  config: {{ .config | b64enc }}
kind: Secret
metadata:
  name: loft-cluster-config-{{ .name }}
  namespace: vcluster-platform
type: Opaque
{{ end }}

With the above template, users would provide an array of maps containing a "name" and a "config" field. For example:

clusters:
  - name: my-connected-cluster
    config: |
      apiVersion: v1
      clusters:
      - cluster:
          certificate-authority-data: YOUR-CA-DATA-HERE
          server: https://1.2.3.4:6443
        name: my-connected-cluster
      contexts:
      - context:
          cluster: my-connected-cluster
          namespace: kube-system
          user: my-connected-cluster-user
        name: my-connected-cluster-context
      current-context: my-connected-cluster-context
      kind: Config
      preferences: {}
      users:
      - name: my-connected-cluster-user
        user:
          token: YOUR-TOKEN-HERE

Missing CRDs?!

You may have noticed a strange annotation on the Cluster resource above -- "argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true". This annotation tells ArgoCD to skip the dry run/check of the resource. We need this for initial deployments as vCluster Platform itself will deploy the Cluster CRD into the cluster. Without this annotation the Application will be unable to be synced!

Agents

If you have disabled agent installation on your vCluster Platform, you will need to ensure that you are installing the vCluster Platform agent into each connected cluster. Without doing this, vCluster Platform will not be able to do anything in the connected cluster!

The most obvious difference with managing the agents as compared to the vCluster Platform manager is that the agents will of course need to be installed in the connected "remote", as in not local to the vCluster Platform manager, cluster(s). After adding the relevant cluster(s) as available clusters in your ArgoCD configuration, you can simply create another ArgoCD Application to manage the agent.

Here is a basic example, but you'll need to update $PLATFORM_VERSION with a valid vCluster Platform version.

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: vcluster-platform-agent-my-other-cluster
  namespace: argocd
spec:
  destination:
    name: ""
    namespace: vcluster-platform
    server: "https://my-other-cluster:6443"
  source:
    path: ""
    repoURL: "https://charts.loft.sh"
    targetRevision: $PLATFORM_VERSION
    chart: vcluster-platform
    helm:
      parameters:
        # required parameter
        - name: agentOnly
          value: true

        # custom parameters
        - name: env.SOMEVAR
          value: my-value

  project: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Once again, note that if you want to manage the agent deployments via ArgoCD or your GitOps tooling of choice, you will want to ensure that the DISABLE_AGENT environment variable is set to "true" for your vCluster Platform deployment!

In Conclusion

One of the core tenets of vCluster Platform is that all vCluster Platform resources are just "normal" Kubernetes resources. This means that you can manage any vCluster Platform objects in the same way you'd manage any other Kubernetes resources in your GitOps workflow.

To generate the appropriate manifests to manage each vCluster Platform resource type, check out the API documentation where you can find example manifests and argument references for nearly all vCluster Platform resource types. Alternatively you can use the vCluster Platform UI to build out your required objects and simply snag the YAML output from the build pane and use that as your manifest content.

Here is an example of creating a Team, and a Project that the Team is a member of. You could manage these resources in your GitOps platform, then allow project users to create resources in vCluster Platform as they wish. This would put the core pieces under GitOps, freeing teams to control their own environments in vCluster Platform manually if they wish.

The following manifests could be added into a custom Helm chart, or used as manifests in a repo connected to ArgoCD. This is a contrived example, of course, but the main point here is that all vCluster Platform resources are just "normal" Kubernetes (custom) resources that can be managed with your GitOps tooling, or any other Kubernetes-centric tooling!

---
apiVersion: management.loft.sh/v1
kind: Team
metadata:
  name: acme-team
spec:
  displayName: acme-team
  owner:
    user: admin
  access:
    - verbs:
        - "*"
      subresources:
        - "*"
      users:
        - admin
    - name: vcluster-platform-access
      verbs:
        - get
        - bind
      subresources:
        - clusters
      teams:
        - acme-team

---
kind: Project
apiVersion: management.loft.sh/v1
metadata:
  name: acme-team-project
spec:
  displayName: acme-team-project
  owner:
    user: admin
  quotas: {}
  allowedClusters:
    - name: "*"
  allowedTemplates:
    - kind: VirtualClusterTemplate
      group: storage.loft.sh
      name: "*"
    - kind: SpaceTemplate
      group: storage.loft.sh
      name: "*"
  members:
    - kind: Team
      group: storage.loft.sh
      name: acme-team
      clusterRole: loft-management-project-admin
  access:
    - name: vcluster-platform-admin-access
      verbs:
        - get
        - update
        - patch
        - delete
      subresources:
        - "*"
      users:
        - admin
      teams:
        - acme-team