Skip to main content
Version: main 🚧

Database connector

Free, Dev, Prod, and Scale are vCluster Platform license plans. Open source does not need a license or a Platform connection. See Compare open source and free tiers.

info
This feature is available from the Platform version v4.1.0 and was introduced in vCluster version v0.21.0

The database connector feature allows an admin to configure a database server to manage backing stores for multiple clusters. For each cluster, the feature automatically creates an independent database and non-privileged user. Read more about the use of external databases in the External Database documentation.

Compatibility matrix​

The database connector works with PostgreSQL and MySQL compatible DBMS. While especially the managed database offerings from hyperscalers sometimes don't expose full admin privileges, ship with a non-default feature set, or use special configuration, it isn't possible to guarantee compatibility with all of them. The following table outlines the current support status:

Database typeEnvironmentSupportedTested versions
PostgreSQLSelf-hosted / upstreamv14, v15, v16, v17, v18
AWS Aurora PostgreSQLv14, v15, v16, v17
AWS RDS PostgreSQLv14, v15, v16, v17, v18
Azure Database for PostgreSQL flexible serverv14, v15, v16, v17, v18
Google Cloud SQL PostgreSQLv14, v15, v16, v17, v18
MySQLSelf-hosted / upstreamv5, v8, v9
AWS Aurora MySQLv5, v8, v9
AWS RDS MySQLv5, v8, v9
Azure Database for MySQL flexible server️1v5, v8, v9
Google Cloud SQL MySQLv5, v8, v9
MariaDBAWS RDS MariaDBv10
v11 ️1
v10, v11

1 Needs explicit configuration to allow unencrypted transport: require_secure_transport set to OFF.

info

While other, not mentioned, PostgreSQL/MySQL compatible databases may work, they haven't been tested.

How cloud database offerings are tested​

The following sections describe how each hyperscaler database offering is tested, including any notable deviations from a default deployment.

AWS

Tested with major versions v14, v15, v16, and v17.

  • Instance class: db.t3.medium
  • High availability: Single-instance Aurora cluster (no read replicas)
Azure

Tested with major versions v14, v15, v16, v17, and v18.

  • SKU / tier: Burstable, Standard_B1ms
  • High availability: No (single server)
GCP

Tested with major versions v14, v15, v16, v17, and v18. Google Cloud SQL PostgreSQL requires vCluster Platform v4.7 or later for all versions.

  • Edition / tier: Enterprise, db-f1-micro
  • High availability: No (single-zone)
  • User setup: A dedicated database user is created because Cloud SQL doesn't expose the postgres superuser with full privileges.

Prerequisites​

  • Administrator access to a Kubernetes cluster: See Accessing Clusters with kubectl for more information. Your current kube-context must have administrative privileges, which you can verify with kubectl auth can-i create clusterrole -A

    info

    To obtain a kube-context with admin access, ensure you have the necessary credentials and permissions for your Kubernetes cluster. This typically involves using kubectl config commands or authenticating through your cloud provider's CLI tools.

  • helm installed: Helm v3.10.0+ is required for deploying the platform. Refer to the Helm Installation Guide if you need to install it.

  • kubectl installed: Kubernetes command-line tool for interacting with the cluster. See Install and Set Up kubectl for installation instructions.

  • A running PostgreSQL or MySQL compatible DBMS and admin credentials to the server

Configuration​

The UI, the Connector API, and direct Secret writes configure the same database connector. Use a Platform management API kubeconfig for Connector resources, or the control plane cluster's kubeconfig for Secrets.

Create in the UI​

  1. Go to Secrets & Credentials > Connectors, and select the Databases tab if it's shown.

  2. Click .

  3. In the Connect External Database form, select the database type, and enter a name and the server's endpoint, port, admin user, and password.

  4. Click .

Create through the Connector API​

Save this manifest as database-connector.yaml. Set the database endpoint, admin user, and password for your server:

Modify the following with your specific values to generate a copyable command:
apiVersion: management.loft.sh/v1
kind: Connector
metadata:
name: prod-db
spec:
displayName: Production database
# Select PostgreSQL or MySQL explicitly.
sharedDatabase:
dialect: postgres
endpoint: postgres.example.com
port: "5432"
user: platform-admin
password: "replace-with-database-password"

Create the resource and check its configuration:

Modify the following with your specific values to generate a copyable command:
kubectl create -f database-connector.yaml
kubectl get connectors.management.loft.sh prod-db -o yaml

The dialect is required (postgres or mysql). On create, an omitted port defaults to 5432 or 3306 respectively. caCert, sslMode, and identityProvider live under sharedDatabase.

Configured=True confirms payload validation. It doesn't establish database connectivity. The cluster's DBConnectorSynced condition reports provisioning results.

Create a backing Secret directly​

A connector secret is a secret that can be used with a platform feature or integration.

For the platform to start using a database server, a secret must be created that contains admin credentials for the server. The secret must:

  • Contain the loft.sh/connector-type label with the shared-database value
  • Reside in the same namespace as the platform
  • Contain the fields:
    • endpoint
    • password
    • port
    • user

Apply a Secret YAML manifest to the platform's control plane cluster.

Shared database secret template
apiVersion: v1
kind: Secret
metadata:
name: <name> # this is a normal metadata name. A descriptive name is recommended. This is the name a cluster's config uses to reference the connector
namespace: <vcluster-platform-namespace> # this is the namespace that vCluster Platform is installed in. "vcluster-platform" and "loft" are common values.
labels:
loft.sh/connector-type: "shared-database"
stringData:
endpoint: <endpoint> # this could be a service endpoint or an external hostname endpoint
password: <password>
port: <port> # 3306 is a common port for database servers
user: <user>

Use the following kubectl command to create the secret:

Create a shared database connector secret
kubectl create secret generic default-secret-name \
--namespace=vcluster-platform \
--from-literal=endpoint="your-database-endpoint" \
--from-literal=password="your-database-password" \
--from-literal=port="3306" \
--from-literal=user="your-database-user" \
-o yaml --dry-run=client > secret.yaml

Before a tenant uses the connector, grant connector access. This applies however you created the connector.

The list-only DatabaseConnector API is deprecated in place. Existing Secrets and consumers remain supported. See Migrate database discovery.

Use cluster with shared database​

A new cluster can now reference the connector by setting the controlPlane.backingStore.database.external.connector field in the cluster config to the connector name.

warning

Modifying a database secret to point to a different database server isn't recommended and may lead to data loss.

Example configuration​

The following example shows a connector secret for the shared database feature. It connects to an internal MySQL instance exposed by a service named mysql in the vcluster-platform namespace (where the platform is assumed to be installed). You can also use any other MySQL-compatible endpoint, such as an AWS RDS instance.

Shared database secret example
apiVersion: v1
kind: Secret
metadata:
name: default-data-source
namespace: vcluster-platform # This must match the namespace that vCluster Platform is in.
labels:
loft.sh/connector-type: "shared-database"
stringData:
endpoint: mysql.vcluster-platform # This service can be in any namespace.
password: <password>
port: "3306"
user: root
type: mysql # This can be mysql or postgres. If left blank, the mysql database type is assumed.

PostgreSQL TLS​

info
This feature is available from the Platform version v4.10.0

Starting with platform v4.10, the PostgreSQL admin connection uses sslmode=require as the minimum. Previously the connection silently fell back to plaintext when the server refused TLS. Two optional fields on the connector secret tune this behavior:

FieldEffect
caCertPEM-encoded CA bundle. When set, the connection uses sslmode=verify-full and the server certificate is validated against this CA.
sslModeExplicit override (disable, require, verify-full, and so on). Wins over the default derived from caCert. Set sslMode: disable to keep an internal-network PostgreSQL without TLS working after upgrading to v4.10.

The example below enables sslmode=verify-full by attaching the CA bundle that signed the PostgreSQL server certificate:

Connector secret with TLS verification
apiVersion: v1
kind: Secret
metadata:
name: postgres-tls
namespace: vcluster-platform
labels:
loft.sh/connector-type: "shared-database"
stringData:
endpoint: postgres.internal.svc.cluster.local
port: "5432"
user: admin
password: <password>
type: postgres
caCert: |
-----BEGIN CERTIFICATE-----
<CA certificate that signed the PostgreSQL server certificate>
-----END CERTIFICATE-----

Recreate / restore clusters​

To accommodate infrastructure updates or disaster recovery needs, follow this process to restore clusters using a database connector powered backing store.

  1. Annotate the target VirtualClusterInstance with vcluster.com/database-reclaim-policy: retain so deleting the cluster doesn't also delete the database.
  2. Record the value of the vcluster.com/database-uid annotation and save the spec from the VirtualClusterInstance.
  3. Delete the cluster.
  4. Create the cluster with the same vcluster.com/database-uid annotation and the saved spec.
Scope

This procedure only restores the backing store connection. Depending on your configuration, resources such as persistent volumes or other external dependencies might not be available for the recreated cluster.

RBAC permissions

Creating VirtualClusterInstance resources with the vcluster.com/database-uid annotation set requires elevated permissions.

Sample ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: vcluster-database-admin
rules:
- apiGroups: [ "management.loft.sh" ]
resources: [ "virtualclusterinstances/externaldatabase" ]
verbs: [ "bind" ]

Sample bulk migration script​

important

This script assumes that every matched cluster uses a database connector.

If a matched cluster doesn't use a database connector, its data isn't preserved.

Sample control plane host bulk migration script
# requires kubectl, jq, and access to the vCluster Platform api

OLD_CLUSTER="old-cluster"
NEW_CLUSTER="new-cluster"

# select required instances running on old cluster
kubectl get virtualclusterinstance.management.loft.sh -A -o json | jq --arg c "$OLD_CLUSTER" '.items=[.items[] | select(.spec.clusterRef.cluster==$c)]' > ./old-vcis.json

# annotate with retain
kubectl annotate -f ./old-vcis.json vcluster.com/database-reclaim-policy=retain --overwrite

# prepare new vcis specs by setting the desired cluster and remove metadata and status fields
cat ./old-vcis.json | jq --arg c "$NEW_CLUSTER" '.items=[.items[] | .spec.clusterRef.cluster=$c | del(.metadata.resourceVersion, .metadata.uid, .metadata.creationTimestamp, .metadata.generation, .status)]' > ./new-vcis.json

# delete old VCIs
kubectl delete -f ./old-vcis.json --wait

# create new VCIs
kubectl create -f ./new-vcis.json

# wait for the new VCIs to be ready
kubectl wait -f ./new-vcis.json --for=condition=Ready