How the resource proxy works

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

vCluster Platform required

This feature requires vCluster Platform. Both the client and target tenant clusters must be managed as VirtualClusterInstance within the platform.

The Resource Proxy feature enables vClustervClusterAn open-source software product that creates and manages tenant clusters within Kubernetes infrastructure. vCluster provides tenant isolation capabilities while reducing infrastructure costs.Related: Tenant cluster, Control plane cluster to proxy requests for custom resources defined by CustomResourceDefinitions (CRDs) to other tenant clusters. When enabled, the client tenant clusterTenant clusterA fully isolated Kubernetes environment provisioned for a single tenant. Each tenant cluster has its own API server, controller manager, and resource namespace, backed by a virtualized control plane hosted on a control plane cluster. From the tenant's perspective it behaves exactly like a standard Kubernetes cluster.Related: Control plane cluster, Tenant cluster transparently forwards resource operations to a target tenant cluster. The objects are stored there, where controllers running on the target can manage them. This enables cross-cluster communication patterns, centralized resource management, and tenant-isolated architectures.

To configure the proxy, see Configure the resource proxy. To set one up end to end, see Set up the resource proxy.

Key capabilities

• Transparent access: Users interact with custom resources as if they were local to their tenant cluster.
• Centralized storage: A target tenant cluster stores the custom resources routed to it.
• Tenant isolationTenant IsolationThe capability to host multiple separate users, teams, or workloads on the same infrastructure while providing strong isolation between them. vCluster delivers tenant isolation through dedicated control planes, isolated resource namespaces, and optional private nodes per tenant.Related: Tenant cluster: Each client tenant cluster only sees objects with its ownership labels by default.
• RBAC enforcement: The target tenant cluster enforces its own RBAC policies on proxied requests.

How it works

When you configure a client tenant cluster to proxy custom resources, vCluster intercepts API requests for those resources. It forwards them to the target tenant cluster through vCluster Platform.

Resource requests cross the vCluster Platform network from the client aggregation layer to the target API and custom resource

The proxy performs several key functions:

1. Request interception: The client's Kubernetes API serverAPI ServerThe core component of Kubernetes that exposes the Kubernetes API. It is the front-end for the Kubernetes control plane and handles all REST operations, validating and configuring data for API objects.Related: Control Plane, rate-limiting intercepts requests for configured API groups and versions and routes them to the proxy. One enabled resource entry activates routing for its entire group and version; target RBAC restricts access to individual resource kinds.
2. Authentication: The proxy authenticates to the target using the client's vCluster Platform identity. Its format is loft:vcluster:<client-project-namespace>:<name>, for example loft:vcluster:p-default:client when the project namespace is p-default.
3. Owner labeling: On create and update operations, the proxy attempts to add owner labels that identify the client responsible for each resource.
4. Visibility filtering: For owned access, the proxy filters lists and watches and masks named GET, PUT, PATCH, and DELETE requests for objects that the client doesn't own. The all mode disables this filtering.
5. Namespace synchronization: Before a namespaced POST, PUT, or PATCH request, the proxy verifies that the namespace exists in the client tenant cluster and creates it in the target if needed.

Mutual proxying is supported. Two tenant clusters can proxy resources from each other, each acting as the client in one direction and the target in the other. The two directions are independent, and the target metric label distinguishes them. See Monitor the proxy.

Multi-client isolation

When multiple client tenant clusters proxy to the same target, each client only sees objects with its ownership labels by default. The proxy normally adds those labels when the client writes an object and injects a matching label selector on lists and watches.

Owner labels and selector injection give each client a filtered view of resources stored in the shared target

To set this up, see Set up multi-client isolation.

Proxy objects in the client tenant cluster

The proxy wires itself into the client's Kubernetes aggregation layer. The troubleshooting guide references these objects and conventions:

What	Where	Notes
APIService <version>.<group>	Client tenant cluster	Registered by the elected leader replica. For example, example.com/v1 uses v1.example.com. kubectl get apiservice is the first health signal.
Service proxy-<group>-<version>	kube-system namespace in the client tenant cluster	ExternalName Service pointing to localhost. Dots in the group become dashes, and long names are shortened with a hash suffix.
Proxy port	localhost:9100+n inside the vCluster control plane pod	One proxy per group and version, sorted alphabetically so all replicas agree on port assignment. Not reachable from outside the pod.
Ownership labels	Objects in the target tenant cluster	vcluster.com/owner-vcluster=<client-name> and vcluster.com/owner-project=<client-project-name>. The project label uses the Platform project name, not its Kubernetes namespace. The proxy attempts to inject both labels into POST, PUT, and PATCH bodies. If injection fails, the request continues without them.
Deletion protection	Client tenant cluster	The APIService and proxy Service carry the vcluster.loft.sh/protected=resource-proxy label, and a ValidatingAdmissionPolicy denies deleting them. See Deleting the proxy APIService or Service is denied.

Connection path and peer health

A caller is anything that talks to the client tenant cluster's API server: kubectl, a controller, or a CI job. A proxied request travels this path:

1. The caller sends a request to the client tenant cluster's API server.
2. The APIService registration routes the request through the proxy-<group>-<version> Service to the resource proxy, which listens on a local port inside the client's vCluster control planeControl PlaneThe container orchestration layer that exposes the API and interfaces to define, deploy, and manage the lifecycle of containers. In vCluster, each tenant cluster has its own control plane components.Related: API Server, vCluster pod.
3. The proxy forwards the request over the platform network, using a direct WireGuard peer-to-peer connection where possible and the vCluster Platform DERP relay as a fallback. See Prefer direct connections over the DERP relay.
4. The target tenant cluster's API server serves the request.

The proxy discovers the target's control plane pods as peers on the platform network and distributes requests across them round-robin. Peer health is tracked per host.

Fixed thresholds

This timing is fixed and not configurable:

Constant	Value	Effect
Peer dial timeout	5s	A single dial attempt gives up after this long, converting a hanging dial into a countable failure.
Quarantine threshold	3 consecutive dial failures	The endpoint leaves rotation.
Quarantine duration	20s	Then a single trial dial. Requests fail fast instead of hanging while a peer is quarantined.
Peer list staleness tolerance	60s	Measured from the moment the peer watch subscription drops, not from the last publication. A successful publication resets the clock. Past the tolerance, the proxy treats the cached peer list as stale and fails requests with 503 rather than trusting it.
Non-watch request timeout	60s	Hard cap on any single proxied request and response, end to end. Watches stream indefinitely and are exempt.
Retry-After on 503	1s	Caller backoff hint.
Proxy ports	9100 and up	One per group and version, alphabetical, identical across replicas.

Behavior when the target is unreachable

Two design decisions explain most confusing symptoms:

1. Discovery stays available during common target failures. When the target is unreachable, or answers discovery with 401, 404, or 5xx, the proxy serves discovery requests locally so the APIService stays Available and the client tenant cluster's API server keeps functioning. kubectl api-resources can therefore look healthy while every non-discovery request fails with 503. Use metrics and logs, not discovery, to judge proxy health.
2. Non-watch resource failures degrade to 503 with Retry-After: 1. Peer discovery problems, quarantined peers, a stale peer list, and target-side 5xx responses for non-watch resource requests surface as 503 Service Unavailable. The discovery statuses listed above use the local fallback, and watch responses stream directly from the target. The cause of a non-watch 503 is distinguishable from logs and metrics, not from the error the caller sees.

If you're chasing a specific symptom rather than reading up on the mechanism, start at the troubleshooting guide instead.