Version: v0.36

Configure auto sleep

		Enterprise
Available in these plans	Free	Dev	Prod	Scale
Auto Sleep

Free, Dev, Prod, and Scale are vCluster Platform license plans. Open source does not need a license or a Platform connection. See Compare open source and free tiers.

Supported Configurations

Running the control plane as a container with:

Shared Nodes

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Auto sleep reduces the compute resources that idle tenant clusters consume. It can respond to inactivity, a cron schedule, or supported HTTP traffic.

Pre-production workloads

Auto sleep is intended for pre-production workloads. Sleeping interrupts workloads and deletes standalone Pods that no controller manages.

How auto sleep is managed

The component that manages auto sleep determines which resources can stop:

Deployment	Sleep behavior
Tenant cluster without a platform agent	The tenant cluster scales down its workloads. Its control plane stays running to detect activity and restore workloads.
Tenant cluster connected to vCluster Platform	The platform agent takes over sleep management and can scale down both workloads and the tenant cluster control plane.

When an existing tenant cluster connects to a compatible platform agent, the agent reads its sleep configuration and takes over automatically. Don't configure a second sleep mechanism. This handoff requires vCluster 0.24.0 or later and vCluster Platform 4.3.0 or later.

For vCluster 0.33.0 or later, add the sleepmode.loft.sh/scope: "workloads-only" annotation to the platform VirtualClusterInstance. Use it when the tenant cluster should keep managing its own workload sleep and its control plane must stay running. With this scope, the platform agent doesn't take over auto sleep.

Set the scope before the platform agent takes over sleep management. If the tenant cluster is already connected, set the scope and then remove the vcluster.loft.sh/agent-installed annotation from its vc-config-<vcluster-name> Secret. The Secret is in the namespace where the tenant cluster control plane runs on the control plane cluster. The platform agent leaves the annotation in place when the scope changes, and native auto sleep remains disabled until you remove it.

See Sleep tenant clusters for the lifecycle and Pause and resume a tenant cluster for manual operations.

Configure inactivity-based auto sleep

vcluster.yaml
Platform UI

Set sleep.auto.afterInactivity to the amount of idle time allowed before workloads sleep:

vcluster.yaml
sleep:
  auto:
    afterInactivity: 1h

The value uses Go duration syntax. Use s, m, or h, such as 30s, 15m, or 24h. The syntax doesn't support d; use 24h instead of 1d.

Configure scheduled sleep and wakeup

Set five-field cron expressions for schedule and wakeup.schedule. Set an IANA time zone with timezone; the default is UTC.

Sleep Friday evening and wake Monday morning
sleep:
  auto:
    timezone: America/Denver
    schedule: 30 17 * * 5
    wakeup:
      schedule: 0 7 * * 1

In vCluster Platform, select the project and click Tenant Clusters. Click Edit for the tenant cluster, expand Auto-Sleep and Auto-Delete, then set Sleep Schedule, Wake up Schedule, and Schedule timezone.

Exclude workloads from auto sleep

Use a label selector when you know which workloads must stay running. The following configuration excludes workloads with the label sleep: no-thanks:

Exclude workloads by label
sleep:
  auto:
    afterInactivity: 3h
    exclude:
      selector:
        labels:
          sleep: no-thanks

For platform-managed sleep, you can instead add the sleepmode.loft.sh/exclude: "true" annotation to a Deployment, StatefulSet, ReplicaSet, or Pod. Sleep creates a quota that prevents new Pods, so Kubernetes can't replace an excluded Pod that stops while the resource sleeps:

Exclude a Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: connectivity-agent
  annotations:
    sleepmode.loft.sh/exclude: "true"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: connectivity-agent
  template:
    metadata:
      labels:
        app: connectivity-agent
    spec:
      containers:
        - name: agent
          image: example.com/connectivity-agent:1.0

Activity detection and wakeup

This section applies to platform-managed namespaces and tenant clusters alike, except where noted. Auto sleep resets the inactivity timer when it detects relevant activity. If the resource is already sleeping, supported activity wakes it.

For platform-managed sleep started by a sleep schedule, activity doesn't wake the resource. It remains asleep until its wakeup schedule or a manual wakeup. Standalone auto sleep, which only applies to a tenant cluster without a platform agent, can wake from activity after scheduled sleep.

Kubernetes API activity

For a platform-managed namespace or tenant cluster, Kubernetes API requests routed through vCluster Platform count as activity. This includes kubectl requests when the kubeconfig routes through the platform proxy.

Direct requests to a tenant cluster API endpoint that bypasses the platform proxy aren't visible to platform activity detection. A standalone tenant cluster detects API activity through its running control plane.

Detect web traffic

For both namespaces and tenant clusters, vCluster Platform can detect traffic to the following resources:

ingress-nginx Ingress resources
Istio Gateway and VirtualService resources
Gateway API HTTPRoute resources when the Gateway controller supports request mirroring

For ingress-nginx, the platform adds mirror-target and mirror-request-body annotations unless the Ingress already has either annotation. The platform preserves existing user-managed mirror annotations, but traffic to that Ingress doesn't refresh the last-activity timestamp through this integration. ingress-nginx is deprecated; for new endpoint deployments, prefer Gateway API.

For Istio, enable the Istio integration. The platform uses istio-proxy metrics to determine activity.

Gateway API HTTPRoute activity

Gateway-backed activity detection requires the controller to support the HTTPRoute request-mirror filter. An HTTPRoute can serve traffic without this feature, but its awake traffic doesn't refresh the last-activity timestamp. The namespace or tenant cluster can therefore enter sleep while the route is receiving traffic. Don't rely on activity-based auto sleep through that route unless its controller supports request mirroring.

If a compatible controller doesn't advertise request mirroring in GatewayClass status, add the sleepmode.loft.sh/request-mirror-controller-allowlist annotation to the platform Cluster resource. Use the GatewayClass.spec.controllerName value. Separate multiple controller names with commas.

Restart the agent after installing Gateway API CRDs

If you install Gateway API CRDs after the connected-cluster agent starts, restart the agent Deployment so it discovers the HTTPRoute controller:

Restart the vCluster Platform agent
kubectl rollout restart deployment/loft --namespace vcluster-platform

The agent Deployment name is always loft. Its namespace defaults to vcluster-platform for newly connected clusters; clusters connected before this default existed may still use loft. Check the platform Cluster resource's spec.managementNamespace if you're unsure.

Ingress wakeup

When platform activity detection supports the route, opening its hostname displays a wakeup page while the namespace or tenant cluster wakes. The page reloads and reaches the application after wakeup completes.

Last activity update interval

The platform limits how often mirrored Ingress and HTTPRoute traffic updates the last-activity timestamp. The default interval is five minutes. Requests received during the interval don't write another timestamp. Istio activity uses its metrics polling interval instead.

Set SLEEP_MODE_LAST_ACTIVITY_UPDATE_INTERVAL on the connected-cluster agent to change the interval. See Agent values for how to set agent environment variables platform-wide or per cluster.

Filter platform activity

Use sleepmode.loft.sh annotations when automated clients or selected requests shouldn't keep a platform-managed namespace or tenant cluster awake. For a namespace, add them to the Namespace. For a template-backed tenant cluster, add the supported annotations under spec.template.metadata.annotations in its VirtualClusterTemplate. For an inline VirtualClusterInstance, use the same path on the instance. ignore-groups, ignore-resources, ignore-verbs, ignore-resource-verbs, and ignore-resource-names apply to namespace sleep mode only. They have no effect on a tenant cluster.

Common filters include:

Annotation	Effect
`sleepmode.loft.sh/ignore-all`	Ignore all activity.
`sleepmode.loft.sh/ignore-ingresses`	Ignore supported Ingress, HTTPRoute, and Istio traffic.
`sleepmode.loft.sh/ignore-groups`	Ignore requests from selected user groups. Namespace only.
`sleepmode.loft.sh/ignore-resources`	Ignore requests to selected Kubernetes resources. Namespace only.
`sleepmode.loft.sh/ignore-verbs`	Ignore selected Kubernetes API verbs. Namespace only.
`sleepmode.loft.sh/ignore-user-agents`	Ignore matching user agents; a trailing wildcard is supported.
`sleepmode.loft.sh/ignore-active-connections`	Don't treat open streaming connections as activity.
`sleepmode.loft.sh/disable-ingress-wakeup`	Don't wake the resource for supported Ingress, HTTPRoute, or Istio access.

You can also add the X-Sleep-Mode-Ignore header to an individual request. See the platform annotation reference for where to set each filter and all value formats.

Handle long-running connections

Streaming requests such as kubectl exec and kubectl port-forward can keep a namespace or tenant cluster active. Choose one of these controls when idle streams should no longer prevent sleep:

Add loft.sh/streaming-connection-idle-timeout: "3600" to the platform Cluster resource. The platform closes new streaming connections after the configured number of idle seconds. Existing connections aren't affected.
Add sleepmode.loft.sh/ignore-active-connections: "true" to the Namespace or tenant cluster template metadata. Open streams then don't count as activity and can be interrupted when the resource sleeps.
Configure the kubelet --streaming-connection-idle-timeout flag on the control plane cluster independently of vCluster Platform.

Configure auto-delete

Auto-delete permanently deletes a platform-managed tenant cluster after an inactivity period. It is separate from sleep and can run whether the tenant cluster is awake or sleeping.

See Configure automatic deletion for the vcluster.yaml and vCluster Platform procedures.

Examples

Standalone deployment example

Configure auto sleep for a deployment resource

Create the kind cluster.

create kind cluster
kind create cluster --name sleep-mode-demo

Deploy a tenant cluster.
Use the following vcluster.yaml to create a tenant cluster on your control plane cluster. Save this file as vcluster.yaml
vCluster config for auto sleep
```
pro: true
sleep:
  auto:
    afterInactivity: 30s
    exclude:
      selector:
        labels:
          sleep: no-thanks
```
And run the following command:
Create vCluster with autoSleep config
```
vcluster create my-vcluster -f vcluster.yaml
```
Workloads with the label sleep: no-thanks don't enter auto sleep after 30 seconds.

Create demo deployments in your tenant cluster.

Use the following deployment YAML to create two deployments.

Example deployments
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleepy-deployment
  labels:
    app: sleepy-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: demo-dep-1
  template:
    metadata:
      labels:
        app: demo-dep-1
    spec:
      containers:
        - command:
            - /agnhost
            - serve-hostname
            - --http=true
            - --port=8080
          image: registry.k8s.io/e2e-test-images/agnhost:2.39
          name: sleepy-demo

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: no-sleep-deployment
  labels:
    sleep: no-thanks
spec:
  replicas: 2
  selector:
    matchLabels:
      app: demo-dep-2
  template:
    metadata:
      labels:
        app: demo-dep-2
    spec:
      containers:
        - command:
            - /agnhost
            - serve-hostname
            - --http=true
            - --port=8080
          image: registry.k8s.io/e2e-test-images/agnhost:2.39
          name: not-sleepy-demo

The first deployment does not have any special configurations for auto sleep. You can replace it with another deployment if needed. The second deployment includes a special label on the Deployment, preventing it from scaling down after 30 seconds.

You can verify this by waiting 30 seconds and then getting information about the Deployments. For example

Verify Deployments sleep status.
deployment sleep check
```
> sleep 30; kubectl get deployments
NAMESPACE     NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
default       no-sleep-deployment   2/2     2            2           1m
default       sleepy-deployment     0/2     0            0           1m
```
The sleepy-deployment reports 0/2 replicas after 30 seconds. Running kubectl counts as cluster activity, which is why it reports 0/2 instead of 0/0. The kubectl command triggers vCluster to update the replica count back to the original value of 2, but the replicas haven't become ready by the time kubectl get ... returns.

Next steps

Experiment with the auto sleep feature by trying the following:

Add the sleep: no-thanks label to the first deployment and verify neither sleeps.
Remove the sleep: no-thanks label from both the deployments and verify that both go to sleep.

Ingress example

Configure auto sleep for an ingress controller

Create the kind cluster.

Create a kind cluster
kind create cluster --name ingress-demo --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "0.0.0.0"
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
EOF

[Deprecated]: Install the NGINX IngressController.

install ingress controller
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.dnsPolicy=ClusterFirstWithHostNet \
  --set controller.hostNetwork=true \
  --set controller.service.type=ClusterIP

Create the vCluster.
Use the following vcluster.yaml to create a tenant cluster on your control plane cluster. Save this file as vcluster.yaml
vCluster config for auto sleep
```
pro: true
sync:
  toHost:
    ingresses:
      enabled: true
sleep:
  auto:
    afterInactivity: 30s
```
And run the following command:
Create vCluster with autoSleep config
```
vcluster create my-vcluster -f vcluster.yaml
```
Enable local DNS resolution for the tenant cluster.
Add 127.0.0.1 backend.local to your /etc/hosts file to match the host configured in the Ingress rules of the next step.

Create resources for the Ingress such as a Deployment and Service.

Use the following manifest to create:

Namespace named bar
Deployment for the pods backing the Service
Service for the Ingress
Ingress resource

Example deployments
apiVersion: v1
kind: Namespace
metadata:
  name: bar

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bar-deployment
  namespace: bar
  labels:
    app: bar-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: bar
  template:
    metadata:
      labels:
        app: bar
    spec:
      containers:
        - command:
            - /agnhost
            - serve-hostname
            - --http=true
            - --port=8080
          image: registry.k8s.io/e2e-test-images/agnhost:2.39
          name: bar-app

---

kind: Service
apiVersion: v1
metadata:
  name: bar-service
  namespace: bar
spec:
  selector:
    app: bar
  ports:
    # Default port used by the image
    - port: 8080

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: bar
spec:
  ingressClassName: nginx # the ingress-nginx project has been deprecated, we recommend using a different ingress class
  rules:
    - http:
        paths:
          - pathType: Prefix
            path: /bar
            backend:
              service:
                name: bar-service
                port:
                  number: 8080
      host: backend.local

Verify the ingress is working properly with curl.
Test the Ingress endpoint within the 30-second activity window by running curl --silent backend.local/bar. The pod name from the Deployment that responds is displayed.
Allow the tenant cluster to go to sleep.
Wait 30 seconds for the cluster to sleep, then run the curl command again. For convenience, run watch -d curl --silent backend.local/bar to repeatedly test the endpoint. On the first attempt, you’ll see the message Client sent an HTTP request to an HTTPS server. because an HTTP request was sent to the HTTPS wake endpoint. Subsequent requests display new pod names.

Istio Gateway example

info

This example continues from the Istio integration example.

Complete the Istio integration example.

Install the Istio ingress gateway controller to allow routing from outside the cluster.

Save the following manifest as ingress.yaml:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: ingress
spec:
  profile: empty # Do not install CRDs or the control plane
  components:
    ingressGateways:
    - name: istio-ingressgateway
      namespace: istio-ingress
      enabled: true
      label:
        istio: ingressgateway
  values:
    gateways:
      istio-ingressgateway:
        injectionTemplate: gateway

Create the istio-ingress namespace and install the ingress gateway:

kubectl create namespace istio-ingress                                                                                                      
istioctl install -f ingress.yaml

Create a Gateway resource that uses a selector matching the newly installed ingress gateway in your vCluster.

apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: istio-sm-gateway
  namespace: test
spec:
  # The selector matches the ingress gateway pod labels.
  # If you installed Istio using Helm following the documentation, this is "istio=ingress"
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "smdemo.local"

Update the vCluster with auto sleep enabled.
Use the following vcluster.yaml to update your tenant cluster on your control plane cluster with --upgrade. Save this file as vcluster.yaml
vCluster config for auto sleep
```
pro: true
sleep:
  auto:
    afterInactivity: 30s
integrations:
  istio:
    enabled: true
```
Run the following command:
Create vCluster with autoSleep config
```
vcluster create <your-vcluster-name> -f vcluster.yaml --upgrade
```
Enable local DNS resolution for the tenant cluster.
Add 127.0.0.1 smdemo.local to your /etc/hosts file to match the host configured in the Gateway configuration.

Add the Gateway you created and the smdemo.local host to the VirtualService from the [Istio integration example](../vcluster- yaml/integrations/istio). The updated VirtualService should look like the following:

Example resources
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: nginx-service
  namespace: test
spec:
  gateways:
    - istio-sm-gateway
  hosts:
    - smdemo.local
  http:
    - match:
        - uri:
            prefix: /v2
      name: nginx-v2
      rewrite:
        uri: /
      route:
        - destination:
            host: nginx-service.test.svc.cluster.local
            subset: v2
    - name: nginx-v1
      route:
        - destination:
            host: nginx-service.test.svc.cluster.local
            subset: v1

Use curl to verify that the Gateway is working correctly.
Test the Gateway endpoint within the 30-second activity window by running either curl --silent smdemo.local/v1 or curl --silent smdemo.local/v2.
Allow the tenant cluster to go to sleep.
Wait 30 seconds for the cluster to sleep. Then run the curl command again. To repeatedly test the endpoint, run watch -d curl --silent smdemo.local/v2. While the cluster is asleep or waking up, a 503 error is displayed.

Configuration reference

`sleep` object

Sleep holds configuration for automatically putting the virtual cluster to sleep.

`auto` object

Auto holds automatic sleep configuration

`afterInactivity` string

AfterInactivity represents how long a vCluster can be idle before workloads are automatically put to sleep

`schedule` string

Schedule represents a cron schedule for when to sleep workloads

`exclude` object

Exclude holds configuration for labels that, if present, will prevent a workload from going to sleep

`selector` object

`labels` object

Labels defines what labels should be looked for

`wakeup` object

Wakeup holds configuration for waking the vCluster on a schedule

`schedule` string

`timezone` string

Timezone specifies time zone used for scheduled sleep operations. Defaults to UTC. Accepts the same format as time.LoadLocation() in Go (https://pkg.go.dev/time#LoadLocation). The value should be a location name corresponding to a file in the IANA Time Zone database, such as "America/New_York".

How auto sleep is managed​

Configure inactivity-based auto sleep​

Configure scheduled sleep and wakeup​

Exclude workloads from auto sleep​

Activity detection and wakeup​

Kubernetes API activity​

Detect web traffic​

Gateway API HTTPRoute activity​

Ingress wakeup​

Last activity update interval​

Filter platform activity​

Handle long-running connections​

Configure auto-delete​

Examples​

Configure auto sleep for a deployment resource​

Next steps​

Configure auto sleep for an ingress controller​

Configuration reference​

sleep object ​

auto object ​

afterInactivity string ​

schedule string ​

exclude object ​

selector object ​

labels object ​

wakeup object ​

schedule string ​

timezone string ​

How auto sleep is managed

Configure inactivity-based auto sleep

Configure scheduled sleep and wakeup

Exclude workloads from auto sleep

Activity detection and wakeup

Kubernetes API activity

Detect web traffic

Gateway API HTTPRoute activity

Ingress wakeup

Last activity update interval

Filter platform activity

Handle long-running connections

Configure auto-delete

Examples

Configure auto sleep for a deployment resource

Next steps

Configure auto sleep for an ingress controller

Configuration reference

`sleep` object

`auto` object

`afterInactivity` string

`schedule` string

`exclude` object

`selector` object

`labels` object

`wakeup` object

`schedule` string

`timezone` string