Central Admission Control

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

important

You must enable MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers to use the Central Admission Control feature. See the Kubernetes Admission Controllers Reference for details.

Centralized Admission Control is an advanced feature for cluster admins that have custom rules that they need to apply to the virtual cluster. Cluster admins can enforce Kubernetes admission webhooks that reference the host cluster or external policy services from within the virtual cluster. Examples of validating and mutating webhook based policy engines include OPA, Kyverno, and jsPolicy.

warning

• Central Admission Control is read-only after virtual cluster creation. Admins cannot bypass or alter hooks after vCluster deployment.
• If you need the ability to modify or delete webhook configurations, do not use Central Admission Control. Instead, manually map the host policy service into the virtual cluster.

Central admission webhooks are different from the other policies:

• LimitRange and ResourceQuota resources are created and enforced on the host cluster. They do do not appear as resources in the virtual cluster.
• A user could define LimitRange and ResourceQuota resources inside the virtual cluster, but they have full control over them and can delete them at will.
• Webhooks are different because they need to be configured inside the virtual cluster in order for them to be called by the vCluster's API server.
• vCluster rewrites these definitions to point to a proxy for a host cluster service that handles webhook requests. The host may also have webhook configurations that use this service.
• A user can still install a webhook service or webhook configuration into the virtual cluster outside of this config, but it would run inside the virtual cluster like any other workload.

Namespace Rewriting

vCluster rewrites the namespace of the object (if the object is namespace scoped) to the host namespace where vCluster is running in. This is not visible to the end-user, but the admission controller is able to use things like namespace selector like usual. Some admission controllers like gatekeeper wouldn't work otherwise as they rely on the namespace to always be there in the underlying cluster. To access the virtual namespace in a webhook, you can access the request.options.vCluster.namespace.metadata.name field in the admission request object that holds the full virtual namespace object.

Kyverno Example

This example ensures that all new ConfigMap resources are sent to a mutating webhook on the host cluster.

centralAdmission:
  mutatingWebhooks:
    - apiVersion: admissionregistration.k8s.io/v1
      kind: MutatingWebhookConfiguration
      metadata:
        name: kyverno-webhook
      webhooks:
      - admissionReviewVersions:
        - v1
        clientConfig:
          service:
            name: kyverno-svc
            namespace: kyverno
            path: /mutate/fail
            port: 443
        failurePolicy: Fail
        matchPolicy: Equivalent
        name: mutate.kyverno.svc-fail
        objectSelector: {}
        reinvocationPolicy: IfNeeded
        rules:
        - apiGroups:
          - ""
          apiVersions:
          - v1
          operations:
          - CREATE
          resources:
          - configmaps
          scope: '*'
        sideEffects: NoneOnDryRun
        timeoutSeconds: 10

Any time a ConfigMap is created, the vCluster API server forwards the admission request to a proxy, which in turn calls the actual admission hook running in the host cluster in the kyverno namespace.

Note that the service providing the webhooks should not rely on the real names of the objects if they are synced on the host cluster (for example, pods). The requests that reach the host admission service holds the objects from the virtual cluster, so your policies need to be able to handle those objects (except for the namespace which is rewritten as outlined below).

The hook services do not require authentication as the proxy does not have access to the AdmissionConfiguration object or the files it references. Most policy engines (like jsPolicy, Kyverno, or Gatekeeper) do not set authentication by default, so it should work with most installations.

If a user inside the virtual cluster (admin or not) tries to delete one of your admission hooks, that uses is prompted with the following error message:

kubectl delete kyverno-webhook
error: the resource kyverno-webhook is protected

Gatekeeper (OPA) Example

First install gatekeeper to your Kubernetes cluster.

Then create the following constraint template that denies pods without specified labels:

apiVersion: config.gatekeeper.sh/v1alpha1
kind: Config
metadata:
  name: config
  namespace: "gatekeeper-system"
spec:
  match:
    - excludedNamespaces: ["kube-*"]
      processes: ["*"]
  sync:
    syncOnly:
      - group: ""
        version: "v1"
        kind: "Namespace"
      - group: ""
        version: "v1"
        kind: "Pod"
---
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

Next create the constraint:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: pod-required-labels
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    labelSelector:
      matchLabels:
        test: gatekeeper
  parameters:
    labels: ["gatekeeper"]

Then start the vCluster with the following configuration:

policies:
  centralAdmission:
    validatingWebhooks:
    - apiVersion: admissionregistration.k8s.io/v1
      kind: ValidatingWebhookConfiguration
      metadata:
        name: gatekeeper-webhook
      webhooks:
      - admissionReviewVersions:
        - v1
        clientConfig:
          service:
            name: gatekeeper-webhook-service
            namespace: gatekeeper-system
            path: /v1/admit
            port: 443
        failurePolicy: Fail
        matchPolicy: Exact
        name: validation.gatekeeper.sh
        objectSelector: {}
        rules:
        - apiGroups:
          - '*'
          apiVersions:
          - '*'
          operations:
          - CREATE
          - UPDATE
          resources:
          - pods
          scope: Namespaced
        sideEffects: None
        timeoutSeconds: 10

Within the vCluster try to create the following pod which should get denied:

$ echo "apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    test: gatekeeper
spec:
  containers:
    - image: nginx
      name: nginx" | kubectl create -f -
admission webhook "validation.gatekeeper.sh" denied the request: you must provide labels: [gatekeeper]

Config reference

centralAdmission required object pro

CentralAdmission defines what validating or mutating webhooks should be enforced within the virtual cluster.

validatingWebhooks required object[] pro

ValidatingWebhooks are validating webhooks that should be enforced in the virtual cluster

kind required string pro

Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to.

apiVersion required string pro

APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values.

metadata required object pro

Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.

name required string pro

Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition.

labels required object pro

Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services.

annotations required object pro

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata.

webhooks required object[] pro

Webhooks is a list of webhooks and the affected resources and operations.

name required string pro

The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where "imagepolicy" is the name of the webhook, and kubernetes.io is the name of the organization.

clientConfig required object pro

ClientConfig defines how to communicate with the hook.

url required string pro

URL gives the location of the webhook, in standard URL form (scheme://host:port/path). Exactly one of url or service must be specified.

service required object pro

Service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

namespace required string pro

Namespace is the namespace of the service.

name required string pro

Name is the name of the service.

path required string pro

Path is an optional URL path which will be sent in any request to this service.

port required integer pro

If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. port should be a valid port number (1-65535, inclusive).

caBundle required string pro

CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate. If unspecified, system trust roots on the apiserver are used.

rules required object[] pro

Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule.

failurePolicy required string pro

FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.

matchPolicy required string pro

matchPolicy defines how the "rules" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".

namespaceSelector required object pro

NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook.

objectSelector required object pro

ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector.

sideEffects required string pro

SideEffects states whether this webhook has side effects.

timeoutSeconds required integer pro

TimeoutSeconds specifies the timeout for this webhook.

admissionReviewVersions required string[] pro

AdmissionReviewVersions is an ordered list of preferred AdmissionReview versions the Webhook expects.

matchConditions required object[] pro

MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.

mutatingWebhooks required object[] pro

MutatingWebhooks are mutating webhooks that should be enforced in the virtual cluster

kind required string pro

Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to.

apiVersion required string pro

APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values.

metadata required object pro

Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.

name required string pro

Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition.

labels required object pro

Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services.

annotations required object pro

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata.

webhooks required object[] pro

Webhooks is a list of webhooks and the affected resources and operations.

reinvocationPolicy required string pro

reinvocationPolicy indicates whether this webhook should be called multiple times as part of a single admission evaluation. Allowed values are "Never" and "IfNeeded".

name required string pro

The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where "imagepolicy" is the name of the webhook, and kubernetes.io is the name of the organization.

clientConfig required object pro

ClientConfig defines how to communicate with the hook.

url required string pro

URL gives the location of the webhook, in standard URL form (scheme://host:port/path). Exactly one of url or service must be specified.

service required object pro

Service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

namespace required string pro

Namespace is the namespace of the service.

name required string pro

Name is the name of the service.

path required string pro

Path is an optional URL path which will be sent in any request to this service.

port required integer pro

If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. port should be a valid port number (1-65535, inclusive).

caBundle required string pro

CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate. If unspecified, system trust roots on the apiserver are used.

rules required object[] pro

Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule.

failurePolicy required string pro

FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.

matchPolicy required string pro

matchPolicy defines how the "rules" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".

namespaceSelector required object pro

NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook.

objectSelector required object pro

ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector.

sideEffects required string pro

SideEffects states whether this webhook has side effects.

timeoutSeconds required integer pro

TimeoutSeconds specifies the timeout for this webhook.

admissionReviewVersions required string[] pro

AdmissionReviewVersions is an ordered list of preferred AdmissionReview versions the Webhook expects.

matchConditions required object[] pro

MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.