etcd configuration
The following section covers security areas related to etcd configuration. The assessment focuses on encryption of sensitive data at rest on server, applications and in transit.
Assessment focus for vCluster involves verifying correct authentication mechanisms are used and safeguarding the data at rest and in transit via TLS encryption. Key areas include validating that certificate-based authentication is properly configured and that automatic TLS generation is disabled in favor of explicit certificate management.
Control numbering
The control numbers used throughout this guide (2.1, 2.2, etc.) correlate directly to the official CIS Kubernetes Benchmark control numbers. This allows you to cross-reference with the official CIS documentation and maintain consistency with standard security frameworks.
2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that the --cert-file and the --key-file arguments are set as appropriate.
Expected Result:
'--cert-file' and '--key-file' arguments are appropriately set
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.2 Ensure that the --client-cert-auth argument is set to true​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that the --client-cert-auth argument is set to true.
Expected Result:
'--client-cert-auth' is set to 'true'
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.3 Ensure that the --auto-tls argument is not set to true​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that if the --auto-tls argument exists, it is not set to true.
Expected Result:
'--auto-tls' argument does not exist
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that the --peer-cert-file and --peer-key-file arguments are set as appropriate.
Expected Result:
'--peer-cert-file' and '--peer-key-file' arguments are appropriately set
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.5 Ensure that the --peer-client-cert-auth argument is set to true​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that the --peer-client-cert-auth argument is set to true.
Expected Result:
'--peer-client-cert-auth' is set to 'true'
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.6 Ensure that the --peer-auto-tls argument is not set to true​
Result: PASS
Audit:
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcd
Verify that if the --peer-auto-tls argument exists, it is not set to true.
Expected Result:
'--peer-auto-tls' argument does not exist
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
2.7 Ensure that a unique Certificate Authority is used for etcd​
Result: PASS
Run the following command against the vCluster pod:
kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep etcdNote the file referenced by the
--trusted-ca-fileargument.Run the following command and note the file referenced by
--client-ca-file:kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep kube-apiserverVerify that the file referenced by the
--client-ca-filefor apiserver is different from the--trusted-ca-fileused by etcd.
Expected Result:
The file referenced by the --client-ca-file for api-server is different from the --trusted-ca-file
Returned Value:
31 root 2:41 etcd --data-dir=/data/etcd --advertise-client-urls=https://emb-0.emb-headless.emb:2379 --initial-advertise-peer-urls=https://emb-0.emb-headless.emb:2380 --initial-cluster-token=vcluster --listen-client-urls=https://0.0.0.0:2379 --listen-metrics-urls=http://0.0.0.0:2381 --listen-peer-urls=https://0.0.0.0:2380 --name=emb-0 --heartbeat-interval=500 --election-timeout=5000 --experimental-watch-progress-notify-interval=5s --experimental-peer-skip-client-san-verification --log-level=info --snapshot-count=10000 --log-outputs=stderr --logger=zap --client-cert-auth=true --cert-file=/data/pki/etcd/server.crt --key-file=/data/pki/etcd/server.key --peer-client-cert-auth=true --peer-key-file=/data/pki/etcd/peer.key --peer-cert-file=/data/pki/etcd/peer.crt --peer-trusted-ca-file=/data/pki/etcd/ca.crt --trusted-ca-file=/data/pki/etcd/ca.crt --initial-cluster=emb-0= https://emb-0.emb-headless.emb:2380 --initial-cluster-state=new --force-new-cluster
47 root 6:43 /binaries/kube-apiserver --advertise-address=127.0.0.1 --service-cluster-ip-range=10.96.0.0/16 --bind-address=127.0.0.1 --allow-privileged=true --authorization-mode=RBAC --client-ca-file=/data/pki/client-ca.crt --enable-bootstrap-token-auth=true --etcd-servers=https://127.0.0.1:2379 --etcd-cafile=/data/pki/etcd/ca.crt --etcd-certfile=/data/pki/apiserver-etcd-client.crt --etcd-keyfile=/data/pki/apiserver-etcd-client.key --proxy-client-cert-file=/data/pki/front-proxy-client.crt --proxy-client-key-file=/data/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/data/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer= https://kubernetes.default.svc.cluster.local --service-account-key-file=/data/pki/sa.pub --service-account-signing-key-file=/data/pki/sa.key --tls-cert-file=/data/pki/apiserver.crt --tls-private-key-file= /data/pki/apiserver.key --endpoint-reconciler-type=none --profiling=false