Ensure Unique Certificate Authority Is Used For Etcd

More Info:

Use a different certificate authority for etcd from the one used for Kubernetes.

Risk Level

Low

Address

Security

Compliance Standards

CIS Kubernetes

Triage and Remediation

Remediation

Remediation

Manual Steps

On every etcd node, identify the current etcd static pod manifest and existing certs (control plane node):

sudo ls -l /etc/kubernetes/manifests/etcd.yaml
sudo grep -n "cert-file\|key-file\|trusted-ca-file\|client-cert-auth" /etc/kubernetes/manifests/etcd.yaml || true
sudo grep -n "etcd" /etc/kubernetes/manifests/etcd.yaml

On every etcd node, create a dedicated etcd Certificate Authority (change CN/OU as needed) (control plane node):

sudo mkdir -p /etc/kubernetes/pki/etcd-ca
cd /etc/kubernetes/pki/etcd-ca

sudo openssl genrsa -out etcd-ca.key 4096

sudo openssl req -x509 -new -nodes -key etcd-ca.key -subj "/CN=etcd-ca" \
  -days 3650 -out etcd-ca.crt

On every etcd node, issue a new server certificate for etcd from the dedicated etcd CA (control plane node):

cd /etc/kubernetes/pki/etcd-ca

sudo openssl genrsa -out etcd-server.key 4096

sudo openssl req -new -key etcd-server.key -subj "/CN=etcd" \
  -out etcd-server.csr

cat << 'EOF' | sudo tee /etc/kubernetes/pki/etcd-ca/etcd-server-ext.cnf
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = localhost
IP.1 = 127.0.0.1
EOF

sudo openssl x509 -req -in etcd-server.csr -CA etcd-ca.crt -CAkey etcd-ca.key \
  -CAcreateserial -out etcd-server.crt -days 3650 -extensions v3_req \
  -extfile /etc/kubernetes/pki/etcd-ca/etcd-server-ext.cnf

sudo chmod 600 /etc/kubernetes/pki/etcd-ca/etcd-ca.key /etc/kubernetes/pki/etcd-ca/etcd-server.key

On every etcd node, update the etcd static pod manifest to use the dedicated etcd CA and certificates (control plane node; this will restart etcd automatically):

sudo cp /etc/kubernetes/manifests/etcd.yaml /etc/kubernetes/manifests/etcd.yaml.bak.$(date +%s)

sudo sed -i '/--cert-file=/d' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/--key-file=/d' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/--trusted-ca-file=/d' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/--client-cert-auth=/d' /etc/kubernetes/manifests/etcd.yaml

sudo sed -i '/- --listen-client-urls=/a\    - --cert-file=/etc/kubernetes/pki/etcd-ca/etcd-server.crt' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/- --cert-file=/a\    - --key-file=/etc/kubernetes/pki/etcd-ca/etcd-server.key' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/- --key-file=/a\    - --trusted-ca-file=/etc/kubernetes/pki/etcd-ca/etcd-ca.crt' /etc/kubernetes/manifests/etcd.yaml
sudo sed -i '/- --trusted-ca-file=/a\    - --client-cert-auth=true' /etc/kubernetes/manifests/etcd.yaml

On every etcd node, ensure the new etcd CA is not the same as the Kubernetes CA (visual/manual comparison) (control plane node):

sudo openssl x509 -in /etc/kubernetes/pki/etcd-ca/etcd-ca.crt -noout -issuer -subject -serial -fingerprint
sudo openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -issuer -subject -serial -fingerprint 2>/dev/null || echo "Kubernetes CA not at /etc/kubernetes/pki/ca.crt; locate and compare manually"

Verification on every etcd node, after kubelet has automatically restarted the etcd pod due to manifest change (control plane node):
```
/bin/ps -ef | /bin/grep etcd | /bin/grep -v grep
sudo grep "trusted-ca-file" /etc/kubernetes/manifests/etcd.yaml
```

Using kubectl

kubectl cannot modify the host-level etcd static pod manifest at /etc/kubernetes/manifests/etcd.yaml or the certificate authority files on the etcd nodes. Perform the remediation directly on every etcd (control plane) node as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
#
# Remediation: Ensure a unique Certificate Authority is used for etcd
# Scope: Run on every etcd (control-plane) node with root privileges.
#
# This script:
#   - Verifies /etc/kubernetes/manifests/etcd.yaml exists
#   - Backs it up once
#   - Ensures --trusted-ca-file is present and points to a dedicated etcd CA
#   - Triggers kubelet to restart the etcd static pod via manifest edit
#   - Verifies etcd is running with the configured --trusted-ca-file
#
# IMPORTANT:
#   - You must create and manage the dedicated etcd CA yourself,
#     as required by your security policy and etcd documentation.
#   - This script ONLY wires that CA into the etcd static pod manifest.
#

set -euo pipefail

ETCD_MANIFEST="/etc/kubernetes/manifests/etcd.yaml"
# Path to the dedicated etcd CA certificate; adjust if your CA is elsewhere.
ETCD_CA_FILE="/etc/kubernetes/pki/etcd/ca.crt"
BACKUP_SUFFIX=".pre-unique-etcd-ca.bak"

echo "[*] Ensuring unique Certificate Authority is configured for etcd"

if [[ $EUID -ne 0 ]]; then
  echo "[ERROR] This script must be run as root on each etcd node." >&2
  exit 1
fi

if [[ ! -f "$ETCD_MANIFEST" ]]; then
  echo "[ERROR] etcd manifest not found at ${ETCD_MANIFEST}. This node may not run etcd as a static pod." >&2
  exit 1
fi

if [[ ! -f "$ETCD_CA_FILE" ]]; then
  cat >&2 <<EOF
[ERROR] Expected etcd CA file not found at ${ETCD_CA_FILE}.

Create and place a dedicated etcd CA certificate at this path
(or adjust ETCD_CA_FILE in this script to match your environment),
following the etcd documentation and your PKI procedures.

This script does not generate or manage the CA; it only configures etcd to use it.
EOF
  exit 1
fi

# One-time backup
if [[ ! -f "${ETCD_MANIFEST}${BACKUP_SUFFIX}" ]]; then
  echo "[*] Backing up ${ETCD_MANIFEST} to ${ETCD_MANIFEST}${BACKUP_SUFFIX}"
  cp -p "${ETCD_MANIFEST}" "${ETCD_MANIFEST}${BACKUP_SUFFIX}"
else
  echo "[*] Backup already exists at ${ETCD_MANIFEST}${BACKUP_SUFFIX}, skipping backup"
fi

# Ensure the etcd container args include --trusted-ca-file pointing to ETCD_CA_FILE
echo "[*] Updating etcd manifest to ensure --trusted-ca-file=${ETCD_CA_FILE}"

TMP_MANIFEST="$(mktemp)"
cp "${ETCD_MANIFEST}" "${TMP_MANIFEST}"

# If --trusted-ca-file exists, update its value; otherwise, insert it into the args list.
if grep -q -- "--trusted-ca-file=" "${TMP_MANIFEST}"; then
  # Idempotently replace the value
  sed -i "s#--trusted-ca-file=[^\" ]*#--trusted-ca-file=${ETCD_CA_FILE}#g" "${TMP_MANIFEST}"
else
  # Insert a new arg line under the etcd container args section
  # Handles typical kubeadm-generated manifests.
  awk -v ca_arg="        - --trusted-ca-file=${ETCD_CA_FILE}" '
    /name: etcd/ { in_etcd=1 }
    in_etcd && /^\s*args:/ { in_args=1 }
    in_args && /^\s*-/ && !inserted {
      print $0
      print ca_arg
      inserted=1
      next
    }
    { print $0 }
  ' "${TMP_MANIFEST}" > "${TMP_MANIFEST}.new" && mv "${TMP_MANIFEST}.new" "${TMP_MANIFEST}"

  # If we failed to insert (no args: block or unusual layout), append to file and warn.
  if ! grep -q -- "--trusted-ca-file=${ETCD_CA_FILE}" "${TMP_MANIFEST}"; then
    cat >&2 <<EOF
[WARN] Could not automatically locate the etcd args section to insert --trusted-ca-file.
Appending the argument at the end of the manifest, but you should verify the layout manually.
EOF
    printf "    - --trusted-ca-file=%s\n" "${ETCD_CA_FILE}" >> "${TMP_MANIFEST}"
  fi
fi

# Move updated manifest into place (this will cause kubelet to restart the etcd static pod)
echo "[*] Applying updated manifest (this restarts the etcd static pod)"
cp "${TMP_MANIFEST}" "${ETCD_MANIFEST}"
rm -f "${TMP_MANIFEST}"

# Wait for etcd to restart and stabilize
echo "[*] Waiting for etcd process to reflect new arguments..."
sleep 10

# Verification: show etcd process and confirm --trusted-ca-file is present and correct
echo "[*] Verifying etcd is running with the configured --trusted-ca-file"

if ! ps -ef | grep -v grep | grep -q "[e]tcd"; then
  echo "[ERROR] etcd process not found after manifest update. Check kubelet and etcd logs." >&2
  exit 1
fi

ETCD_CMDLINE="$(ps -ef | grep -v grep | grep '[e]tcd' | head -n1)"

if echo "${ETCD_CMDLINE}" | grep -q -- "--trusted-ca-file=${ETCD_CA_FILE}"; then
  echo "[OK] etcd is running with --trusted-ca-file=${ETCD_CA_FILE}"
else
  echo "[ERROR] etcd process is running but --trusted-ca-file does not match ${ETCD_CA_FILE}." >&2
  echo "       Current command line:" >&2
  echo "       ${ETCD_CMDLINE}" >&2
  exit 1
fi

echo "[*] Remediation complete on this node."

Additional Reading:

https://coreos.com/etcd/docs/latest/op-guide/security.html

More Info:​

Risk Level​

Address​

Compliance Standards​

Triage and Remediation​

Remediation​

Additional Reading:​