Etcd Certfile And Etcd Keyfile Arguments Are Appropriate

More Info:

etcd should be configured to make use of TLS encryption for client connections.

Risk Level

Medium

Address

Security

Compliance Standards

• CIS Kubernetes

Triage and Remediation

Remediation

Remediation

Manual Steps

1. On every control plane node, identify the API server manifest and existing etcd TLS files:

ls -l /etc/kubernetes/manifests/kube-apiserver.yaml
ls -l /etc/kubernetes/pki/etcd

If your etcd client cert/key live elsewhere, adjust the paths in the next step accordingly.

2. On every control plane node, back up the API server manifest before editing:

cp -p /etc/kubernetes/manifests/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml.bak

3. On every control plane node, edit /etc/kubernetes/manifests/kube-apiserver.yaml to add or correct the --etcd-certfile and --etcd-keyfile arguments under the kube-apiserver container’s command list. For example (snippet only, keep existing args):

sed -i '/--etcd-servers=/a\    - --etcd-certfile=/etc/kubernetes/pki/etcd/healthcheck-client.crt\n    - --etcd-keyfile=/etc/kubernetes/pki/etcd/healthcheck-client.key' /etc/kubernetes/manifests/kube-apiserver.yaml

Adjust the certificate/key paths in the command if your etcd client certificate/key are different. Saving this file will cause the kubelet to restart the kube-apiserver static pod on that node.

4. On every control plane node, ensure the specified cert and key are readable by the kube-apiserver process (typically user root inside the pod):

ls -l /etc/kubernetes/pki/etcd/healthcheck-client.crt /etc/kubernetes/pki/etcd/healthcheck-client.key
chmod 600 /etc/kubernetes/pki/etcd/healthcheck-client.key
chmod 644 /etc/kubernetes/pki/etcd/healthcheck-client.crt

5. Wait for the kube-apiserver pod to be recreated, then verify it is running (from any machine with kubectl access):

kubectl get pods -n kube-system -l component=kube-apiserver -o wide

Ensure all kube-apiserver pods are in Running status.

6. On every control plane node, verify the kube-apiserver process now includes the --etcd-certfile and --etcd-keyfile flags with the correct paths:

/bin/ps -ef | grep kube-apiserver | grep -v grep

Confirm that the output shows --etcd-certfile= and --etcd-keyfile= pointing to your intended certificate and key files.

Using kubectl

kubectl cannot modify the kube-apiserver static pod manifest or its process flags, so it cannot be used to set --etcd-certfile and --etcd-keyfile. This must be fixed directly on every control plane node by editing /etc/kubernetes/manifests/kube-apiserver.yaml; see the Manual Steps section for the required host-level changes.

Automation

#!/usr/bin/env bash
#
# Hardens kube-apiserver etcd client TLS flags on all control plane nodes.
# Surface: host-level static pod manifest /etc/kubernetes/manifests/kube-apiserver.yaml
#
# REQUIREMENTS / ASSUMPTIONS:
# - Run on each control plane node (via SSH, Ansible shell, or similar).
# - etcd client cert/key already exist and are trusted by etcd.
#   Adjust CERT_FILE and KEY_FILE below if your paths differ.
#
# Usage:
#   sudo bash ./fix-etcd-certfile-keyfile.sh

set -euo pipefail

CERT_FILE="/etc/kubernetes/pki/apiserver-etcd-client.crt"
KEY_FILE="/etc/kubernetes/pki/apiserver-etcd-client.key"
MANIFEST="/etc/kubernetes/manifests/kube-apiserver.yaml"
BACKUP_DIR="/etc/kubernetes/manifests/backup-$(date +%Y%m%d-%H%M%S)"

echo "[INFO] Target manifest: ${MANIFEST}"

if [[ ! -f "${MANIFEST}" ]]; then
  echo "[ERROR] kube-apiserver manifest not found at ${MANIFEST}."
  echo "       Run this script only on a control plane node with static pod manifests."
  exit 1
fi

# Validate cert/key exist before changing anything
if [[ ! -f "${CERT_FILE}" ]]; then
  echo "[ERROR] etcd client certificate not found at ${CERT_FILE}."
  echo "       Create or place the correct certificate, then re-run this script."
  exit 1
fi

if [[ ! -f "${KEY_FILE}" ]]; then
  echo "[ERROR] etcd client key not found at ${KEY_FILE}."
  echo "       Create or place the correct key, then re-run this script."
  exit 1
fi

# One-time backup of manifest
mkdir -p "${BACKUP_DIR}"
cp -a "${MANIFEST}" "${BACKUP_DIR}/kube-apiserver.yaml"
echo "[INFO] Backup created at ${BACKUP_DIR}/kube-apiserver.yaml"

# Helper: ensure a --etcd-* flag exists (add if missing, update value if present).
ensure_flag() {
  local flag_name="$1"
  local flag_value="$2"
  local file="$3"

  if grep -qE "^[[:space:]]*- ${flag_name}=" "$file"; then
    # Update existing flag value idempotently
    sed -i "s|^\([[:space:]]*-\s\)${flag_name}=.*|\1${flag_name}=${flag_value}|" "$file"
  else
    # Insert under the 'command:' list, keeping YAML indentation
    # This assumes kubeadm-style manifest with a single 'command:' section.
    awk -v FLAG_NAME="${flag_name}" -v FLAG_VALUE="${flag_value}" '
      $1 == "command:" && in_cmd == 0 {
        print $0
        in_cmd = 1
        next
      }
      in_cmd == 1 && $1 ~ /^-/ && !inserted {
        print "    - " FLAG_NAME "=" FLAG_VALUE
        inserted = 1
      }
      { print $0 }
      END {
        if (in_cmd == 1 && !inserted) {
          # No existing flags after command:, append at end of file
          print "    - " FLAG_NAME "=" FLAG_VALUE
        }
      }
    ' "$file" > "${file}.tmp" && mv "${file}.tmp" "$file"
  fi
}

echo "[INFO] Ensuring --etcd-certfile and --etcd-keyfile flags are set in ${MANIFEST}"

ensure_flag "--etcd-certfile" "${CERT_FILE}" "${MANIFEST}"
ensure_flag "--etcd-keyfile"  "${KEY_FILE}"  "${MANIFEST}"

echo "[INFO] Manifest updated. kubelet will automatically restart the kube-apiserver static pod."
echo "[INFO] NOTE: Editing a static pod manifest causes the kube-apiserver container to restart."

# Wait for kube-apiserver process to come up with new flags
echo "[INFO] Waiting for kube-apiserver process to use updated etcd TLS flags..."
timeout=120
interval=5
end=$((SECONDS + timeout))
success=0

while (( SECONDS < end )); do
  if /bin/ps -ef | grep kube-apiserver | grep -v grep >/dev/null 2>&1; then
    # Verify both flags are present with correct values
    if /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q -- "--etcd-certfile=${CERT_FILE}" \
       && /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q -- "--etcd-keyfile=${KEY_FILE}"; then
      success=1
      break
    fi
  fi
  sleep "${interval}"
done

if [[ "${success}" -eq 1 ]]; then
  echo "[SUCCESS] kube-apiserver is running with:"
  echo "         --etcd-certfile=${CERT_FILE}"
  echo "         --etcd-keyfile=${KEY_FILE}"
  echo "[VERIFY] Current process flags:"
  /bin/ps -ef | grep kube-apiserver | grep -v grep
  exit 0
else
  echo "[WARNING] kube-apiserver process did not show the expected etcd TLS flags within ${timeout}s."
  echo "[ACTION] Manually check the manifest and kubelet status."
  echo "[DEBUG] Current kube-apiserver processes:"
  /bin/ps -ef | grep kube-apiserver | grep -v grep || true
  exit 1
fi

Additional Reading:

• https://kubernetes.io/docs/admin/kube-apiserver/