> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# API Server Should Set Kubelet Client Certificate And Key

### More Info:

Verifies that --kubelet-client-certificate and --kubelet-client-key are set so the API server authenticates to kubelets over TLS. Without them the apiserver-kubelet connection is not mutually authenticated.

### Risk Level

High

### Address

Security

### Compliance Standards

* CIS Kubernetes

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Manual Steps" defaultOpen="true">
        1. Generate or obtain a dedicated client certificate and key for the apiserver to authenticate to kubelets (run on any secure machine, then copy to each control plane node):

        ```bash theme={null}
        # Example: create a key and CSR (adjust CN/O/hosts per your PKI policy)
        openssl genrsa -out kubelet-client.key 2048
        openssl req -new -key kubelet-client.key -out kubelet-client.csr -subj "/CN=kube-apiserver-kubelet-client"
        # Sign kubelet-client.csr with your cluster CA, producing kubelet-client.crt
        # Then securely copy kubelet-client.crt and kubelet-client.key to each control plane node:
        scp kubelet-client.crt kubelet-client.key root@CONTROL_PLANE_IP:/etc/kubernetes/pki/
        ```

        2. On every control plane node, back up the existing API server manifest:

        ```bash theme={null}
        sudo cp -a /etc/kubernetes/manifests/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml.bak
        ```

        3. On every control plane node, edit the API server static pod manifest to add the kubelet client certificate and key flags (this edit will cause the kube-apiserver static pod to restart):

        ```bash theme={null}
        sudo sed -i '/- kube-apiserver/a\  - --kubelet-client-certificate=/etc/kubernetes/pki/kubelet-client.crt\n  - --kubelet-client-key=/etc/kubernetes/pki/kubelet-client.key' /etc/kubernetes/manifests/kube-apiserver.yaml
        ```

        Confirm the resulting `kube-apiserver.yaml` includes:

        ```yaml theme={null}
            - kube-apiserver
            - --kubelet-client-certificate=/etc/kubernetes/pki/kubelet-client.crt
            - --kubelet-client-key=/etc/kubernetes/pki/kubelet-client.key
        ```

        4. Ensure file permissions on the certificate and key are appropriately restricted on every control plane node:

        ```bash theme={null}
        sudo chown root:root /etc/kubernetes/pki/kubelet-client.crt /etc/kubernetes/pki/kubelet-client.key
        sudo chmod 600 /etc/kubernetes/pki/kubelet-client.key
        sudo chmod 644 /etc/kubernetes/pki/kubelet-client.crt
        ```

        5. Wait for the kube-apiserver static pod to be recreated and become Ready (run on any machine with kubectl configured):

        ```bash theme={null}
        kubectl get pods -n kube-system -l component=kube-apiserver -o wide
        ```

        Do not proceed until all kube-apiserver pods show `STATUS` as `Running` and `READY` as `1/1`.

        6. Verify on every control plane node that the kube-apiserver process is now running with the required flags:

        ```bash theme={null}
        /bin/ps -ef | grep kube-apiserver | grep -v grep
        ```

        Confirm the output includes both `--kubelet-client-certificate=/etc/kubernetes/pki/kubelet-client.crt` and `--kubelet-client-key=/etc/kubernetes/pki/kubelet-client.key`.
      </Accordion>

      <Accordion title="Using kubectl">
        kubectl cannot modify the API server’s host-level configuration or the static pod manifest at `/etc/kubernetes/manifests/kube-apiserver.yaml` on control plane nodes. To remediate this finding, you must edit that file directly on every control plane node; see the Manual Steps section for the exact procedure.
      </Accordion>

      <Accordion title="Automation">
        ```bash theme={null}
        #!/usr/bin/env bash
        #
        # automate_kubelet_client_tls.sh
        #
        # Idempotently ensure kube-apiserver is configured with
        # --kubelet-client-certificate and --kubelet-client-key on every control plane node.
        #
        # RUN THIS ON: every control plane node (with root privileges).
        #
        # Requirements:
        #   - /etc/kubernetes/manifests/kube-apiserver.yaml (static pod)
        #   - OpenSSL available
        #
        # Behavior:
        #   - Creates a dedicated kubelet client cert/key if missing
        #   - Updates kube-apiserver manifest to use them
        #   - Kubelet will auto-restart the kube-apiserver static pod
        #   - Safe to re-run

        set -euo pipefail

        APISERVER_MANIFEST="/etc/kubernetes/manifests/kube-apiserver.yaml"
        CERT_DIR="/etc/kubernetes/pki"
        CLIENT_KEY="${CERT_DIR}/apiserver-kubelet-client.key"
        CLIENT_CSR="${CERT_DIR}/apiserver-kubelet-client.csr"
        CLIENT_CERT="${CERT_DIR}/apiserver-kubelet-client.crt"
        CA_CERT="${CERT_DIR}/ca.crt"
        CA_KEY="${CERT_DIR}/ca.key"
        BACKUP_SUFFIX="$(date +%Y%m%d%H%M%S)"

        if [[ $EUID -ne 0 ]]; then
          echo "ERROR: This script must be run as root." >&2
          exit 1
        fi

        if [[ ! -f "${APISERVER_MANIFEST}" ]]; then
          echo "ERROR: kube-apiserver manifest not found at ${APISERVER_MANIFEST}" >&2
          exit 1
        fi

        if [[ ! -f "${CA_CERT}" || ! -f "${CA_KEY}" ]]; then
          echo "ERROR: CA files ${CA_CERT} and/or ${CA_KEY} not found." >&2
          echo "       Ensure cluster CA is present before running this script." >&2
          exit 1
        fi

        mkdir -p "${CERT_DIR}"
        chmod 700 "${CERT_DIR}"

        # 1) Create kubelet client key/cert if missing
        if [[ ! -f "${CLIENT_KEY}" || ! -f "${CLIENT_CERT}" ]]; then
          echo "Generating kubelet client key and certificate in ${CERT_DIR}..."

          openssl genrsa -out "${CLIENT_KEY}" 2048
          chmod 600 "${CLIENT_KEY}"

          cat > "${CLIENT_CSR}.cnf" <<EOF
        [ req ]
        default_bits       = 2048
        prompt             = no
        distinguished_name = dn
        req_extensions     = v3_req

        [ dn ]
        CN = kube-apiserver-kubelet-client

        [ v3_req ]
        keyUsage = critical, digitalSignature, keyEncipherment
        extendedKeyUsage = clientAuth
        EOF

          openssl req -new -key "${CLIENT_KEY}" -out "${CLIENT_CSR}" -config "${CLIENT_CSR}.cnf"

          openssl x509 -req -in "${CLIENT_CSR}" \
            -CA "${CA_CERT}" -CAkey "${CA_KEY}" -CAcreateserial \
            -out "${CLIENT_CERT}" -days 3650 -extensions v3_req -extfile "${CLIENT_CSR}.cnf"

          chmod 644 "${CLIENT_CERT}"
          rm -f "${CLIENT_CSR}" "${CLIENT_CSR}.cnf"
        fi

        # 2) Backup existing manifest once per run
        BACKUP_FILE="${APISERVER_MANIFEST}.${BACKUP_SUFFIX}.bak"
        cp "${APISERVER_MANIFEST}" "${BACKUP_FILE}"
        chmod --reference="${APISERVER_MANIFEST}" "${BACKUP_FILE}"
        echo "Backup of kube-apiserver manifest created at ${BACKUP_FILE}"

        # 3) Ensure flags exist or are updated
        #   Handles both "--flag=value" and "--flag value" styles idempotently.

        ensure_flag() {
          local flag="$1"
          local value="$2"
          local file="$3"
          local tmp
          tmp="$(mktemp)"

          # Remove any existing occurrences (both styles)
          sed -E "/${flag}(=|\s)/d" "${file}" > "${tmp}"
          mv "${tmp}" "${file}"

          # Add new flag under the 'command:' or argument list
          # We simply append it as a new line with proper YAML list item.
          # This is safe for manifests using standard 'command: [...]' args.
          if grep -qE '^\s*- kube-apiserver' "${file}"; then
            # Append directly after the kube-apiserver command entry
            tmp="$(mktemp)"
            awk -v flag="${flag}" -v value="${value}" '
              {
                print $0
                if ($0 ~ /^\s*- kube-apiserver/) {
                  print "    - " flag "=" value
                }
              }' "${file}" > "${tmp}"
            mv "${tmp}" "${file}"
          else
            # Fallback: append at end of file as an extra argument line
            echo "    - ${flag}=${value}" >> "${file}"
          fi
        }

        ensure_flag "--kubelet-client-certificate" "${CLIENT_CERT}" "${APISERVER_MANIFEST}"
        ensure_flag "--kubelet-client-key" "${CLIENT_KEY}" "${APISERVER_MANIFEST}"

        echo "Updated ${APISERVER_MANIFEST} with kubelet client certificate and key flags."
        echo "NOTE: Editing a static pod manifest causes kubelet to restart the kube-apiserver pod."

        # 4) Verification: confirm kube-apiserver is running with the expected flags
        echo "Waiting for kube-apiserver process to pick up new flags..."
        sleep 20

        echo "Verification output (should show kubelet-client-certificate and kubelet-client-key):"
        /bin/ps -ef | grep kube-apiserver | grep -v grep || {
          echo "WARNING: kube-apiserver process not found. Check kubelet and pod status." >&2
          exit 1
        }

        if /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q -- "--kubelet-client-certificate=${CLIENT_CERT}" \
           && /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q -- "--kubelet-client-key=${CLIENT_KEY}"; then
          echo "SUCCESS: kube-apiserver is running with the configured kubelet client certificate and key."
        else
          echo "ERROR: kube-apiserver process does not show the expected kubelet client TLS flags." >&2
          echo "       Inspect ${APISERVER_MANIFEST} and kube-apiserver pod logs." >&2
          exit 1
        fi
        ```
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>