Skip to main content

Ensure That A Client CA File Is Configured

More Info:​

Configuring a client CA file enables the kubelet to authenticate client certificates against a trusted certificate authority, preventing unverified clients from connecting.

Risk Level​

High

Address​

Security

Compliance Standards​

  • CIS EKS

Triage and Remediation​

Remediation​

Manual Steps

  1. On every worker node, SSH in and identify how kubelet is configured:

    ps -ef | grep kubelet
    • If you see a --config=/var/lib/kubelet/config.yaml (or similar) flag, follow Step 2.
    • If you see --client-ca-file=... directly in the arguments and no --config flag, skip to Step 3.

  2. If using /var/lib/kubelet/config.yaml, edit it to set authentication.x509.clientCAFile (create the hierarchy if missing):

    sudo vi /var/lib/kubelet/config.yaml

    Add or update under the top level:

    authentication:
      x509:
        clientCAFile: "/etc/kubernetes/pki/client-ca.crt"

    Ensure /etc/kubernetes/pki/client-ca.crt exists and contains the intended CA.

  3. If using executable arguments, edit the kubelet systemd drop-in on the worker node:

    sudo vi /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf

    In the line that sets kubelet arguments (e.g. KUBELET_ARGS= or ExecStart=), ensure it includes:

    --client-ca-file=/etc/kubernetes/pki/client-ca.crt

    Adjust the path if your client CA is stored elsewhere, and ensure the file exists.

  4. Reload systemd and restart kubelet on the worker node (required for any of the above changes to take effect):

    sudo systemctl daemon-reload
    sudo systemctl restart kubelet.service

  5. Check kubelet status for errors on the worker node:

    sudo systemctl status kubelet -l

  6. Verify kubelet is now running with a client CA configured on the worker node:

    /bin/ps -fC kubelet

    Confirm the output shows either a --config=/var/lib/kubelet/config.yaml (where the file contains authentication.x509.clientCAFile) or a --client-ca-file=/etc/kubernetes/pki/client-ca.crt (or your chosen CA path).

Using kubectl

kubectl cannot configure the kubelet’s clientCAFile because this setting lives in the kubelet host-level configuration (/var/lib/kubelet/config.yaml or kubelet service flags) on every worker node. Make this change directly on the nodes as described in the Manual Steps section.

Automation
#!/usr/bin/env bash
#
# Purpose: Ensure kubelet is configured with a client CA file on every worker node.
# Scope:   Run on EACH WORKER NODE via SSH (or with a fleet/orchestration tool).
# Impact:  This restarts the kubelet. On static-pod control planes, this also restarts
#          control-plane pods on that node.

set -euo pipefail

#-----------------------------
# Configuration (edit as needed)
#-----------------------------

# Desired client CA file path as used by kubelet
CLIENT_CA_FILE="/etc/kubernetes/pki/kubelet-client-ca.crt"

# Source CA file to install if CLIENT_CA_FILE does not yet exist.
# Adjust this to where your trusted CA actually lives.
SOURCE_CA_FILE="/etc/kubernetes/pki/ca.crt"

# Kubelet config file (as per the finding)
KUBELET_CONFIG_FILE="/var/lib/kubelet/config.yaml"

# Kubelet drop-in for CLI flags (EKS-style systemd units)
KUBELET_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf"

#-----------------------------
# Helper functions
#-----------------------------

log() {
  printf '[%s] %s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" "$*" >&2
}

ensure_ca_file() {
  if [[ -f "$CLIENT_CA_FILE" ]]; then
    log "Client CA file already present at $CLIENT_CA_FILE"
    return 0
  fi

  if [[ ! -f "$SOURCE_CA_FILE" ]]; then
    log "ERROR: SOURCE_CA_FILE $SOURCE_CA_FILE not found; cannot install client CA."
    log "Please place your trusted client CA at $CLIENT_CA_FILE and re-run."
    return 1
  fi

  log "Installing client CA file to $CLIENT_CA_FILE from $SOURCE_CA_FILE"
  install -o root -g root -m 0644 "$SOURCE_CA_FILE" "$CLIENT_CA_FILE"
}

backup_file() {
  local f="$1"
  if [[ -f "$f" ]]; then
    local ts
    ts="$(date -u +'%Y%m%dT%H%M%SZ')"
    local backup="${f}.bak.${ts}"
    cp -p "$f" "$backup"
    log "Backup of $f created at $backup"
  fi
}

configure_kubelet_via_config_file() {
  if [[ ! -f "$KUBELET_CONFIG_FILE" ]]; then
    log "Kubelet config file $KUBELET_CONFIG_FILE not found; skipping file-based config."
    return 1
  fi

  log "Configuring clientCAFile in $KUBELET_CONFIG_FILE"

  backup_file "$KUBELET_CONFIG_FILE"

  # Ensure authentication.x509.clientCAFile is present and correct.
  # This uses yq if available to be robust; otherwise falls back to a simple edit.
  if command -v yq >/dev/null 2>&1; then
    yq -y "
      .authentication //= {} |
      .authentication.x509 //= {} |
      .authentication.x509.clientCAFile = \"$CLIENT_CA_FILE\"
    " "$KUBELET_CONFIG_FILE" > "${KUBELET_CONFIG_FILE}.tmp"
    mv "${KUBELET_CONFIG_FILE}.tmp" "$KUBELET_CONFIG_FILE"
  else
    # Minimal sed/awk-based approach:
    # If 'authentication:' block missing, append it; then ensure clientCAFile line exists.
    if ! grep -qE '^\s*authentication:\s*$' "$KUBELET_CONFIG_FILE"; then
      cat >>"$KUBELET_CONFIG_FILE" <<EOF

authentication:
  x509:
    clientCAFile: $CLIENT_CA_FILE
EOF
    else
      # Ensure x509 and clientCAFile are present/updated under authentication:
      awk -v ca="$CLIENT_CA_FILE" '
        BEGIN { in_auth=0; in_x509=0; done=0 }
        /^\s*authentication:\s*$/ { in_auth=1; in_x509=0; print; next }
        in_auth && /^\s*[A-Za-z0-9_-]+:/ && !/authentication:/ { in_auth=0; in_x509=0 }
        in_auth && /^\s*x509:\s*$/ { in_x509=1; print; next }
        in_x509 && /^\s*[A-Za-z0-9_-]+:/ && !/clientCAFile:/ {
          if (!done) {
            printf "    clientCAFile: %s\n", ca
            done=1
          }
          in_x509=0
        }
        {
          print
        }
        END {
          if (in_auth && !in_x509 && !done) {
            print "  x509:"
            printf "    clientCAFile: %s\n", ca
          }
        }
      ' "$KUBELET_CONFIG_FILE" > "${KUBELET_CONFIG_FILE}.tmp"
      mv "${KUBELET_CONFIG_FILE}.tmp" "$KUBELET_CONFIG_FILE"
    fi
  fi

  log "Configured clientCAFile in $KUBELET_CONFIG_FILE"
  return 0
}

configure_kubelet_via_systemd_args() {
  if [[ ! -f "$KUBELET_DROPIN" ]]; then
    log "Kubelet drop-in $KUBELET_DROPIN not found; skipping arg-based config."
    return 1
  fi

  log "Configuring --client-ca-file in $KUBELET_DROPIN"

  backup_file "$KUBELET_DROPIN"

  # Ensure Environment= or KUBELET_ARGS contains the client-ca-file flag.
  if grep -q -- '--client-ca-file=' "$KUBELET_DROPIN"; then
    # Update existing flag value to desired path.
    sed -i "s#--client-ca-file=[^ \"']*#--client-ca-file=${CLIENT_CA_FILE}#g" "$KUBELET_DROPIN"
  else
    # Append the flag to KUBELET_ARGS or Environment line.
    if grep -q '^Environment="KUBELET_ARGS=' "$KUBELET_DROPIN"; then
      sed -i "s#^Environment=\"KUBELET_ARGS=\(.*\)\"#Environment=\"KUBELET_ARGS=\1 --client-ca-file=${CLIENT_CA_FILE}\"#" "$KUBELET_DROPIN"
    else
      # Add a generic Environment line if none exists.
      cat >>"$KUBELET_DROPIN" <<EOF

Environment="KUBELET_ARGS=--client-ca-file=${CLIENT_CA_FILE}"
EOF
    fi
  fi

  log "Configured --client-ca-file in $KUBELET_DROPIN"
  return 0
}

restart_kubelet() {
  if ! command -v systemctl >/dev/null 2>&1; then
    log "systemctl not available; please restart kubelet with your init system."
    return 1
  fi
  log "Reloading systemd and restarting kubelet"
  systemctl daemon-reload
  systemctl restart kubelet.service
  systemctl status kubelet.service -l --no-pager || true
}

verify() {
  log "Verifying kubelet client CA configuration"

  # Verify via config file if it exists
  if [[ -f "$KUBELET_CONFIG_FILE" ]]; then
    if grep -q "clientCAFile: ${CLIENT_CA_FILE}" "$KUBELET_CONFIG_FILE"; then
      log "OK: clientCAFile set in $KUBELET_CONFIG_FILE"
    else
      log "WARN: clientCAFile not found or mismatched in $KUBELET_CONFIG_FILE"
    fi
  fi

  # Verify via process flags
  if command -v ps >/dev/null 2>&1; then
    if ps -fC kubelet >/dev/null 2>&1; then
      local ps_out
      ps_out="$(ps -fC kubelet || true)"
      printf '%s\n' "$ps_out" | grep -q -- "--client-ca-file=${CLIENT_CA_FILE}" && \
        log "OK: kubelet process has --client-ca-file=${CLIENT_CA_FILE} flag" || \
        log "INFO: kubelet process does not show --client-ca-file=${CLIENT_CA_FILE} flag (may be using config file only)"
      log "Current kubelet process:\n$ps_out"
    else
      log "ERROR: kubelet process not found via 'ps -fC kubelet'"
    fi
  fi
}

#-----------------------------
# Main
#-----------------------------

if [[ "$(id -u)" -ne 0 ]]; then
  log "ERROR: This script must be run as root."
  exit 1
fi

log "Starting remediation to ensure kubelet client CA file is configured"

ensure_ca_file

configured=0
if configure_kubelet_via_config_file; then
  configured=1
fi

if [[ $configured -eq 0 ]]; then
  if configure_kubelet_via_systemd_args; then
    configured=1
  fi
fi

if [[ $configured -eq 0 ]]; then
  log "ERROR: Unable to locate kubelet config file or kubelet systemd drop-in."
  log "Please inspect kubelet startup configuration (e.g. 'ps -ef | grep kubelet') and set:"
  log "  --client-ca-file=${CLIENT_CA_FILE}"
  exit 1
fi

restart_kubelet
verify

log "Remediation completed on this worker node"