Skip to main content

Ensure Rotate Kubelet Server Certificate Argument Is Enabled

More Info:​

Enable kubelet server certificate rotation.

Risk Level​

Low

Address​

Security

Compliance Standards​

  • APRA CPS 234 (Australia)
  • BSI C5 (Germany)
  • Brazil LGPD
  • CCPA / CPRA (California)
  • CIS Critical Security Controls v8
  • CIS EKS
  • CMMC 2.0
  • CSA Cloud Controls Matrix v4
  • DPDPA
  • Digital Operational Resilience Act (EU)
  • Essential 8
  • ISO/IEC 27017
  • ISO/IEC 27018
  • ISO/IEC 27701
  • KSA PDPL
  • MAS Technology Risk Management (Singapore)
  • MITRE ATT&CK (Cloud)
  • NIST SP 800-171
  • NYDFS 23 NYCRR 500
  • SWIFT Customer Security Controls Framework
  • Sarbanes-Oxley IT General Controls
  • UK NCSC Cyber Assessment Framework

Triage and Remediation​

Remediation​

Manual Steps

  1. On every worker node, open the kubelet config file and enable the feature gate:

    sudo sed -i '/^featureGates:/,/^[^ ]/d' /var/lib/kubelet/config.yaml
    sudo tee -a /var/lib/kubelet/config.yaml >/dev/null <<'EOF'
    featureGates:
      RotateKubeletServerCertificate: true
    EOF

  2. On every worker node, ensure no conflicting flag is set in the systemd drop-in; remove any existing --rotate-kubelet-server-certificate setting:

    sudo sed -i 's/--rotate-kubelet-server-certificate=[^ ]*//g' /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf

  3. (Optional alternative to step 1 if you prefer flags instead of config file; still on every worker node) Append the flag to the kubelet args line:

    sudo sed -i 's/^KUBELET_ARGS="/KUBELET_ARGS="--rotate-kubelet-server-certificate=true /' /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf

  4. On every worker node, reload systemd and restart kubelet (this restarts the kubelet process, affecting node registration/Pods briefly):

    sudo systemctl daemon-reload
    sudo systemctl restart kubelet.service
    sudo systemctl status kubelet -l --no-pager

  5. On any machine with kubectl access, verify kubelet config via the configz endpoint for each worker node (replace NODE_NAME with each node’s name from kubectl get nodes):

    kubectl proxy --port=8001 >/tmp/kubectl-proxy.log 2>&1 &
    export HOSTNAME_PORT=localhost:8001
    export NODE_NAME=<worker-node-name>
    curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz" | grep -A3 RotateKubeletServerCertificate

  6. On every worker node, confirm the running kubelet process includes the correct setting and no false override:

    /bin/ps -fC kubelet

    Inspect the output to ensure either:

    • no --rotate-kubelet-server-certificate flag is present (so the config file controls it and configz shows RotateKubeletServerCertificate:true), or
    • if present, it is --rotate-kubelet-server-certificate=true and not false.
Using kubectl

kubectl cannot be used to enable RotateKubeletServerCertificate because this setting is applied via kubelet host-level configuration on each worker node (for example /etc/kubernetes/kubelet/kubelet-config.json, /var/lib/kubelet/config.yaml, and the systemd unit drop-in). Make the changes directly on every worker node as described in the Manual Steps section and then restart the kubelet service.

Automation
#!/usr/bin/env bash
#
# Enable RotateKubeletServerCertificate on all worker nodes (run via SSH, Ansible, or similar).
# This script is idempotent and safe to re-run.
#
# Usage (example with a single node):
#   ssh root@worker-node 'bash -s' < enable-kubelet-server-cert-rotation.sh
#
# Apply this to every worker node.

set -euo pipefail

KUBELET_CONFIG_FILE="/var/lib/kubelet/config.yaml"
KUBELET_ARGS_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf"

changed=0

echo "=== [$(hostname)] Configuring kubelet server certificate rotation ==="

#############################################
# 1) Ensure featureGates in kubelet config  #
#############################################

if [[ -f "${KUBELET_CONFIG_FILE}" ]]; then
  echo "-> Updating ${KUBELET_CONFIG_FILE}"

  # Backup once per run
  cp -a "${KUBELET_CONFIG_FILE}" "${KUBELET_CONFIG_FILE}.$(date +%Y%m%d%H%M%S).bak"

  # Ensure featureGates block exists; use yq if available, otherwise do minimal YAML edit
  if command -v yq >/dev/null 2>&1; then
    # Use yq to set featureGates.RotateKubeletServerCertificate=true
    yq -i '.featureGates.RotateKubeletServerCertificate = true' "${KUBELET_CONFIG_FILE}"
  else
    # Minimal YAML manipulation without external tools
    # If featureGates block exists, replace/ensure key; else append block at end
    if grep -qE '^featureGates:' "${KUBELET_CONFIG_FILE}"; then
      # Remove any existing RotateKubeletServerCertificate entry
      sed -i '/featureGates:/,/^[^[:space:]]/ s/^[[:space:]]*RotateKubeletServerCertificate:.*$/  RotateKubeletServerCertificate: true/' "${KUBELET_CONFIG_FILE}"
      # If key was not present, append it under existing featureGates:
      if ! sed -n '/featureGates:/,/^[^[:space:]]/p' "${KUBELET_CONFIG_FILE}" | grep -q 'RotateKubeletServerCertificate:'; then
        # Insert after the 'featureGates:' line
        awk '
          /^featureGates:/ && !added { print; print "  RotateKubeletServerCertificate: true"; added=1; next }
          { print }
        ' "${KUBELET_CONFIG_FILE}" > "${KUBELET_CONFIG_FILE}.tmp" && mv "${KUBELET_CONFIG_FILE}.tmp" "${KUBELET_CONFIG_FILE}"
      fi
    else
      cat <<'EOF' >> "${KUBELET_CONFIG_FILE}"

featureGates:
  RotateKubeletServerCertificate: true
EOF
    fi
  fi

  changed=1
else
  echo "-> WARNING: ${KUBELET_CONFIG_FILE} not found; skipping featureGates update"
fi

##############################################################
# 2) Ensure no flag overrides to false; set flag to true     #
#    in 10-kubelet-args.conf when using executable args      #
##############################################################

if [[ -f "${KUBELET_ARGS_DROPIN}" ]]; then
  echo "-> Updating ${KUBELET_ARGS_DROPIN}"

  cp -a "${KUBELET_ARGS_DROPIN}" "${KUBELET_ARGS_DROPIN}.$(date +%Y%m%d%H%M%S).bak"

  # Remove any explicit false setting
  sed -i 's/--rotate-kubelet-server-certificate=false//g' "${KUBELET_ARGS_DROPIN}"

  # Ensure true flag is present at the end of KUBELET_ARGS or ExecStart line

  if grep -q 'KUBELET_ARGS' "${KUBELET_ARGS_DROPIN}"; then
    if ! grep -q -- '--rotate-kubelet-server-certificate=true' "${KUBELET_ARGS_DROPIN}"; then
      # Append to existing KUBELET_ARGS
      awk '
        /KUBELET_ARGS/ && !done {
          sub(/"$/, " --rotate-kubelet-server-certificate=true\"")
          done=1
        }
        { print }
      ' "${KUBELET_ARGS_DROPIN}" > "${KUBELET_ARGS_DROPIN}.tmp" && mv "${KUBELET_ARGS_DROPIN}.tmp" "${KUBELET_ARGS_DROPIN}"
      changed=1
    fi
  elif grep -q '^ExecStart=' "${KUBELET_ARGS_DROPIN}"; then
    if ! grep -q -- '--rotate-kubelet-server-certificate=true' "${KUBELET_ARGS_DROPIN}"; then
      sed -i 's#^ExecStart=\(.*kubelet.*\)$#ExecStart=\1 --rotate-kubelet-server-certificate=true#' "${KUBELET_ARGS_DROPIN}"
      changed=1
    fi
  else
    # If the drop-in is present but does not define args, leave it unchanged.
    echo "-> INFO: ${KUBELET_ARGS_DROPIN} does not define kubelet args; no changes"
  fi
else
  echo "-> INFO: ${KUBELET_ARGS_DROPIN} not found; assuming kubelet fully configured via config file"
fi

########################################
# 3) Reload systemd and restart kubelet
########################################

if (( changed == 1 )); then
  echo "-> Reloading systemd and restarting kubelet (this will restart the kubelet on this node)"
  systemctl daemon-reload
  systemctl restart kubelet.service
else
  echo "-> No configuration changes detected; kubelet restart not required"
fi

echo "-> Checking kubelet status"
systemctl status kubelet.service -l --no-pager || true

########################################
# 4) Verification                      #
########################################

echo "=== Verification: kubelet process flags ==="
/bin/ps -fC kubelet || true

echo
echo "=== Verification: RotateKubeletServerCertificate setting ==="

if command -v yq >/dev/null 2>&1 && [[ -f "${KUBELET_CONFIG_FILE}" ]]; then
  echo "- From ${KUBELET_CONFIG_FILE} (via yq):"
  yq '.featureGates.RotateKubeletServerCertificate' "${KUBELET_CONFIG_FILE}" 2>/dev/null || echo "  (key not found)"
else
  if [[ -f "${KUBELET_CONFIG_FILE}" ]]; then
    echo "- From ${KUBELET_CONFIG_FILE} (grep):"
    grep -A3 '^featureGates:' "${KUBELET_CONFIG_FILE}" || echo "  (featureGates block not found)"
  else
    echo "- ${KUBELET_CONFIG_FILE} not present; skipping file-based verification"
  fi
fi

echo
echo "=== Completed on $(hostname) ==="

Additional Reading:​