> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Ensure That The RotateKubeletServerCertificate Argument Is Set To True

### More Info:

Enabling the RotateKubeletServerCertificate feature gate lets the kubelet automatically request and rotate its serving certificate, avoiding the use of expired or long-lived server certificates.

### Risk Level

High

### Address

Security

### Compliance Standards

* CIS EKS

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Manual Steps" defaultOpen="true">
        1. On every worker node, check how kubelet is configured:
           ```bash theme={null}
           ps -fC kubelet
           ```
           Review the command line for any `--config` flag (config file) and any `--rotate-kubelet-server-certificate` flag (exec arg override).

        2. If using a kubelet config file (recommended), edit it and enable the feature gate (example path from finding):
           ```bash theme={null}
           sudo sed -i 's/^featureGates:.*/featureGates:\n  RotateKubeletServerCertificate: true/' /var/lib/kubelet/config.yaml
           ```
           If `featureGates` is not present, add this block under the top-level YAML:
           ```bash theme={null}
           sudo tee -a /var/lib/kubelet/config.yaml >/dev/null << 'EOF'
           ```

        featureGates:
        RotateKubeletServerCertificate: true
        EOF

        ````

        3. Ensure no systemd drop-in overrides the setting to false (if these files exist):  
        ```bash
        sudo grep -R --color=always 'rotate-kubelet-server-certificate' /etc/systemd/system/kubelet.service.d || true
        ````

        If you see `--rotate-kubelet-server-certificate=false` in `/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf`, edit it:

        ```bash theme={null}
        sudo sed -i 's/--rotate-kubelet-server-certificate=false/--rotate-kubelet-server-certificate=true/g' /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf
        ```

        Or remove the flag entirely so the config file controls it.

        4. If you are configuring kubelet only with executable arguments (no config file referenced in the `ps` output), append the flag in the systemd drop-in (create or edit as needed):
           ```bash theme={null}
           sudo mkdir -p /etc/systemd/system/kubelet.service.d
           sudo tee /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf >/dev/null << 'EOF'
           ```

        \[Service]
        Environment="KUBELET\_ARGS=--rotate-kubelet-server-certificate=true"
        EOF

        ````

        5. Restart kubelet on the worker node to apply changes (operational impact: this briefly restarts the kubelet and may momentarily affect pod reporting):  
        ```bash
        sudo systemctl daemon-reload
        sudo systemctl restart kubelet.service
        sudo systemctl status kubelet -l
        ````

        6. Verify kubelet is running with the correct setting on the worker node:
           ```bash theme={null}
           ps -fC kubelet
           ```
           Confirm either that `--rotate-kubelet-server-certificate=true` appears in the command line, or that no `--rotate-kubelet-server-certificate=false` flag is present and the config file has `RotateKubeletServerCertificate: true`.
      </Accordion>

      <Accordion title="Using kubectl">
        kubectl cannot be used to enable `RotateKubeletServerCertificate` because this setting is applied via kubelet host-level configuration (`/var/lib/kubelet/config.yaml`, `/etc/kubernetes/kubelet/kubelet-config.json`, and the kubelet systemd unit) on every worker node. To remediate this finding, make the changes directly on each node as described in the Manual Steps section.
      </Accordion>

      <Accordion title="Automation">
        ```bash theme={null}
        #!/usr/bin/env bash
        #
        # Enable RotateKubeletServerCertificate on all worker nodes.
        # Run on each worker node as root.
        #
        # Behavior:
        # - Ensures featureGates.RotateKubeletServerCertificate=true in kubelet config file if present
        # - Ensures no kubelet systemd args set it to false
        # - Adds --rotate-kubelet-server-certificate=true to kubelet args if using flags
        # - Restarts kubelet only if a change was made
        # - Verifies setting via kubelet configz endpoint if possible, otherwise via process args

        set -euo pipefail

        CHANGED=0

        echo "[INFO] Starting kubelet RotateKubeletServerCertificate remediation"

        ########################################
        # Helper: backup a file once per run
        ########################################
        backup_file() {
          local f="$1"
          if [[ -f "$f" ]]; then
            local backup="${f}.$(date +%Y%m%d%H%M%S).bak"
            cp -p "$f" "$backup"
            echo "[INFO] Backed up $f to $backup"
          fi
        }

        ########################################
        # Method 1: kubelet config file
        ########################################
        # Primary path from finding
        CONFIG_FILE="/var/lib/kubelet/config.yaml"
        ALT_CONFIG_JSON="/etc/kubernetes/kubelet/kubelet-config.json"

        update_yaml_feature_gate() {
          local file="$1"
          if ! command -v python3 >/dev/null 2>&1; then
            echo "[WARN] python3 not available; skipping structured edit of $file"
            return 1
          fi

          backup_file "$file"

          python3 - <<'PYEOF'
        import sys, yaml, os, datetime

        path = os.environ["KUBELET_CFG_PATH"]
        with open(path) as f:
            data = yaml.safe_load(f) or {}

        fg = data.get("featureGates")
        if fg is None:
            fg = {}
            data["featureGates"] = fg

        orig = fg.get("RotateKubeletServerCertificate")
        if orig is True:
            # already correct
            sys.exit(10)

        fg["RotateKubeletServerCertificate"] = True

        bak = f"{path}.{datetime.datetime.now().strftime('%Y%m%d%H%M%S')}.pybak"
        with open(bak, "w") as f:
            yaml.safe_dump(data, f, default_flow_style=False)

        with open(path, "w") as f:
            yaml.safe_dump(data, f, default_flow_style=False)
        PYEOF
          rc=$?
          if [[ $rc -eq 0 ]]; then
            echo "[INFO] Updated RotateKubeletServerCertificate in $file"
            return 0
          elif [[ $rc -eq 10 ]]; then
            echo "[INFO] RotateKubeletServerCertificate already true in $file"
            return 2
          else
            echo "[WARN] Failed to update $file via python/yaml"
            return 1
          fi
        }

        # Try YAML config first if it exists
        if [[ -f "$CONFIG_FILE" ]]; then
          echo "[INFO] Detected kubelet config file at $CONFIG_FILE"
          KUBELET_CFG_PATH="$CONFIG_FILE" update_yaml_feature_gate "$CONFIG_FILE" || true
          case $? in
            0) CHANGED=1 ;;
            2) : ;; # no change
          esac
        fi

        # Try legacy JSON config if present (per remediation Method 1)
        if [[ -f "$ALT_CONFIG_JSON" ]]; then
          echo "[INFO] Detected kubelet config file at $ALT_CONFIG_JSON"
          if command -v python3 >/dev/null 2>&1; then
            backup_file "$ALT_CONFIG_JSON"
            python3 - <<'PYEOF'
        import sys, json, os, datetime

        path = os.environ["KUBELET_CFG_JSON"]
        with open(path) as f:
            data = json.load(f)

        fg = data.get("featureGates")
        if fg is None:
            fg = {}
            data["featureGates"] = fg

        orig = fg.get("RotateKubeletServerCertificate")
        if orig is True:
            sys.exit(10)

        fg["RotateKubeletServerCertificate"] = True

        bak = f"{path}.{datetime.datetime.now().strftime('%Y%m%d%H%M%S')}.pybak"
        with open(bak, "w") as f:
            json.dump(data, f, indent=2, sort_keys=True)

        with open(path, "w") as f:
            json.dump(data, f, indent=2, sort_keys=True)
        PYEOF
            rc=$?
            if [[ $rc -eq 0 ]]; then
              echo "[INFO] Updated RotateKubeletServerCertificate in $ALT_CONFIG_JSON"
              CHANGED=1
            elif [[ $rc -eq 10 ]]; then
              echo "[INFO] RotateKubeletServerCertificate already true in $ALT_CONFIG_JSON"
            else
              echo "[WARN] Failed to update $ALT_CONFIG_JSON via python/json"
            fi
          else
            echo "[WARN] python3 not available; cannot safely edit $ALT_CONFIG_JSON"
          fi
        fi

        ########################################
        # Method 2: systemd kubelet args
        ########################################
        KUBELET_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf"
        if [[ -f "$KUBELET_DROPIN" ]]; then
          echo "[INFO] Processing kubelet systemd drop-in at $KUBELET_DROPIN"
          backup_file "$KUBELET_DROPIN"

          # Remove any explicit false override
          if grep -q -- '--rotate-kubelet-server-certificate=false' "$KUBELET_DROPIN"; then
            sed -i 's/--rotate-kubelet-server-certificate=false//g' "$KUBELET_DROPIN"
            echo "[INFO] Removed --rotate-kubelet-server-certificate=false from $KUBELET_DROPIN"
            CHANGED=1
          fi

          # Ensure true flag present in either Environment or KUBELET_ARGS
          if ! grep -q -- '--rotate-kubelet-server-certificate=true' "$KUBELET_DROPIN"; then
            if grep -q '^Environment="KUBELET_ARGS=' "$KUBELET_DROPIN"; then
              # Append to existing KUBELET_ARGS
              sed -i 's/^Environment="KUBELET_ARGS=\(.*\)"/Environment="KUBELET_ARGS=\1 --rotate-kubelet-server-certificate=true"/' "$KUBELET_DROPIN"
              echo "[INFO] Appended --rotate-kubelet-server-certificate=true to existing KUBELET_ARGS"
            else
              # Add a new Environment line
              cat <<'EOF' >> "$KUBELET_DROPIN"
        Environment="KUBELET_ARGS=--rotate-kubelet-server-certificate=true"
        EOF
              echo "[INFO] Added new KUBELET_ARGS with --rotate-kubelet-server-certificate=true"
            fi
            CHANGED=1
          else
            echo "[INFO] --rotate-kubelet-server-certificate=true already present in $KUBELET_DROPIN"
          fi
        else
          echo "[INFO] No kubelet systemd drop-in at $KUBELET_DROPIN; assuming config file or other mechanism controls feature gates"
        fi

        ########################################
        # Restart kubelet if configuration changed
        ########################################
        if [[ "$CHANGED" -eq 1 ]]; then
          echo "[INFO] Changes detected; restarting kubelet"
          systemctl daemon-reload
          systemctl restart kubelet.service
        else
          echo "[INFO] No changes made; kubelet restart not required"
        fi

        echo "[INFO] Kubelet status:"
        systemctl status kubelet -l --no-pager || true

        ########################################
        # Verification (adapted from audit)
        ########################################
        echo "[INFO] Verifying RotateKubeletServerCertificate setting"

        # Preferred: configz endpoint (if running and accessible)
        VERIFY_OK=0
        if command -v curl >/dev/null 2>&1 && command -v jq >/dev/null 2>&1; then
          # Try common localhost configz endpoint
          if curl -fsS "http://127.0.0.1:10248/configz" >/dev/null 2>&1; then
            if curl -fsS "http://127.0.0.1:10248/configz" \
              | jq -e '.kubeletconfig.featureGates.RotateKubeletServerCertificate == true' >/dev/null 2>&1; then
              echo "[INFO] Verified via configz: RotateKubeletServerCertificate=true"
              VERIFY_OK=1
            else
              echo "[WARN] configz reachable but RotateKubeletServerCertificate is not true"
            fi
          fi
        fi

        # Fallback: check process arguments (original audit path)
        if [[ "$VERIFY_OK" -ne 1 ]]; then
          echo "[INFO] Falling back to process-args verification"
          if /bin/ps -fC kubelet | grep -q -- '--rotate-kubelet-server-certificate=true'; then
            echo "[INFO] Verified: kubelet process includes --rotate-kubelet-server-certificate=true"
            VERIFY_OK=1
          else
            echo "[WARN] kubelet process does not show --rotate-kubelet-server-certificate=true"
          fi
        fi

        if [[ "$VERIFY_OK" -eq 1 ]]; then
          echo "[INFO] Remediation verification PASSED on this node"
          exit 0
        else
          echo "[ERROR] Remediation verification FAILED on this node"
          exit 1
        fi
        ```
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>