API Server Should Enable The AlwaysPullImages Admission

More Info:

Verifies that the AlwaysPullImages admission plugin is enabled so images are always pulled and re-authorized, preventing pods from using cached images they are not entitled to.

Risk Level

High

Address

Security

Compliance Standards

• CIS Kubernetes

Triage and Remediation

Remediation

Remediation

Manual Steps

1. On every control plane node, back up the current manifest:

   sudo cp -a /etc/kubernetes/manifests/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml.bak

2. On every control plane node, open the API server manifest for editing:

   sudo vi /etc/kubernetes/manifests/kube-apiserver.yaml

3. In the spec.containers[0].command list, locate the existing --enable-admission-plugins= argument.

   • If it exists, append ,AlwaysPullImages to the list, for example:

     - --enable-admission-plugins=NodeRestriction,AlwaysPullImages

   • If it does not exist, add a new line under the other -- flags, for example:

     - --enable-admission-plugins=AlwaysPullImages

4. Save the file and exit the editor. The kubelet will automatically restart the kube-apiserver static pod when the manifest changes; expect a brief control-plane disruption while it restarts.

5. On every control plane node, wait for the API server pod to become Running:

   sudo crictl ps | grep kube-apiserver

   Confirm the kube-apiserver container is in a running state.

6. On any control plane node, verify that the process now includes AlwaysPullImages in --enable-admission-plugins:

   /bin/ps -ef | grep kube-apiserver | grep -v grep

   Ensure the output shows an argument similar to: --enable-admission-plugins=...AlwaysPullImages...

Using kubectl

kubectl cannot be used to enable the AlwaysPullImages admission plugin because this setting is defined in the API server static pod manifest on each control plane node. To fix this finding, edit /etc/kubernetes/manifests/kube-apiserver.yaml directly on every control plane node as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
#
# Enable the AlwaysPullImages admission plugin on all control-plane nodes.
# Run this on each control-plane node (or via SSH/Ansible). Requires root.
#
# Operational impact:
#   - Editing /etc/kubernetes/manifests/kube-apiserver.yaml will trigger the
#     kubelet to restart the kube-apiserver static pod on this node.

set -euo pipefail

APISERVER_MANIFEST="/etc/kubernetes/manifests/kube-apiserver.yaml"
BACKUP_DIR="/etc/kubernetes/manifests/backup-$(date +%Y%m%d)"
ENABLE_FLAG="--enable-admission-plugins"

require_root() {
  if [ "$(id -u)" -ne 0 ]; then
    echo "This script must be run as root on each control-plane node." >&2
    exit 1
  fi
}

backup_manifest() {
  mkdir -p "${BACKUP_DIR}"
  if [ ! -f "${BACKUP_DIR}/kube-apiserver.yaml" ]; then
    cp "${APISERVER_MANIFEST}" "${BACKUP_DIR}/kube-apiserver.yaml"
    echo "Backup created at ${BACKUP_DIR}/kube-apiserver.yaml"
  else
    echo "Backup already exists at ${BACKUP_DIR}/kube-apiserver.yaml"
  fi
}

ensure_always_pull_images() {
  if ! grep -qE "^\s*- ${ENABLE_FLAG}" "${APISERVER_MANIFEST}"; then
    echo "No ${ENABLE_FLAG} flag found; adding with AlwaysPullImages."
    # Add the flag under the first occurrence of 'command:' in the container spec
    # This assumes the standard static pod manifest structure.
    tmpfile="$(mktemp)"
    awk -v flag="${ENABLE_FLAG}=AlwaysPullImages" '
      /command:/ && c==0 {
        print
        print "    - " flag
        c=1
        next
      }
      { print }
    ' "${APISERVER_MANIFEST}" > "${tmpfile}"
    mv "${tmpfile}" "${APISERVER_MANIFEST}"
  else
    echo "${ENABLE_FLAG} flag already present; ensuring AlwaysPullImages is included."
    tmpfile="$(mktemp)"
    awk -v flagname="${ENABLE_FLAG}" '
      $0 ~ "^[[:space:]]*- " flagname {
        # Line like: - --enable-admission-plugins=X,Y
        split($0, a, "=")
        if (length(a) == 2) {
          plugins=a[2]
          # Remove leading/trailing spaces
          sub(/^[[:space:]]*/, "", plugins)
          sub(/[[:space:]]*$/, "", plugins)
          # Check if AlwaysPullImages is present as a comma-separated token
          n=split(plugins, p, ",")
          found=0
          for (i=1; i<=n; i++) {
            if (p[i] == "AlwaysPullImages") {
              found=1
              break
            }
          }
          if (!found) {
            if (plugins == "") {
              plugins="AlwaysPullImages"
            } else {
              plugins=plugins ",AlwaysPullImages"
            }
          }
          print a[1] "=" plugins
        } else {
          # Malformed; just append AlwaysPullImages safely
          print $0 ",AlwaysPullImages"
        }
        next
      }
      { print }
    ' "${APISERVER_MANIFEST}" > "${tmpfile}"
    mv "${tmpfile}" "${APISERVER_MANIFEST}"
  fi
}

verify_process_flag() {
  echo "Waiting for kube-apiserver to restart with updated flags..."
  # Give kubelet some time to restart the static pod
  sleep 20

  # Verify with the audit-style command
  echo "Verification (process flags):"
  /bin/ps -ef | grep kube-apiserver | grep -v grep || {
    echo "kube-apiserver process not found. Check pod status." >&2
    return 1
  }

  if /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q -- "--enable-admission-plugins"; then
    if /bin/ps -ef | grep kube-apiserver | grep -v grep | grep -q "AlwaysPullImages"; then
      echo "SUCCESS: kube-apiserver is running with AlwaysPullImages enabled."
    else
      echo "WARNING: --enable-admission-plugins is set but AlwaysPullImages not detected in process args." >&2
      return 1
    fi
  else
    echo "WARNING: --enable-admission-plugins flag not detected in kube-apiserver process." >&2
    return 1
  fi
}

main() {
  require_root

  if [ ! -f "${APISERVER_MANIFEST}" ]; then
    echo "Manifest ${APISERVER_MANIFEST} not found on this node. Is this a control-plane node?" >&2
    exit 1
  fi

  backup_manifest
  ensure_always_pull_images
  verify_process_flag
}

main "$@"