Skip to main content

Container-Optimized Os (Cos) Is Used For Gke Node Images

More Info:​

Use Container-Optimized Os (Cos) As A Managed, Optimized And Hardened Base Os That Limits The HostS Attack Surface.

Risk Level​

High

Address​

Security

Compliance Standards​

  • APRA CPS 234 (Australia)
  • BSI C5 (Germany)
  • Brazil LGPD
  • CCPA / CPRA (California)
  • CIS Critical Security Controls v8
  • CIS GKE
  • CMMC 2.0
  • CSA Cloud Controls Matrix v4
  • DPDPA
  • Digital Operational Resilience Act (EU)
  • Essential 8
  • ISO/IEC 27017
  • ISO/IEC 27018
  • ISO/IEC 27701
  • KSA PDPL
  • MAS Technology Risk Management (Singapore)
  • MITRE ATT&CK (Cloud)
  • NIS2 Directive
  • NIST SP 800-171
  • NYDFS 23 NYCRR 500
  • SWIFT Customer Security Controls Framework
  • Sarbanes-Oxley IT General Controls
  • UK NCSC Cyber Assessment Framework

Triage and Remediation​

Remediation​

Manual Steps

  1. List all clusters and node pools using non-COS images

    • Run on: any machine with gcloud and project access.
    PROJECT_ID="<PROJECT_ID>"
    gcloud config set project "${PROJECT_ID}"

    for CLUSTER in $(gcloud container clusters list --format='value(name)'); do
      LOCATION=$(gcloud container clusters list --filter="name=${CLUSTER}" --format='value(location)')
      echo "Cluster: ${CLUSTER} (${LOCATION})"
      gcloud container node-pools list \
        --cluster "${CLUSTER}" \
        --location "${LOCATION}" \
        --format='table(name,config.imageType)'
      echo
    done
    • Identify node pools where config.imageType is not cos_containerd.

  2. Confirm workloads and constraints that might prevent migration

    • For each affected cluster, list Pods and note any privileged/host-level requirements:
    CLUSTER_NAME="<CLUSTER_NAME>"
    LOCATION="<LOCATION>"

    gcloud container clusters get-credentials "${CLUSTER_NAME}" --location "${LOCATION}"

    kubectl get pods -A -o wide
    kubectl get podsecuritypolicy -A 2>/dev/null || true
    kubectl get constraints -A 2>/dev/null || true
    • Review whether any workloads rely on OS-specific features or custom images incompatible with COS.

  3. Review autoscaling, maintenance windows, and disruption impact

    • Check node pool size and autoscaling to understand rolling upgrade impact:
    CLUSTER_NAME="<CLUSTER_NAME>"
    LOCATION="<LOCATION>"

    gcloud container node-pools list \
      --cluster "${CLUSTER_NAME}" \
      --location "${LOCATION}" \
      --format='table(name,initialNodeCount,autoscaling.enabled,autoscaling.minNodeCount,autoscaling.maxNodeCount,management.upgradeOptions.autoUpgrade)'
    • Decide whether a rolling upgrade is acceptable now or requires a maintenance window.

  4. Plan the change and, if acceptable, update node pools to cos_containerd

    • For each node pool you decide to migrate:
    CLUSTER_NAME="<CLUSTER_NAME>"
    LOCATION="<LOCATION>"
    NODE_POOL_NAME="<NODE_POOL_NAME>"

    gcloud container clusters upgrade "${CLUSTER_NAME}" \
      --location "${LOCATION}" \
      --node-pool "${NODE_POOL_NAME}" \
      --image-type cos_containerd
    • This triggers a rolling node replacement; pods will be rescheduled and may be briefly disrupted if PodDisruptionBudgets/replicas are insufficient.

  5. Verify the node pools now use cos_containerd

    • Re-run the audit on each updated node pool:
    CLUSTER_NAME="<CLUSTER_NAME>"
    LOCATION="<LOCATION>"
    NODE_POOL_NAME="<NODE_POOL_NAME>"

    gcloud container node-pools describe "${NODE_POOL_NAME}" \
      --cluster "${CLUSTER_NAME}" \
      --location "${LOCATION}" \
      --project "${PROJECT_ID}" \
      --format json | jq '.config.imageType'
    • Confirm the output is "cos_containerd".

  6. Document exceptions where COS cannot be used

    • If you determine COS is not suitable for a specific node pool (due to workload or vendor constraints), record:
      • Cluster and node pool name.
      • Current image type and reason COS is not viable.
      • Compensating controls (e.g., hardened custom image, additional host security tooling).
    • Keep this documentation with your risk register and security exceptions for future review.
Using kubectl

kubectl cannot change the GKE node image or OS type, because this is managed at the GKE control-plane / node pool configuration level via gcloud, console, or IaC. To remediate this finding, follow the guidance in the Manual Steps section using the Google Cloud provider tools.

Automation
#!/usr/bin/env bash
# Report GKE node pool image types across all clusters in a project
# Requirements: gcloud, jq

PROJECT_ID="<GCP_PROJECT_ID>"   # <-- set this explicitly
REGION_FILTER=""                # optional: e.g. "--regions=us-central1,us-east1"
ZONE_FILTER=""                  # optional: e.g. "--zones=us-central1-a,us-central1-b"

set -euo pipefail

if [[ -z "${PROJECT_ID}" ]]; then
  echo "ERROR: Set PROJECT_ID in this script before running." >&2
  exit 1
fi

gcloud config set project "${PROJECT_ID}" >/dev/null

echo "Scanning GKE clusters and node pool image types in project: ${PROJECT_ID}"
echo

# List all clusters (both regional and zonal if no filters specified)
gcloud container clusters list \
  ${REGION_FILTER} ${ZONE_FILTER} \
  --format="table[box,title='GKE Clusters'](
      name,
      location,
      status
  )"

echo
echo "Node pool image types (non-cos_containerd are POTENTIAL PROBLEMS)"
echo

# JSON output for programmatic use
echo "=== JSON SUMMARY (for tooling) ==="
gcloud container clusters list \
  ${REGION_FILTER} ${ZONE_FILTER} \
  --format="json(name,location)" \
| jq -r '.[] | [.name,.location] | @tsv' \
| while IFS=$'\t' read -r CLUSTER LOCATION; do
    gcloud container node-pools list \
      --cluster "${CLUSTER}" \
      --location "${LOCATION}" \
      --format="json(name,config.imageType)" \
    | jq --arg cluster "${CLUSTER}" --arg loc "${LOCATION}" '
        .[] | {
          project: "'"${PROJECT_ID}"'",
          cluster: $cluster,
          location: $loc,
          nodePool: .name,
          imageType: .config.imageType,
          compliant: (.config.imageType == "cos_containerd")
        }'
  done | jq -s '.'   # array of all node pools

echo
echo "=== HUMAN-READABLE SUMMARY ==="
printf "%-30s %-20s %-30s %-20s %-12s\n" "CLUSTER" "LOCATION" "NODE_POOL" "IMAGE_TYPE" "COMPLIANT"
printf "%-30s %-20s %-30s %-20s %-12s\n" "-------" "--------" "---------" "----------" "---------"

gcloud container clusters list \
  ${REGION_FILTER} ${ZONE_FILTER} \
  --format="json(name,location)" \
| jq -r '.[] | [.name,.location] | @tsv' \
| while IFS=$'\t' read -r CLUSTER LOCATION; do
    gcloud container node-pools list \
      --cluster "${CLUSTER}" \
      --location "${LOCATION}" \
      --format="json(name,config.imageType)" \
    | jq -r --arg cluster "${CLUSTER}" --arg loc "${LOCATION}" '
        .[] | [
          $cluster,
          $loc,
          .name,
          .config.imageType,
          (if .config.imageType == "cos_containerd" then "YES" else "NO" end)
        ] | @tsv' \
    | while IFS=$'\t' read -r C L NP IMG COMPLIANT; do
        printf "%-30s %-20s %-30s %-20s %-12s\n" "$C" "$L" "$NP" "$IMG" "$COMPLIANT"
      done
  done

echo
echo "Interpretation:"
echo "- Node pools where IMAGE_TYPE != 'cos_containerd' (COMPLIANT = NO) are in scope for review."
echo "- Depending on workload requirements, you may choose to:"
echo "    * Migrate those node pools to cos_containerd, or"
echo "    * Formally accept the risk and document the exception."

Explanation of what indicates a problem:

  • In the “HUMAN-READABLE SUMMARY” table, any row where:

    • IMAGE_TYPE is not cos_containerd, and therefore
    • COMPLIANT is NO

    represents a node pool that does not meet CIS GKE 5.5.1 and should be reviewed.

  • In the JSON summary, any object with "compliant": false has an imageType other than cos_containerd and should be reviewed.

Additional Reading:​