Ensure Node Auto-Upgrade Is Enabled For Gke Nodes

More Info:

Node Auto-Upgrade Keeps Nodes At The Current Kubernetes And Os Security Patch Level To Mitigate Known Vulnerabilities

Risk Level

Medium

Address

Security

Compliance Standards

• CIS GKE

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Identify all node pools and their auto-upgrade status

   • Run on: any machine with gcloud access.

   gcloud container node-pools list \
     --cluster CLUSTER_NAME \
     --location LOCATION \
     --project PROJECT_ID \
     --format="table(name,config.imageType,management.autoUpgrade)"

   • Note any node pools where management.autoUpgrade is not True.

2. Get detailed management settings per node pool

   • For each node pool from step 1, run:

   gcloud container node-pools describe NODE_POOL_NAME \
     --cluster CLUSTER_NAME \
     --location LOCATION \
     --project PROJECT_ID \
     --format=json | jq '.name, .config.imageType, .management'

   • Confirm whether autoUpgrade is true or false and record it.

3. Review operational and change-management constraints

   • For node pools with autoUpgrade: false, discuss with application owners/SREs:
     • Whether they rely on manual control of node versions (e.g., for strict change windows, legacy workloads, or custom kernel features).
     • Whether there is an alternative patching process (documented, tested, and regularly executed).
   • Decide for each pool whether automatic upgrades are acceptable, and if not, document the business justification and the alternate upgrade process.

4. Enable node auto-upgrade where acceptable

   • For each node pool where you decide auto-upgrade should be enabled, run:

   gcloud container node-pools update NODE_POOL_NAME \
     --cluster CLUSTER_NAME \
     --location LOCATION \
     --project PROJECT_ID \
     --enable-autoupgrade

   • Be aware this changes how upgrades are scheduled; future automatic upgrades may cause node recreations during Google-managed windows.

5. If you must keep auto-upgrade disabled, harden manual processes

   • For node pools intentionally left with autoUpgrade: false, define and document:
     • A regular schedule to run:

       gcloud container clusters describe CLUSTER_NAME \
         --location LOCATION \
         --project PROJECT_ID \
         --format="value(currentMasterVersion, currentNodeVersion, releaseChannel.channel)"

       and plan manual node pool upgrades when new patch versions are available.
     • Change-control procedures and maintenance windows for manual upgrades:

       gcloud container node-pools upgrade NODE_POOL_NAME \
         --cluster CLUSTER_NAME \
         --location LOCATION \
         --project PROJECT_ID \
         --cluster-version LATEST_SAFE_VERSION

6. Verify the final state

   • Re-run the audit to confirm the configuration matches your decisions:

   gcloud container node-pools list \
     --cluster CLUSTER_NAME \
     --location LOCATION \
     --project PROJECT_ID \
     --format="table(name,management.autoUpgrade)"

   • Optionally, for each node pool:

   gcloud container node-pools describe NODE_POOL_NAME \
     --cluster CLUSTER_NAME \
     --location LOCATION \
     --project PROJECT_ID \
     --format=json | jq '.management'

   • Ensure auto-upgrade is enabled where required and disabled only where formally justified.

Using kubectl

kubectl cannot enable Node Auto-Upgrade because this setting is managed at the GKE control-plane / node pool configuration level via gcloud, console, or IaC, not via Kubernetes API objects. Refer to the Manual Steps section for the exact gcloud commands and configuration changes to enable auto-upgrade on the affected node pools.

Automation

#!/usr/bin/env bash
# Report GKE node-pool auto-upgrade status for all node pools in a project

# REQUIREMENTS:
# - gcloud CLI configured and authenticated
# - jq installed
# - You must know: PROJECT_ID and LOCATION (region or zone)

PROJECT_ID="my-gcp-project-id"
LOCATION="us-central1"   # e.g. us-central1 / us-central1-a

set -euo pipefail

echo "Project:  ${PROJECT_ID}"
echo "Location: ${LOCATION}"
echo

# List all clusters in the location
clusters_json="$(gcloud container clusters list \
  --project "${PROJECT_ID}" \
  --location "${LOCATION}" \
  --format json)"

if [[ "$(jq 'length' <<< "${clusters_json}")" -eq 0 ]]; then
  echo "No GKE clusters found in ${PROJECT_ID}/${LOCATION}"
  exit 0
fi

# Iterate clusters and their node pools
jq -r '.[].name' <<< "${clusters_json}" | while read -r CLUSTER_NAME; do
  echo "=== Cluster: ${CLUSTER_NAME} ==="

  node_pools_json="$(gcloud container node-pools list \
    --cluster "${CLUSTER_NAME}" \
    --location "${LOCATION}" \
    --project "${PROJECT_ID}" \
    --format json)"

  if [[ "$(jq 'length' <<< "${node_pools_json}")" -eq 0 ]]; then
    echo "  (no node pools found)"
    echo
    continue
  fi

  # For each node pool, show auto-upgrade status
  jq -r '.[].name' <<< "${node_pools_json}" | while read -r POOL_NAME; do
    mgmt_json="$(gcloud container node-pools describe "${POOL_NAME}" \
      --cluster "${CLUSTER_NAME}" \
      --location "${LOCATION}" \
      --project "${PROJECT_ID}" \
      --format json | jq '.management')"

    auto_upgrade="$(jq -r '.autoUpgrade // "null"' <<< "${mgmt_json}")"
    echo "  Node pool: ${POOL_NAME}"
    echo "    management: ${mgmt_json}" | sed 's/^/    /'

    # Flag non-compliant status
    if [[ "${auto_upgrade}" != "true" ]]; then
      echo "    >>> PROBLEM: autoUpgrade is NOT enabled for this node pool"
    fi

    echo
  done

  echo
done

Explanation of output indicating a problem:

• For each node pool, look at the management block, specifically autoUpgrade.
• Any node pool where:
  • "autoUpgrade": false, or
  • autoUpgrade is missing or null

is non-compliant with CIS GKE 5.5.3 and will be flagged in the script output as:

>>> PROBLEM: autoUpgrade is NOT enabled for this node pool

Additional Reading:

• https://cloud.google.com/kubernetes-engine/docs/concepts/node-auto-upgrades