Legacy Compute Engine Instance Metadata Apis Are Disabled

More Info:

Disable The Legacy Gce Instance Metadata Apis For Gke Nodes. Under Some Circumstances, These Can Be Used From Within A Pod To Extract The NodeS Credentials

Risk Level

High

Address

Security

Compliance Standards

• CIS GKE

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Identify node pools and their locations
   Run on: any machine with gcloud and access to the project.

   gcloud container clusters list --project PROJECT_ID
   gcloud container node-pools list \
     --cluster CLUSTER_NAME \
     --zone COMPUTE_ZONE \
     --project PROJECT_ID

2. Check each node pool’s metadata configuration
   For each node pool from step 1, inspect its metadata.

   gcloud container node-pools describe POOL_NAME \
     --cluster CLUSTER_NAME \
     --zone COMPUTE_ZONE \
     --project PROJECT_ID \
     --format="json(config.metadata)"

   Review the output for disable-legacy-endpoints.

   • Compliant: "disable-legacy-endpoints": "true"
   • Non-compliant or default: key missing or value not "true".

3. Decide which node pools must be remediated
   For each non-compliant node pool, determine:

   • Whether it still hosts any critical or stateful workloads.
   • Whether it is managed by IaC (Terraform/Deployment Manager) and needs changes there first, to avoid drift.

4. Create a compliant replacement node pool
   For each non-compliant node pool, create a new one with legacy metadata disabled (match machine type, labels, taints, node count, etc. as needed).

   gcloud container node-pools create NEW_POOL_NAME \
     --cluster CLUSTER_NAME \
     --zone COMPUTE_ZONE \
     --project PROJECT_ID \
     --metadata disable-legacy-endpoints=true \
     --num-nodes 3  \
     --machine-type e2-standard-4

   Adjust --num-nodes, --machine-type, and any other options to mirror the old pool.

5. Migrate workloads from old to new node pools
   Run on: any machine with kubectl access.

   • Confirm nodes in each pool:

     kubectl get nodes --show-labels

   • For each workload, ensure its nodeSelector, topologySpreadConstraints, and affinity allow scheduling on the new pool (update manifests/Helm charts if needed).
   • Cordon and drain old pool nodes one by one:

     kubectl cordon NODE_NAME
     kubectl drain NODE_NAME --ignore-daemonsets --delete-emptydir-data

6. Delete non-compliant node pools and re-verify
   After confirming workloads are stable on the new pool(s), delete each non-compliant pool:

   gcloud container node-pools delete OLD_POOL_NAME \
     --cluster CLUSTER_NAME \
     --zone COMPUTE_ZONE \
     --project PROJECT_ID

   Re-run the verification:

   gcloud container node-pools describe POOL_NAME \
     --cluster CLUSTER_NAME \
     --zone COMPUTE_ZONE \
     --project PROJECT_ID \
     --format="json(config.metadata)"

   Confirm all remaining node pools have "disable-legacy-endpoints": "true".

Using kubectl

kubectl cannot modify GKE node pool metadata or disable the legacy GCE instance metadata APIs, because this setting is only configurable at the cloud provider/node pool level. To remediate this finding, make the changes via the GCP console, gcloud, or your IaC as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
#
# Check all GKE node pools in all GCP projects for legacy GCE metadata endpoint usage.
# Requirements:
#   - gcloud installed and authenticated
#   - jq installed
#
# Run from: any machine with gcloud access

set -euo pipefail

echo "Listing projects..."
PROJECTS=$(gcloud projects list --format="value(projectId)")

if [[ -z "${PROJECTS}" ]]; then
  echo "No projects found via gcloud."
  exit 0
fi

for PROJECT in ${PROJECTS}; do
  echo
  echo "=== Project: ${PROJECT} ==="
  CLUSTERS=$(gcloud container clusters list \
    --project "${PROJECT}" \
    --format="value(name,location)")

  if [[ -z "${CLUSTERS}" ]]; then
    echo "  No GKE clusters."
    continue
  fi

  while read -r CLUSTER LOC; do
    [[ -z "${CLUSTER}" ]] && continue
    echo
    echo "  Cluster: ${CLUSTER} (location: ${LOC})"

    # List node pools with metadata settings
    POOLS_JSON=$(gcloud container node-pools list \
      --cluster "${CLUSTER}" \
      --location "${LOC}" \
      --project "${PROJECT}" \
      --format=json)

    if [[ "${POOLS_JSON}" == "[]" ]]; then
      echo "    No node pools."
      continue
    fi

    echo "${POOLS_JSON}" | jq -r '
      .[] | {
        name: .name,
        metadata: (.config.metadata // {}),
        workloadMetadataConfig: (.config.workloadMetadataConfig // {})
      } | 
      "    Node pool: \(.name)\n" +
      "      metadata:\n" +
      (if (.metadata | length) == 0
       then "        (none)\n"
       else ( .metadata | to_entries[]
              | "        \(.key) = \(.value)\n" )
       end) +
      "      workloadMetadataConfig:\n" +
      (if (.workloadMetadataConfig | length) == 0
       then "        (none)\n"
       else ( .workloadMetadataConfig | to_entries[]
              | "        \(.key) = \(.value)\n" )
       end)
    '

  done <<< "${CLUSTERS}"
done

cat <<'EOF'

Interpretation:

1) Primary indicator of a problem (per CIS GKE 5.4.1):
   - Any node pool where metadata.disable-legacy-endpoints is NOT explicitly set
     to "true" is POTENTIALLY NON-COMPLIANT.

   You should look in the "metadata:" section for each "Node pool:" block:

   - Compliant example:
       metadata:
         disable-legacy-endpoints = true

   - Problematic examples:
       metadata:
         disable-legacy-endpoints = false
     or
       metadata:
         (none)
     or
       metadata:
         some-other-key = value
         # but no disable-legacy-endpoints key present

2) Workload metadata mode:
   - This script also prints workloadMetadataConfig (e.g. GKE_METADATA, GCE_METADATA).
     That setting affects how metadata is exposed to pods, but CIS 5.4.1 is
     specifically about legacy GCE instance metadata APIs, which are controlled
     by the disable-legacy-endpoints metadata key above.

Action for any problematic node pool:
   - Plan to create a new node pool for the affected cluster with:
       --metadata disable-legacy-endpoints=true
   - Migrate workloads from the old node pool to the new one.
   - Delete the non-conforming node pool after migration.

This cannot be fully automated safely because:
   - Moving workloads between node pools and deleting pools involves
     application-specific availability and disruption considerations that
     require human review and change control.
EOF

Additional Reading:

• https://cloud.google.com/kubernetes-engine/docs/how-to/protecting-cluster-metadata#disable-legacy-apis