Enable Customer-Managed Encryption Keys (CMEK) For Boot

More Info:

Encrypt GKE node boot disks with Customer-Managed Encryption Keys so the key lifecycle for node disks is under your control. CMEK provides stronger governance than default encryption.

Risk Level

Medium

Address

Security

Compliance Standards

• CIS GKE

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Identify node pools and check current boot disk CMEK use

   • On any machine with gcloud configured for the project:

     PROJECT_ID="your-project-id"
     LOCATION="your-cluster-location"   # e.g. us-central1, europe-west1-b
     CLUSTER_NAME="your-cluster-name"
     
     gcloud container node-pools list \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}" \
       --format="table(name)"

     For each node pool name returned:

     NODE_POOL="name-from-above"
     
     gcloud container node-pools describe "${NODE_POOL}" \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}" \
       --format="table(name, config.diskType, config.bootDiskKmsKey)"

     • If config.bootDiskKmsKey is empty, that node pool’s boot disks are not using CMEK.

2. Review CMEK policy/requirements with security/compliance

   • Confirm which clusters/node pools must use CMEK, what key project/location/ring/key must be used, and whether any node pools are exempt (e.g. non‑prod, short‑lived test).
   • Decide whether to migrate all non‑CMEK node pools now or in phases, based on risk and change windows.

3. Verify Cloud KMS key readiness and permissions

   • On any machine with gcloud access:

     KEY_PROJECT_ID="kms-project-id"
     KEY_LOCATION="kms-location"           # e.g. us-central1
     KEY_RING="kms-ring-name"
     KEY_NAME="kms-key-name"
     
     gcloud kms keys describe "${KEY_NAME}" \
       --location "${KEY_LOCATION}" \
       --keyring "${KEY_RING}" \
       --project "${KEY_PROJECT_ID}"

   • Ensure the key is ENABLED, in the correct location/region for your cluster, and that the GKE node service accounts (or the project’s Compute Engine default service account) have roles/cloudkms.cryptoKeyEncrypterDecrypter on this key.

4. Plan node pool replacement/migration strategy

   • Determine disk type (pd-standard or pd-ssd) and size currently in use for each non‑CMEK node pool:

     gcloud container node-pools describe "${NODE_POOL}" \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}" \
       --format="yaml(config.diskType, config.diskSizeGb, config.machineType, maxPodsConstraint)"

   • Decide per node pool whether you will:
     • Create a new CMEK‑enabled node pool and cordon/drain/migrate workloads, then delete the old pool (recommended), or
     • Recreate the entire cluster with CMEK from the start if many pools/configs must change.

5. Create replacement CMEK‑enabled node pools and migrate workloads

   • For each node pool that must be CMEK‑encrypted, on any machine with gcloud access:

     NEW_NODE_POOL="cmeq-${NODE_POOL}"
     
     gcloud container node-pools create "${NEW_NODE_POOL}" \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}" \
       --disk-type "pd-ssd" \
       --boot-disk-kms-key "projects/${KEY_PROJECT_ID}/locations/${KEY_LOCATION}/keyRings/${KEY_RING}/cryptoKeys/${KEY_NAME}" \
       --machine-type "same-machine-type-as-old" \
       --num-nodes 3

   • Cordon and drain old nodes, then delete the old node pool when workloads are stable on the new one:

     # list nodes in old pool
     kubectl get nodes -l cloud.google.com/gke-nodepool="${NODE_POOL}"
     
     # cordon and drain each node (run on any machine with kubectl access)
     kubectl cordon <node-name>
     kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data
     
     # delete old pool (gcloud)
     gcloud container node-pools delete "${NODE_POOL}" \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}"

6. Verify remediation and document exceptions

   • Re-run the audit for the cluster:

     gcloud container node-pools list \
       --cluster "${CLUSTER_NAME}" \
       --location "${LOCATION}" \
       --project "${PROJECT_ID}" \
       --format="value(name)" | while read NP; do
       gcloud container node-pools describe "${NP}" \
         --cluster "${CLUSTER_NAME}" \
         --location "${LOCATION}" \
         --project "${PROJECT_ID}" \
         --format="table(name, config.diskType, config.bootDiskKmsKey)"
     done

   • Confirm all required node pools now show the correct non‑empty config.bootDiskKmsKey.
   • For any node pool intentionally left without CMEK, record the justification and approval per your governance process.

Using kubectl

kubectl cannot configure Customer-Managed Encryption Keys (CMEK) for GKE node boot disks, because this is a Google Cloud / cluster provisioning setting, not a Kubernetes API object. To remediate, use the Google Cloud Console, gcloud CLI, or your IaC as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
#
# Report GKE node pool boot disk CMEK usage for all clusters in a project.
# REQUIREMENTS:
#   - gcloud and jq installed
#   - Authenticated to the desired project
#
# USAGE:
#   PROJECT_ID="my-project" ./report_gke_bootdisk_cmek.sh

set -euo pipefail

PROJECT_ID="${PROJECT_ID:-}"
if [[ -z "${PROJECT_ID}" ]]; then
  echo "ERROR: PROJECT_ID environment variable must be set" >&2
  exit 1
fi

gcloud config set project "${PROJECT_ID}" >/dev/null

# Get all clusters (zonal and regional)
echo "Collecting cluster list for project: ${PROJECT_ID}..." >&2
clusters_json="$(gcloud container clusters list --format=json)"

if [[ "${clusters_json}" == "[]" ]]; then
  echo "No GKE clusters found in project: ${PROJECT_ID}"
  exit 0
fi

# Header
printf "%-60s %-40s %-15s %-12s %-10s %-80s\n" \
  "CLUSTER" "NODE_POOL" "LOCATION" "DISK_TYPE" "USES_CMEK" "BOOT_DISK_KMS_KEY"

echo "${clusters_json}" | jq -r '.[] | @base64' | while read -r cluster_b64; do
  _c() { echo "${cluster_b64}" | base64 --decode | jq -r "${1}"; }

  cluster_name=$(_c '.name')
  cluster_location=$(_c '.location')

  # List node pools for this cluster
  nodepools_json="$(gcloud container node-pools list \
    --cluster "${cluster_name}" \
    --location "${cluster_location}" \
    --format=json || echo "[]")"

  if [[ "${nodepools_json}" == "[]" ]]; then
    printf "%-60s %-40s %-15s %-12s %-10s %-80s\n" \
      "${cluster_name}" "<no-node-pools>" "${cluster_location}" "-" "-" "-"
    continue
  fi

  echo "${nodepools_json}" | jq -r '.[] | @base64' | while read -r np_b64; do
    _n() { echo "${np_b64}" | base64 --decode | jq -r "${1}"; }

    np_name=$(_n '.name')
    disk_type=$(_n '.config.diskType // "-"')
    boot_disk_kms_key=$(_n '.config.bootDiskKmsKey // ""')

    if [[ -n "${boot_disk_kms_key}" && "${boot_disk_kms_key}" != "null" ]]; then
      uses_cmek="true"
    else
      uses_cmek="false"
      boot_disk_kms_key="-"
    fi

    printf "%-60s %-40s %-15s %-12s %-10s %-80s\n" \
      "${cluster_name}" "${np_name}" "${cluster_location}" \
      "${disk_type}" "${uses_cmek}" "${boot_disk_kms_key}"
  done
done

How to interpret the output

• Each row shows: CLUSTER, NODE_POOL, LOCATION, DISK_TYPE, USES_CMEK, BOOT_DISK_KMS_KEY.
• A problem for this control is any node pool where:
  • USES_CMEK is false, and
  • DISK_TYPE is pd-standard or pd-ssd (i.e., the node boot disks are not using a customer-managed key).
• For compliant node pools:
  • USES_CMEK is true, and
  • BOOT_DISK_KMS_KEY contains a full KMS key resource: projects/<key_project_id>/locations/<location>/keyRings/<ring_name>/cryptoKeys/<key_name>.