Consider Gke Sandbox For Running Untrusted Workloads

More Info:

Use Gke Sandbox To Restrict Untrusted Workloads As An Additional Layer Of Protection When Running In A Multi-Tenant Environment.

Risk Level

Medium

Address

Security

Compliance Standards

APRA CPS 234 (Australia)
BSI C5 (Germany)
Brazil LGPD
CCPA / CPRA (California)
CIS Critical Security Controls v8
CIS GKE
CMMC 2.0
CSA Cloud Controls Matrix v4
DPDPA
Digital Operational Resilience Act (EU)
Essential 8
ISO/IEC 27017
ISO/IEC 27018
ISO/IEC 27701
KSA PDPL
MAS Technology Risk Management (Singapore)
MITRE ATT&CK (Cloud)
NIS2 Directive
NIST SP 800-171
NYDFS 23 NYCRR 500
SWIFT Customer Security Controls Framework
Sarbanes-Oxley IT General Controls
UK NCSC Cyber Assessment Framework

Triage and Remediation

Remediation

Remediation

Manual Steps

Identify existing node pools and their sandbox configuration

Run on: any machine with gcloud access

Command:

gcloud container node-pools list \
  --cluster CLUSTER_NAME \
  --location LOCATION \
  --project PROJECT_ID

For each node pool name from the output:

gcloud container node-pools describe NODE_POOL_NAME \
  --cluster CLUSTER_NAME \
  --location LOCATION \
  --project PROJECT_ID \
  --format json | jq '.config.sandboxConfig'

Review: Note which node pools show {"type": "gvisor"} and which show null or no sandbox.

Determine which workloads are untrusted or multi-tenant
- Run on: any machine with kubectl access
- Commands to list namespaces and deployments (for your own review; no change yet):
```
kubectl get namespaces
kubectl get deployments --all-namespaces -o wide
```
- Review: Identify namespaces/workloads that:
  - Are operated by external tenants/teams, or
  - Run third-party or less-trusted code, or
  - Process unvalidated/untrusted input at scale.
    These are candidates to be scheduled onto sandboxed (gVisor) nodes.
Decide the sandboxing strategy and scheduling model
- Review and decide:
  - Will all workloads be sandboxed, or only specific namespaces/apps?
  - Resource overhead: gVisor imposes performance cost; ensure capacity is acceptable.
  - Scheduling mechanism:
    - Option A: Taint gVisor node pools and use tolerations on untrusted workloads.
    - Option B: Use nodeSelector or topology spread constraints to pin untrusted workloads to gVisor nodes.
  - Pod Security / Admission policies: ensure they won’t block running on COS with containerd and gVisor.

Create a new gVisor-enabled node pool for untrusted workloads

Run on: any machine with gcloud access

Command (adapt names and sizes to your decision):

gcloud container node-pools create gvisor-untrusted-pool \
  --cluster CLUSTER_NAME \
  --location LOCATION \
  --project PROJECT_ID \
  --image-type=cos_containerd \
  --sandbox="type=gvisor" \
  --num-nodes=3

Optional (if you plan to use taints for isolation):

gcloud container node-pools update gvisor-untrusted-pool \
  --cluster CLUSTER_NAME \
  --location LOCATION \
  --project PROJECT_ID \
  --node-taints=role=gvisor-untrusted:NoSchedule

Migrate untrusted workloads to the sandboxed node pool

Run on: any machine with kubectl access

For workloads you identified as untrusted:

If using taints/tolerations, patch a deployment example:

kubectl -n NAMESPACE patch deployment DEPLOYMENT_NAME \
  --type merge \
  -p '{
    "spec": {
      "template": {
        "spec": {
          "tolerations": [
            {
              "key": "role",
              "operator": "Equal",
              "value": "gvisor-untrusted",
              "effect": "NoSchedule"
            }
          ]
        }
      }
    }
  }'

If using nodeSelector instead:

Label gVisor nodes (once they are ready):

kubectl get nodes -l cloud.google.com/gke-nodepool=gvisor-untrusted-pool
kubectl label node NODE_NAME sandbox=gvisor-untrusted

Patch deployment to select those nodes:

kubectl -n NAMESPACE patch deployment DEPLOYMENT_NAME \
  --type merge \
  -p '{
    "spec": {
      "template": {
        "spec": {
          "nodeSelector": {
            "sandbox": "gvisor-untrusted"
          }
        }
      }
    }
  }'

Review: Ensure pods from untrusted workloads are now scheduled only onto the gVisor node pool.

Verify sandbox configuration and workload placement
- Run on: any machine with gcloud and kubectl access
- Verify node pool sandbox:
```
gcloud container node-pools describe gvisor-untrusted-pool \
  --cluster CLUSTER_NAME \
  --location LOCATION \
  --project PROJECT_ID \
  --format json | jq '.config.sandboxConfig'
```
  Confirm it outputs {"type":"gvisor"} (or equivalent).
- Verify workloads run only on sandbox nodes:
```
kubectl get pods -n NAMESPACE -o wide
```
  Confirm the NODE column for untrusted workloads matches nodes in the gvisor-untrusted-pool and not any non-sandbox node pools.

Using kubectl

kubectl cannot be used to enable or configure GKE Sandbox because this setting is applied at the node pool / cluster level through Google Cloud (gcloud CLI, console, or IaC), not via Kubernetes API objects. Refer to the Manual Steps section for guidance on enabling GKE Sandbox using the appropriate cloud provider configuration tools.

Automation

#!/usr/bin/env bash
# Report GKE Sandbox (gVisor) usage across all node pools in a project.
# Requires: gcloud, jq

set -euo pipefail

PROJECT_ID="$1"

if [[ -z "${PROJECT_ID}" ]]; then
  echo "Usage: $0 <PROJECT_ID>" >&2
  exit 1
fi

gcloud config set project "${PROJECT_ID}" >/dev/null

# List all clusters (both zonal and regional)
clusters_json="$(gcloud container clusters list --format=json)"

if [[ "${clusters_json}" == "[]" ]]; then
  echo "No GKE clusters found in project ${PROJECT_ID}"
  exit 0
fi

echo "PROJECT,CLUSTER,LOCATION,MODE,NODE_POOL,IMAGE_TYPE,SANDBOX_TYPE,ISSUE" 

jq -c '.[]' <<< "${clusters_json}" | while read -r cluster; do
  CLUSTER_NAME="$(jq -r '.name' <<< "${cluster}")"
  LOCATION="$(jq -r '.location' <<< "${cluster}")"
  MODE="$(jq -r '.locationType // "unknown"' <<< "${cluster}")"

  # List node pools for this cluster
  node_pools_json="$(gcloud container node-pools list \
    --cluster "${CLUSTER_NAME}" \
    --location "${LOCATION}" \
    --format=json || echo "[]")"

  if [[ "${node_pools_json}" == "[]" ]]; then
    echo "${PROJECT_ID},${CLUSTER_NAME},${LOCATION},${MODE},<none>,<none>,<none>,NO_NODE_POOLS"
    continue
  fi

  jq -c '.[]' <<< "${node_pools_json}" | while read -r np; do
    NP_NAME="$(jq -r '.name' <<< "${np}")"

    # Describe node pool to inspect sandbox and image type
    np_desc="$(gcloud container node-pools describe "${NP_NAME}" \
      --cluster "${CLUSTER_NAME}" \
      --location "${LOCATION}" \
      --format=json)"

    IMAGE_TYPE="$(jq -r '.config.imageType // "unknown"' <<< "${np_desc}")"
    SANDBOX_TYPE="$(jq -r '.config.sandboxConfig.type // "none"' <<< "${np_desc}")"

    ISSUE=""

    # Per CIS guidance, gVisor requires cos_containerd and sandbox type=gvisor
    if [[ "${SANDBOX_TYPE}" == "none" || "${SANDBOX_TYPE}" == "null" ]]; then
      ISSUE="NO_SANDBOX_CONFIGURED"
    elif [[ "${SANDBOX_TYPE}" != "gvisor" ]]; then
      ISSUE="NON_GVISOR_SANDBOX(${SANDBOX_TYPE})"
    fi

    if [[ "${SANDBOX_TYPE}" == "gvisor" && "${IMAGE_TYPE}" != "COS_CONTAINERD" && "${IMAGE_TYPE}" != "cos_containerd" ]]; then
      # Misaligned configuration; gVisor requires cos_containerd nodes
      if [[ -n "${ISSUE}" ]]; then
        ISSUE="${ISSUE}|GVISOR_WITH_UNSUPPORTED_IMAGE(${IMAGE_TYPE})"
      else
        ISSUE="GVISOR_WITH_UNSUPPORTED_IMAGE(${IMAGE_TYPE})"
      fi
    fi

    # If there is no issue, mark as OK for easier filtering
    if [[ -z "${ISSUE}" ]]; then
      ISSUE="OK"
    fi

    echo "${PROJECT_ID},${CLUSTER_NAME},${LOCATION},${MODE},${NP_NAME},${IMAGE_TYPE},${SANDBOX_TYPE},${ISSUE}"
  done
done

How to run (from any machine with gcloud configured):

chmod +x report-gke-sandbox.sh
./report-gke-sandbox.sh YOUR_PROJECT_ID > gke-sandbox-report.csv

How to interpret output (problem indicators):

Look at the ISSUE column:

NO_SANDBOX_CONFIGURED
- Node pool does not use GKE Sandbox (gVisor).
- In a multi-tenant or untrusted workload scenario, this is what you review and likely treat as non-compliant.
NON_GVISOR_SANDBOX(<type>)
- Some other sandbox type is configured instead of gvisor; verify if it meets your policy (typically a finding for this CIS control).
GVISOR_WITH_UNSUPPORTED_IMAGE(<IMAGE_TYPE>)
- gVisor is set but the image type is not cos_containerd, which is required per the benchmark remediation. Treat as misconfiguration.
OK
- Node pool uses sandboxConfig.type=gvisor and imageType=cos_containerd and aligns with the remediation.

Use the report to decide which clusters/node pools running untrusted or multi-tenant workloads should be migrated onto new node pools created with:

gcloud container node-pools create <node_pool_name> \
  --location <location> \
  --cluster <cluster_name> \
  --image-type=cos_containerd \
  --sandbox="type=gvisor"

Additional Reading:

https://cloud.google.com/kubernetes-engine/docs/how-to/sandbox-pods

More Info:​

Risk Level​

Address​

Compliance Standards​

Triage and Remediation​

Remediation​

Additional Reading:​