Skip to main content

Containers Should Run As Non-Root

More Info:​

Verifies runAsNonRoot is set at pod or container level. Running as root inside a container widens the impact of a container escape.

Risk Level​

High

Address​

Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Manual Steps

  1. Identify non-compliant pods (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
      | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
      | .metadata as $m
      | (.spec.nodeName // "") as $node
      | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
      | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
      | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
      | ((.spec.containers // []) + (.spec.initContainers // []))[]
      | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
      | select($ok | not)
      | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
        + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
        + (if $node   == ""   then "" else " node=\($node)" end)
        + (if $labels == ""   then "" else " labels=\($labels)" end)
        + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
        + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
        + " is_compliant=false"
      ][]'

  2. For standalone Pods (not controlled by a higher-level object), export, edit, and re-apply the manifest (run on any machine with kubectl access). Example for one pod:

    kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod-POD_NAME.yaml

    Edit /tmp/pod-POD_NAME.yaml and add or adjust at the pod spec level:

    spec:
      securityContext:
        runAsNonRoot: true

    or, if you cannot set it at pod level, add it to every container and initContainer:

    spec:
      containers:
        - name: app
          image: your-image
          securityContext:
            runAsNonRoot: true
      initContainers:
        - name: init
          image: your-init-image
          securityContext:
            runAsNonRoot: true

    Then re-create the pod (Pods are immutable):

    kubectl -n NAMESPACE delete pod POD_NAME
    kubectl -n NAMESPACE apply -f /tmp/pod-POD_NAME.yaml

  3. For pods owned by a controller (Deployment/ReplicaSet/StatefulSet/DaemonSet/Job/CronJob), edit the controller so all future pods are compliant (run on any machine with kubectl access). Example for a Deployment:

    kubectl -n NAMESPACE edit deployment DEPLOYMENT_NAME

    In the editor, under spec.template.spec, set:

    spec:
      template:
        spec:
          securityContext:
            runAsNonRoot: true

    If needed, also set on each container:

    spec:
      template:
        spec:
          containers:
            - name: app
              image: your-image
              securityContext:
                runAsNonRoot: true

    Save and exit; Kubernetes will roll out updated pods.

  4. Repeat step 3 for other controllers (StatefulSet, DaemonSet, Job, CronJob) that own non-compliant pods, using kubectl edit on the appropriate resource type and setting spec.template.spec.securityContext.runAsNonRoot: true or per-container securityContext.runAsNonRoot: true.

  5. If any image fails to start with runAsNonRoot: true, review that image on your image build system: ensure the container’s default user is non-root (e.g., via a USER directive in the Dockerfile) or explicitly set a non-root runAsUser together with runAsNonRoot: true in the pod spec, then redeploy.

  6. Verify compliance (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
      | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
      | .metadata as $m
      | (.spec.nodeName // "") as $node
      | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
      | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
      | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
      | ((.spec.containers // []) + (.spec.initContainers // []))[]
      | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
      | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
        + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
        + " is_compliant=\(if $ok then "true" else "false" end)"
      ] as $rows
      | if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
        then "is_compliant=true"
        else $rows[]
        end'
Using kubectl

On any machine with kubectl access:

  1. Identify non-compliant Pods
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | select($ok|not)
  | "ns=\($m.namespace) name=\($m.name)"
  ] | unique[]'
  1. Edit each affected workload manifest and set pod-level runAsNonRoot: true (preferred). For example, for a Deployment owning the Pod:
kubectl -n NAMESPACE edit deployment DEPLOYMENT_NAME

Under spec.template.spec, ensure:

spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true

If you cannot set it at pod level (e.g., mixed containers), set it on each container instead:

spec:
  template:
    spec:
      containers:
        - name: app
          securityContext:
            runAsNonRoot: true
      initContainers:
        - name: init
          securityContext:
            runAsNonRoot: true

Apply the same pattern to other controllers (StatefulSet, DaemonSet, Job, CronJob) using:

kubectl -n NAMESPACE edit statefulset STATEFULSET_NAME
kubectl -n NAMESPACE edit daemonset DAEMONSET_NAME
kubectl -n NAMESPACE edit job JOB_NAME
kubectl -n NAMESPACE edit cronjob CRONJOB_NAME

For standalone Pods managed directly (not recommended in GKE production), edit and re-apply:

kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod.yaml

Edit /tmp/pod.yaml to include:

spec:
  securityContext:
    runAsNonRoot: true

Then delete and recreate:

kubectl -n NAMESPACE delete pod POD_NAME
kubectl -n NAMESPACE apply -f /tmp/pod.yaml
  1. Verification

Run the benchmark audit command again from any machine with kubectl access:

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'
Automation
#!/usr/bin/env bash
# Remediates: Ensure Pods/containers set securityContext.runAsNonRoot: true
# Scope: any machine with kubectl access to the cluster
# Requirements: kubectl, jq, yq (v4+)

set -euo pipefail

# ---- Config ----
# Namespaces to exclude (system namespaces managed by GKE)
EXCLUDED_NS_REGEX='^(kube-system|kube-public|kube-node-lease)$'

# ---- Functions ----

need_bin() {
  if ! command -v "$1" >/dev/null 2>&1; then
    echo "ERROR: '$1' is required but not found in PATH" >&2
    exit 1
  fi
}

# Return 0 if the resource kind is namespaced
is_namespaced_kind() {
  local kind="$1"
  case "$kind" in
    Pod|Deployment|ReplicaSet|StatefulSet|DaemonSet|Job|CronJob)
      return 0
      ;;
    *)
      return 1
      ;;
  esac
}

# Patch a single workload manifest to enforce runAsNonRoot
patch_manifest_run_as_non_root() {
  local file="$1"

  # 1) ensure pod-level securityContext.runAsNonRoot: true
  # 2) ensure every (init)container has securityContext.runAsNonRoot: true
  yq eval '
    .spec.template.spec |= (
      .securityContext //= {} |
      .securityContext.runAsNonRoot = true |
      (.containers // []) |=
        map(.securityContext //= {} | .securityContext.runAsNonRoot = true) |
      (.initContainers // []) |=
        map(.securityContext //= {} | .securityContext.runAsNonRoot = true)
    )
  ' "$file"
}

# For plain Pod manifests (no template)
patch_pod_manifest_run_as_non_root() {
  local file="$1"

  yq eval '
    .spec |= (
      .securityContext //= {} |
      .securityContext.runAsNonRoot = true |
      (.containers // []) |=
        map(.securityContext //= {} | .securityContext.runAsNonRoot = true) |
      (.initContainers // []) |=
        map(.securityContext //= {} | .securityContext.runAsNonRoot = true)
    )
  ' "$file"
}

# ---- Pre-flight ----
need_bin kubectl
need_bin jq
need_bin yq

# ---- Discover non-compliant Pods ----
echo "Discovering non-compliant Pods (excluding kube-system, kube-public, kube-node-lease)..."

non_compliant_json="$(kubectl get pods --all-namespaces -o json | jq --arg re "$EXCLUDED_NS_REGEX" '
  .items[]
  | select(.metadata.namespace | test($re) | not)
  | . as $pod
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | select($ok | not)
  | {
      pod_namespace: $pod.metadata.namespace,
      pod_name: $pod.metadata.name,
      container_name: .name,
      owner: ([$pod.metadata.ownerReferences[]? | select(.controller)] | first // null)
    }
' 2>/dev/null || true)"

if [[ -z "$non_compliant_json" ]]; then
  echo "No Pods found."
  exit 0
fi

non_compliant_count="$(echo "$non_compliant_json" | jq -s 'length')"
if [[ "$non_compliant_count" -eq 0 ]]; then
  echo "All Pods are compliant."
else
  echo "Found $non_compliant_count non-compliant container instances."
fi

# ---- Group by owner to identify manageable workloads ----
# We will patch only supported, namespaced controllers with server-side apply.
echo "Identifying parent workloads to patch..."

# Build list of unique (kind, namespace, name) for owners
owners_json="$(
  echo "$non_compliant_json" | jq -s '
    map(select(.owner != null))
    | map({
        kind: .owner.kind,
        namespace: .pod_namespace,
        name: .owner.name
      })
    | unique
  '
)"

# Also collect standalone Pods (no controller) to patch or flag for manual fix
standalone_pods_json="$(
  echo "$non_compliant_json" | jq -s '
    map(select(.owner == null))
    | map({
        kind: "Pod",
        namespace: .pod_namespace,
        name: .pod_name
      })
    | unique
  '
)"

echo "Workloads with non-compliant containers:"
echo "$owners_json" | jq -r '.[] | "\(.kind) \(.namespace)/\(.name)"' 2>/dev/null || true

echo "Standalone Pods with non-compliant containers (may not be safe to patch automatically):"
echo "$standalone_pods_json" | jq -r '.[] | "\(.kind) \(.namespace)/\(.name)"' 2>/dev/null || true

# ---- Patch controllers (Deployments, etc.) via kubectl ----
echo "Patching parent workloads via kubectl..."

echo "$owners_json" | jq -c '.[]' | while read -r owner; do
  kind="$(echo "$owner" | jq -r '.kind')"
  ns="$(echo "$owner" | jq -r '.namespace')"
  name="$(echo "$owner" | jq -r '.name')"

  if ! is_namespaced_kind "$kind"; then
    echo "Skipping unsupported owner kind: $kind $ns/$name"
    continue
  fi

  echo "Processing $kind $ns/$name ..."

  tmpfile="$(mktemp)"
  trap 'rm -f "$tmpfile"' EXIT

  # Export manifest
  if ! kubectl get "$kind" "$name" -n "$ns" -o yaml > "$tmpfile"; then
    echo "  WARN: could not get $kind $ns/$name, skipping."
    rm -f "$tmpfile"
    trap - EXIT
    continue
  fi

  # Patch manifest in-place, depending on kind
  case "$kind" in
    Pod)
      patched="$(patch_pod_manifest_run_as_non_root "$tmpfile")"
      ;;
    *)
      patched="$(patch_manifest_run_as_non_root "$tmpfile")"
      ;;
  esac

  echo "$patched" > "$tmpfile"

  # Apply with server-side apply (idempotent)
  if ! kubectl apply -f "$tmpfile" --server-side --force-conflicts >/dev/null; then
    echo "  WARN: failed to apply patched manifest for $kind $ns/$name"
  else
    echo "  Patched $kind $ns/$name"
  fi

  rm -f "$tmpfile"
  trap - EXIT
done

# ---- Standalone Pods ----
# These are often ephemeral or manually managed. We only print guidance.
standalone_count="$(echo "$standalone_pods_json" | jq -s 'length' 2>/dev/null || echo 0)"
if [[ "$standalone_count" -gt 0 ]]; then
  echo
  echo "NOTE: Standalone Pods listed above are still non-compliant."
  echo "To fix them, update their Pod specs or the process that creates them to set:"
  echo "  spec.securityContext.runAsNonRoot: true"
  echo "  spec.containers[].securityContext.runAsNonRoot: true"
fi

# ---- Verification ----
echo
echo "Re-running compliance audit to verify remediation..."

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
    then "is_compliant=true"
    else ($rows[] | select(. | contains("is_compliant=false")))
    end
'