Containers Should Run As Non-Root

More Info:

Verifies runAsNonRoot is set at pod or container level. Running as root inside a container widens the impact of a container escape.

Risk Level

High

Address

Security

Compliance Standards

• Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

1. On any machine with kubectl access, list the non‑compliant pods and pick the one(s) to fix:

   kubectl get pods --all-namespaces -o json | jq -r '
     [ .items[]
     | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
     | .metadata as $m
     | (.spec.nodeName // "") as $node
     | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
     | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
     | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
     | ((.spec.containers // []) + (.spec.initContainers // []))[]
     | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
     | select($ok | not)
     | "ns=\($m.namespace) name=\($m.name) container=\(.name) image=\(.image)"
   ][]'

2. For a pod created from a higher‑level controller (Deployment, DaemonSet, StatefulSet, Job, CronJob), edit the controller manifest so the setting persists. For example, for a Deployment (run on any kubectl machine):

   kubectl -n NAMESPACE edit deployment DEPLOYMENT_NAME

   Under spec.template.spec, add or update a pod‑level security context:

   spec:
     template:
       spec:
         securityContext:
           runAsNonRoot: true

   If you need per‑container overrides instead (e.g., only some containers must be non‑root), set on each container:

   spec:
     template:
       spec:
         containers:
           - name: CONTAINER_NAME
             securityContext:
               runAsNonRoot: true

   Save and exit; Kubernetes will roll out new pods.

3. For standalone Pods not managed by a controller, first export the manifest (any kubectl machine):

   kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod-POD_NAME.yaml

   In /tmp/pod-POD_NAME.yaml, delete runtime‑only fields like status:, metadata.resourceVersion, metadata.uid, metadata.creationTimestamp, and metadata.managedFields. Under spec, add either:

   spec:
     securityContext:
       runAsNonRoot: true

   or, per container:

   spec:
     containers:
       - name: CONTAINER_NAME
         securityContext:
           runAsNonRoot: true

4. Apply the corrected standalone Pod manifest and recreate the pod (any kubectl machine):

   kubectl -n NAMESPACE delete pod POD_NAME
   kubectl apply -f /tmp/pod-POD_NAME.yaml

5. For OKE workloads managed via IaC or GitOps, mirror the same securityContext.runAsNonRoot: true changes in the source manifests or Helm charts (e.g., in your Git repository) so future deployments do not revert the setting. Commit and redeploy via your normal pipeline.

6. Verify all pods now comply (any kubectl machine):

   kubectl get pods --all-namespaces -o json | jq -r '
     [ .items[]
     | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
     | .metadata as $m
     | (.spec.nodeName // "") as $node
     | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
     | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
     | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
     | ((.spec.containers // []) + (.spec.initContainers // []))[]
     | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
     | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
       + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
       + (if $node   == ""   then "" else " node=\($node)" end)
       + (if $labels == ""   then "" else " labels=\($labels)" end)
       + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
       + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
       + " is_compliant=\(if $ok then "true" else "false" end)"
     ] as $rows
     | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

   Confirm that either the output is exactly is_compliant=true or every listed container line ends with is_compliant=true.

Using kubectl

On any machine with kubectl access:

1. Identify non-compliant pods

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | select($ok | not)
  | "ns=\($m.namespace) name=\($m.name)"
  ] | unique[]'

2. For a pod controlled by a Deployment/ReplicaSet/DaemonSet/Job/etc., patch the controller (example: Deployment)

kubectl -n <namespace> get deployment <name> -o yaml > /tmp/deployment-<name>.yaml

Edit /tmp/deployment-<name>.yaml and under spec.template.spec add (or adjust) the pod-level security context:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <name>
  namespace: <namespace>
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
      containers:
        - name: <container-name>
          image: <image>
          # container-specific securityContext is optional if pod-level is set
          # securityContext:
          #   runAsNonRoot: true

Apply the updated manifest:

kubectl apply -f /tmp/deployment-<name>.yaml

If you prefer to set it per container instead of pod-level:

spec:
  template:
    spec:
      containers:
        - name: <container-name>
          image: <image>
          securityContext:
            runAsNonRoot: true

3. For standalone Pods (no ownerReferences.controller), edit and re-apply

kubectl -n <namespace> get pod <pod-name> -o yaml > /tmp/pod-<pod-name>.yaml

Edit /tmp/pod-<pod-name>.yaml so the pod spec has one of:

Pod-level:

spec:
  securityContext:
    runAsNonRoot: true
  containers:
    - name: <container-name>
      image: <image>

Or per container:

spec:
  containers:
    - name: <container-name>
      image: <image>
      securityContext:
        runAsNonRoot: true

Then recreate the pod:

kubectl delete -n <namespace> pod <pod-name>
kubectl apply -f /tmp/pod-<pod-name>.yaml

4. Verification (same audit logic as check)

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) container=\(.name) runAsNonRoot=\($ok)"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
    then "is_compliant=true"
    else $rows[]
    end'

Automation

#!/usr/bin/env bash
#
# Enforce securityContext.runAsNonRoot: true on all non-excluded Pods
# by updating their owning workload manifests via kubectl.
#
# Runs from: any machine with kubectl access to the OKE cluster.
# Requirements: bash, kubectl, jq, yq (https://github.com/mikefarah/yq)
#
# Notes:
# - This is idempotent: re-running only re-applies the same settings.
# - It patches the OWNER workload (Deployment, StatefulSet, DaemonSet,
#   Job, CronJob, ReplicaSet) rather than the live Pod.
# - Pods in kube-system, kube-public, kube-node-lease are skipped,
#   matching the audit command.
# - You MUST review and test in non‑prod first; some images may REQUIRE root.
#

set -euo pipefail

# Fail fast if required tools not present
for bin in kubectl jq yq; do
  if ! command -v "$bin" >/dev/null 2>&1; then
    echo "ERROR: $bin not found in PATH. Please install it before running this script." >&2
    exit 1
  fi
done

# Get non-compliant pod rows from the audit query
echo "Discovering non-compliant containers (runAsNonRoot=false or unset)..."
mapfile -t NON_COMPLIANT_ROWS < <(
  kubectl get pods --all-namespaces -o json | jq -r '
    [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
    | select($ok | not)
    | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
      + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
      + (if $node   == ""   then "" else " node=\($node)" end)
      + (if $labels == ""   then "" else " labels=\($labels)" end)
      + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
      + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
      + " is_compliant=false"
    ] as $rows
    | if ($rows | length) == 0 then empty else $rows[] end
  '
)

if [ "${#NON_COMPLIANT_ROWS[@]}" -eq 0 ]; then
  echo "No non-compliant containers found. Nothing to do."
  exit 0
fi

echo "Found ${#NON_COMPLIANT_ROWS[@]} non-compliant container entries."

# Build a unique list of owners (kind, namespace, name)
declare -A OWNERS
for row in "${NON_COMPLIANT_ROWS[@]}"; do
  # owner=Kind/ns/name/uid
  owner_field=$(grep -o 'owner=[^ ]*' <<<"$row" || true)
  if [ -z "$owner_field" ]; then
    # Standalone Pod (no controller) – patch the Pod template directly
    # Key format: Pod|<namespace>|<name>
    ns=$(grep -o 'ns=[^ ]*' <<<"$row" | cut -d= -f2)
    podname=$(grep -o 'name=[^ ]*' <<<"$row" | cut -d= -f2)
    key="Pod|${ns}|${podname}"
  else
    owner_val=${owner_field#owner=}
    IFS=/ read -r owner_kind owner_ns owner_name _uid <<<"$owner_val"
    key="${owner_kind}|${owner_ns}|${owner_name}"
  fi
  OWNERS["$key"]=1
done

echo "Will patch ${#OWNERS[@]} owning resources (Pods/Controllers)."

# Function: patch a workload YAML to set runAsNonRoot: true
patch_manifest_run_as_non_root() {
  local yaml_file=$1
  # Pod spec path may vary by kind
  # - Pod, Deployment, DaemonSet, StatefulSet, ReplicaSet, Job: .spec.template.spec or .spec
  # - CronJob: .spec.jobTemplate.spec.template.spec
  # We’ll set both pod-level and per-container for safety.

  yq -i '
    # Detect base path for pod spec
    ( .kind == "Pod" )
    as $isPod |

    ( .kind == "CronJob" )
    as $isCron |

    # Helper for setting runAsNonRoot on a pod spec node
    def set_runnr(path):
      . as $root
      | (path | $root) as $pod
      | if $pod == null then $root
        else
          $root
          | (path + ".securityContext.runAsNonRoot") |= (true)
          | (path + ".containers[]?.securityContext.runAsNonRoot") |= (true)
          | (path + ".initContainers[]?.securityContext.runAsNonRoot") |= (true)
        end;

    # For Pod
    if $isPod then
      set_runnr(".spec")
    # For CronJob: jobTemplate.spec.template.spec
    elif $isCron then
      set_runnr(".spec.jobTemplate.spec.template.spec")
    # For other workload types: spec.template.spec
    else
      set_runnr(".spec.template.spec")
    end
  ' "$yaml_file"
}

# Process each unique owner
for key in "${!OWNERS[@]}"; do
  IFS='|' read -r kind ns name <<<"$key"

  echo "Processing owner: kind=${kind} ns=${ns} name=${name}"

  tmp_yaml=$(mktemp)
  trap 'rm -f "$tmp_yaml"' EXIT

  if [ "$kind" = "Pod" ]; then
    # Standalone Pod: get and patch the Pod manifest
    if ! kubectl get pod "$name" -n "$ns" -o yaml >"$tmp_yaml"; then
      echo "WARNING: Pod ${ns}/${name} not found, skipping."
      continue
    fi
  else
    if ! kubectl get "$kind" "$name" -n "$ns" -o yaml >"$tmp_yaml"; then
      echo "WARNING: ${kind} ${ns}/${name} not found, skipping."
      continue
    fi
  fi

  # Patch the manifest locally
  patch_manifest_run_as_non_root "$tmp_yaml"

  # Apply back to the cluster
  echo "  Applying patched manifest for ${kind} ${ns}/${name}..."
  kubectl apply -f "$tmp_yaml"

  rm -f "$tmp_yaml"
  trap - EXIT
done

echo "Patching complete. Waiting for workloads to reconcile..."
sleep 10

echo "Re-running compliance audit to verify..."
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.securityContext.runAsNonRoot // false) as $podNonRoot
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | ($podNonRoot or (.securityContext.runAsNonRoot // false)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image) runAsNonRoot=\($ok)"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end
' | tee /tmp/runAsNonRoot_verification.txt

echo "Verification output saved to /tmp/runAsNonRoot_verification.txt"
echo "Review any lines with is_compliant=false; those require manual analysis (image may require root or owner kind unsupported by this script)."