Containers Should Use A Read-Only Root Filesystem

More Info:

Verifies readOnlyRootFilesystem is true. A writable root filesystem lets an attacker persist tools or modify binaries inside a running container.

Risk Level

Medium

Address

Security

Compliance Standards

• Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Identify noncompliant pods and their owning workloads (run on any machine with kubectl access):

   kubectl get pods --all-namespaces -o json | jq -r '
     [ .items[]
     | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
     | .metadata as $m
     | (.spec.nodeName // "") as $node
     | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
     | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
     | ((.spec.containers // []) + (.spec.initContainers // []))[]
     | (.securityContext.readOnlyRootFilesystem == true) as $ok
     | select($ok|not)
     | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
       + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
       + (if $node   == ""   then "" else " node=\($node)" end)
       + (if $labels == ""   then "" else " labels=\($labels)" end)
       + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
       + " container=\(.name) image=\(.image)"
       + " readOnlyRootFilesystem=\(.securityContext.readOnlyRootFilesystem // "unset")"
       + " is_compliant=false"
     ][]'

2. For each affected Pod that is controlled by a Deployment/StatefulSet/DaemonSet/Job/CronJob, edit the owning workload manifest to set a read-only root filesystem (run on any machine with kubectl access). Example for a Deployment:

   kubectl -n NAMESPACE edit deployment DEPLOYMENT_NAME

   In each container (including initContainers if present) under spec.template.spec.containers[] add or update:

   securityContext:
     readOnlyRootFilesystem: true

3. If a container needs write access to specific paths, mount an emptyDir instead of relying on a writable root (same edit session as step 2). Under spec.template.spec.volumes add:

   volumes:
   - name: writable-tmp
     emptyDir: {}

   Then, in the relevant container, add a volumeMounts entry:

   volumeMounts:
   - name: writable-tmp
     mountPath: /path/that/needs/write

4. For standalone Pods (no controller in ownerReferences), export, modify, and re-apply (run on any machine with kubectl access):

   kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod-POD_NAME.yaml

   Edit /tmp/pod-POD_NAME.yaml:

   • Remove fields status, metadata.resourceVersion, metadata.uid, metadata.selfLink, metadata.creationTimestamp, metadata.managedFields.
   • Under each container and initContainer, set:

     securityContext:
       readOnlyRootFilesystem: true

   • Optionally define emptyDir volumes and volumeMounts for writable paths as in step 3. Then delete and recreate the Pod:

   kubectl -n NAMESPACE delete pod POD_NAME
   kubectl -n NAMESPACE apply -f /tmp/pod-POD_NAME.yaml

5. Wait for updated workloads to roll out and ensure Pods are running (run on any machine with kubectl access):

   kubectl -n NAMESPACE get pods -o wide

6. Verify compliance (run on any machine with kubectl access):

   kubectl get pods --all-namespaces -o json | jq -r '
     [ .items[]
     | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
     | .metadata as $m
     | (.spec.nodeName // "") as $node
     | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
     | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
     | ((.spec.containers // []) + (.spec.initContainers // []))[]
     | (.securityContext.readOnlyRootFilesystem == true) as $ok
     | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
       + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
       + (if $node   == ""   then "" else " node=\($node)" end)
       + (if $labels == ""   then "" else " labels=\($labels)" end)
       + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
       + " container=\(.name) image=\(.image)"
       + " readOnlyRootFilesystem=\(.securityContext.readOnlyRootFilesystem // "unset")"
       + " is_compliant=\(if $ok then "true" else "false" end)"
     ] as $rows
     | if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
       then "is_compliant=true"
       else $rows[]
       end'

Using kubectl

On any machine with kubectl access:

1. Identify non‑compliant pods and owning controllers

kubectl get pods --all-namespaces -o wide

For each non‑compliant pod, note the OWNER from the audit output (e.g., Deployment, StatefulSet, DaemonSet, Job, CronJob) and patch that controller, not the Pod.

2. Example: patch a Deployment to use a read‑only root filesystem

kubectl -n <namespace> get deploy <deployment-name> -o yaml > /tmp/deploy.yaml

Edit /tmp/deploy.yaml and, for each affected container, ensure:

spec:
  template:
    spec:
      containers:
        - name: <container-name>
          image: <image>
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: writable-tmp
              mountPath: /tmp
      volumes:
        - name: writable-tmp
          emptyDir: {}

Apply the manifest:

kubectl apply -f /tmp/deploy.yaml

3. Example: patch a single container in a Deployment (no extra volumes needed)

kubectl -n <namespace> patch deploy <deployment-name> \
  --type='json' \
  -p='[
    {
      "op": "add",
      "path": "/spec/template/spec/containers/0/securityContext",
      "value": { "readOnlyRootFilesystem": true }
    }
  ]'

Adjust the container index in /containers/0/ if needed.

4. Example: patch a DaemonSet similarly

kubectl -n <namespace> get ds <daemonset-name> -o yaml > /tmp/ds.yaml
# edit as in the Deployment example, then:
kubectl apply -f /tmp/ds.yaml

5. Example manifest snippet for new workloads

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
    spec:
      containers:
        - name: app
          image: nginx:stable
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: writable-tmp
              mountPath: /tmp
      volumes:
        - name: writable-tmp
          emptyDir: {}

Apply with:

kubectl apply -f example-deploy.yaml

6. Verification

Run the original audit command on any machine with kubectl access and confirm all listed is_compliant=true:

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | (.securityContext.readOnlyRootFilesystem == true) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image)"
    + " readOnlyRootFilesystem=\(.securityContext.readOnlyRootFilesystem // "unset")"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Automation

#!/usr/bin/env bash
#
# Enforce readOnlyRootFilesystem=true on all non-system workloads
# by patching Deployments, StatefulSets, and DaemonSets.
#
# Requirements:
#   - Run on any machine with kubectl access and jq installed.
#   - kubectl current-context must point to the target cluster.
#
# Notes:
#   - Only affects namespaces other than: kube-system, kube-public, kube-node-lease
#   - Only touches containers/initContainers that do NOT already set readOnlyRootFilesystem.
#   - Safe to re-run (idempotent patches).
#   - You MUST ensure affected containers do not need to write to the root FS.
#     If they do, update manifests to mount an emptyDir at the writable path
#     before/after running this script.

set -euo pipefail

# Namespaces to exclude (system namespaces)
EXCLUDED_NAMESPACES="kube-system kube-public kube-node-lease"

# Verify required tools
command -v kubectl >/dev/null 2>&1 || {
  echo "ERROR: kubectl not found in PATH" >&2
  exit 1
}
command -v jq >/dev/null 2>&1 || {
  echo "ERROR: jq not found in PATH" >&2
  exit 1
}

echo "Discovering target namespaces..."
all_ns=$(kubectl get ns -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}')
targets=()
for ns in $all_ns; do
  skip=false
  for ex in $EXCLUDED_NAMESPACES; do
    if [[ "$ns" == "$ex" ]]; then
      skip=true
      break
    fi
  done
  if ! $skip; then
    targets+=("$ns")
  fi
done

if [[ ${#targets[@]} -eq 0 ]]; then
  echo "No non-system namespaces found. Nothing to do."
  exit 0
fi

echo "Target namespaces: ${targets[*]}"

patch_workload_type() {
  local kind="$1"  # Deployment, StatefulSet, DaemonSet

  echo
  echo "Processing $kind objects..."

  for ns in "${targets[@]}"; do
    # Get all objects of this kind in the namespace
    mapfile -t objs < <(kubectl get "$kind" -n "$ns" -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true)
    [[ ${#objs[@]} -eq 0 ]] && continue

    for name in "${objs[@]}"; do
      # Build a strategic merge patch that:
      # - Ensures securityContext exists for each container/initContainer
      # - Sets readOnlyRootFilesystem: true only where it is currently unset
      #
      # This uses jq on the existing object to construct the patch, which
      # makes it idempotent and avoids clobbering other settings.

      obj_json=$(kubectl get "$kind" "$name" -n "$ns" -o json)

      patch=$(echo "$obj_json" | jq '{
        "spec": {
          "template": {
            "spec": {
              "containers": (
                (.spec.template.spec.containers // [])
                | map(
                    if (.securityContext.readOnlyRootFilesystem // null) == null then
                      .securityContext = (.securityContext // {}) |
                      .securityContext.readOnlyRootFilesystem = true
                    else
                      .
                    end
                  )
              ),
              "initContainers": (
                (.spec.template.spec.initContainers // [])
                | map(
                    if (.securityContext.readOnlyRootFilesystem // null) == null then
                      .securityContext = (.securityContext // {}) |
                      .securityContext.readOnlyRootFilesystem = true
                    else
                      .
                    end
                  )
              )
            }
          }
        }
      }')

      # Skip patch if it would not change anything (no containers/initContainers)
      # or all already have readOnlyRootFilesystem set.
      # We detect a no-op by comparing serialized templates before and after.
      before_tpl=$(echo "$obj_json" | jq '.spec.template.spec')
      after_tpl=$(echo "$before_tpl" | jq '
        . as $orig |
        {
          "containers": (
            (.containers // [])
            | map(
                if (.securityContext.readOnlyRootFilesystem // null) == null then
                  .securityContext = (.securityContext // {}) |
                  .securityContext.readOnlyRootFilesystem = true
                else
                  .
                end
              )
          ),
          "initContainers": (
            (.initContainers // [])
            | map(
                if (.securityContext.readOnlyRootFilesystem // null) == null then
                  .securityContext = (.securityContext // {}) |
                  .securityContext.readOnlyRootFilesystem = true
                else
                  .
                end
              )
          )
        }')

      if [[ "$(echo "$before_tpl" | jq -c '.')" == "$(echo "$after_tpl" | jq -c '.')" ]]; then
        echo "$kind/$ns/$name: already compliant or no containers; skipping"
        continue
      fi

      echo "$kind/$ns/$name: applying readOnlyRootFilesystem=true to unset containers..."
      echo "$patch" | kubectl patch "$kind" "$name" -n "$ns" --type=merge -p "$(cat)" >/dev/null
    done
  done
}

# Apply patches to common workload types
patch_workload_type Deployment
patch_workload_type StatefulSet
patch_workload_type DaemonSet

echo
echo "Waiting for updated Pods to be ready..."
kubectl wait --for=condition=Available deploy --all -A --timeout=5m 2>/dev/null || true
kubectl wait --for=condition=Ready pod --all -A --timeout=5m 2>/dev/null || true

echo
echo "Verification (should show is_compliant=true OR no rows):"
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | ((.spec.containers // []) + (.spec.initContainers // []))[]
  | (.securityContext.readOnlyRootFilesystem == true) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image)"
    + " readOnlyRootFilesystem=\(.securityContext.readOnlyRootFilesystem // "unset")"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'