Containers Should Set CPU And Memory Limits

More Info:

Verifies every container sets resources.limits.cpu and resources.limits.memory so a single workload cannot exhaust a node.

Risk Level

Medium

Address

Security

Compliance Standards

Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

Identify noncompliant pods and their owning workloads (run on any machine with kubectl access):

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.containers // [])[]
  | ((.resources.limits.cpu != null) and (.resources.limits.memory != null)) as $ok
  | select($ok | not)
  | "ns=\($m.namespace) pod=\($m.name) container=\(.name)"
    + (if $own == null then "" else " ownerKind=\($own.kind) ownerName=\($own.name)" end)
  ][]'

For each owning workload (e.g., Deployment, StatefulSet, DaemonSet, Job, CronJob) that has noncompliant containers, export its manifest (run on any machine with kubectl access, substitute actual values):
```
kubectl get deployment DEPLOYMENT_NAME -n NAMESPACE -o yaml > /tmp/deployment-DEPLOYMENT_NAME.yaml
```
(Replace deployment with the actual kind, e.g. statefulset, daemonset, job, cronjob as needed.)
Edit the exported manifest to set CPU and memory limits on every container in the pod template (run on any machine with kubectl access, using your editor of choice, example with vi):
```
vi /tmp/deployment-DEPLOYMENT_NAME.yaml
```
Under each .spec.template.spec.containers[].resources, ensure you have both:
```
resources:
  limits:
    cpu: "500m"       # choose appropriate CPU limit
    memory: "512Mi"   # choose appropriate memory limit
```
If resources or limits is missing, add these sections. Set values according to your application requirements and node capacities.
Apply the updated manifest back to the cluster (run on any machine with kubectl access):
```
kubectl apply -f /tmp/deployment-DEPLOYMENT_NAME.yaml
```
Repeat steps 2–4 for each noncompliant owning workload identified in step 1.
For standalone Pods without a controller (no ownerKind/ownerName in step 1 output), recreate them with limits set:
- Get the current pod spec:
```
kubectl get pod POD_NAME -n NAMESPACE -o yaml > /tmp/pod-POD_NAME.yaml
```
- Edit /tmp/pod-POD_NAME.yaml to add resources.limits.cpu and resources.limits.memory for every container as in step 3, and remove cluster-assigned fields (e.g. metadata.resourceVersion, metadata.uid, metadata.creationTimestamp, status block).
- Delete and recreate the pod (run on any machine with kubectl access):
```
kubectl delete pod POD_NAME -n NAMESPACE
kubectl apply -f /tmp/pod-POD_NAME.yaml
```

Verify all non-excluded pods are now compliant (run on any machine with kubectl access):

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.containers // [])[]
  | ((.resources.limits.cpu != null) and (.resources.limits.memory != null)) as $ok
  | select($ok | not)
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else "is_compliant=false" end'

Using kubectl

On any machine with kubectl access:

Identify non-compliant pods and their controllers

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.containers // [])[]
  | ((.resources.limits.cpu != null) and (.resources.limits.memory != null)) as $ok
  | select($ok | not)
  | "\($m.namespace) \($m.name) \($own.kind // "Pod") \($own.name // $m.name)"
  ] | unique[]' | column -t

Focus on the owning controller kinds such as Deployment, StatefulSet, DaemonSet, Job, CronJob; editing those will fix their pods.

Edit a workload to add limits (example for a Deployment)

kubectl -n YOUR_NAMESPACE edit deployment YOUR_DEPLOYMENT_NAME

Under each spec.template.spec.containers[], add both cpu and memory limits, for example:

spec:
  template:
    spec:
      containers:
      - name: app
        image: your-image:tag
        resources:
          limits:
            cpu: "500m"
            memory: "256Mi"
          requests:
            cpu: "250m"
            memory: "128Mi"

Save and exit; Kubernetes will roll out updated pods.

Apply limits via manifest (declarative)

If you manage manifests (e.g., Git, Helm, Kustomize), edit the YAML for each controller and add the same resources block under each container, then apply:

kubectl apply -f PATH/TO/your-workload.yaml

Repeat for all controllers that create non-compliant pods.

For standalone Pods (no controller)

For pods whose owner kind is Pod (no higher-level controller), you must recreate them because pod specs are immutable:

kubectl -n YOUR_NAMESPACE get pod YOUR_POD -o yaml > /tmp/your-pod.yaml

Edit /tmp/your-pod.yaml:

Remove fields under metadata such as uid, resourceVersion, creationTimestamp, managedFields, selfLink, generation.
Remove status completely.
Under each spec.containers[], add:

resources:
  limits:
    cpu: "500m"
    memory: "256Mi"
  requests:
    cpu: "250m"
    memory: "128Mi"

Then delete and recreate:

kubectl -n YOUR_NAMESPACE delete pod YOUR_POD
kubectl apply -f /tmp/your-pod.yaml

Verification

Run the benchmark audit command again from any machine with kubectl access:

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.containers // [])[]
  | ((.resources.limits.cpu != null) and (.resources.limits.memory != null)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image)"
    + " limitsCpu=\(.resources.limits.cpu // "unset") limitsMemory=\(.resources.limits.memory // "unset")"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Non-compliant containers will show limitsCpu=unset or limitsMemory=unset; adjust remaining workloads until all are compliant.

Automation

#!/usr/bin/env bash
# Purpose: Ensure every non-excluded Pod container in an OKE cluster has CPU and memory limits.
# Scope: Run on any machine with kubectl access to the cluster.
# Requirements: kubectl, jq, and yq (https://github.com/mikefarah/yq) installed and in PATH.

set -euo pipefail

# --- Configuration: adjust as needed ---
# Default limits to apply where missing. Use Kubernetes resource quantities.
DEFAULT_CPU_LIMIT="500m"
DEFAULT_MEM_LIMIT="512Mi"

# Namespaces to exclude (system namespaces already excluded by the audit query)
EXCLUDED_NAMESPACES=("kube-system" "kube-public" "kube-node-lease")

# --- Helper functions ---

is_excluded_ns() {
  local ns="$1"
  for e in "${EXCLUDED_NAMESPACES[@]}"; do
    if [[ "$ns" == "$e" ]]; then
      return 0
    fi
  done
  return 1
}

kubectl_json() {
  kubectl "$@" -o json
}

# --- Discover non-compliant pods (same logic as audit, but only non-compliant) ---

echo "[INFO] Discovering Pods with missing CPU or memory limits..."

NONCOMPLIANT_PODS=$(kubectl_json get pods --all-namespaces | jq -r '
  .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | . as $pod
  | [ .spec.containers[] |
      select((.resources.limits.cpu == null) or (.resources.limits.memory == null))
    ] as $bad
  | select(($bad | length) > 0)
  | "\($pod.metadata.namespace) \($pod.metadata.name)"
')

if [[ -z "${NONCOMPLIANT_PODS}" ]]; then
  echo "[INFO] All Pods are already compliant; nothing to do."
else
  echo "[INFO] The following Pods are non-compliant (namespace name):"
  echo "$NONCOMPLIANT_PODS"
fi

# --- Attempt to fix via owning workload objects (Deployment, StatefulSet, etc.) ---

echo "[INFO] Updating owning workload specs where possible..."

while read -r NS POD; do
  [[ -z "$NS" || -z "$POD" ]] && continue
  if is_excluded_ns "$NS"; then
    continue
  fi

  OWNER_JSON=$(kubectl_json get pod "$POD" -n "$NS" | jq -c '
    ([.metadata.ownerReferences // [] | .[] | select(.controller)] | first) // null
  ')

  if [[ "$OWNER_JSON" == "null" ]]; then
    echo "[WARN] Pod $NS/$POD has no controller owner; skipping controller-based fix."
    continue
  fi

  OWNER_KIND=$(jq -r '.kind' <<<"$OWNER_JSON")
  OWNER_NAME=$(jq -r '.name' <<<"$OWNER_JSON")

  echo "[INFO] Processing owner $OWNER_KIND $NS/$OWNER_NAME for Pod $NS/$POD"

  # Determine the path to containers in the owner spec
  # Supported kinds: Deployment, StatefulSet, DaemonSet, Job, CronJob, ReplicaSet, ReplicationController
  case "$OWNER_KIND" in
    Deployment|StatefulSet|DaemonSet|ReplicaSet|ReplicationController)
      CONTAINERS_PATH=".spec.template.spec.containers"
      ;;
    Job)
      CONTAINERS_PATH=".spec.template.spec.containers"
      ;;
    CronJob)
      CONTAINERS_PATH=".spec.jobTemplate.spec.template.spec.containers"
      ;;
    *)
      echo "[WARN] Unsupported owner kind $OWNER_KIND for $NS/$OWNER_NAME; skipping."
      continue
      ;;
  esac

  # Fetch owner manifest as YAML, update limits with yq, and apply.
  TMPFILE="$(mktemp)"
  kubectl get "$OWNER_KIND" "$OWNER_NAME" -n "$NS" -o yaml > "$TMPFILE"

  # Use yq to add default limits only where missing.
  # This is idempotent: if limits exist, they are left unchanged.
  yq eval -i "
    ${CONTAINERS_PATH} |= ( . // [] | map(
      .resources |= (. // {}) |
      .resources.limits |= (. // {}) |
      (
        if (.resources.limits.cpu == null) then
          .resources.limits.cpu = \"${DEFAULT_CPU_LIMIT}\"
        else . end
      ) |
      (
        if (.resources.limits.memory == null) then
          .resources.limits.memory = \"${DEFAULT_MEM_LIMIT}\"
        else . end
      )
    ))
  " "$TMPFILE"

  echo "[INFO] Applying updated $OWNER_KIND $NS/$OWNER_NAME"
  kubectl apply -f "$TMPFILE"

  rm -f "$TMPFILE"
done <<< "$NONCOMPLIANT_PODS"

# --- Verification (same query as original audit) ---

echo "[INFO] Waiting briefly for controller Pods to be recreated..."
sleep 10

echo "[INFO] Verifying compliance with CPU and memory limits..."
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.containers // [])[]
  | ((.resources.limits.cpu != null) and (.resources.limits.memory != null)) as $ok
  | select($ok | not)
  ] as $rows
  | if ($rows | length) == 0 then
      "is_compliant=true"
    else
      "is_compliant=false"
    end
'

More Info:​

Risk Level​

Address​

Compliance Standards​

Triage and Remediation​

Remediation​