Skip to main content

Containers Should Set CPU And Memory Requests

More Info:​

Verifies every container sets resources.requests so the scheduler can place the pod correctly and QoS is not BestEffort.

Risk Level​

Low

Address​

Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Manual Steps

  1. Identify non-compliant pods (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
      | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
      | .metadata as $m
      | (.spec.containers // [])[]
      | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok
      | select($ok | not)
      | "ns=\($m.namespace) pod=\($m.name) container=\(.name)"
      ][]'

  2. For a non-compliant pod owned by a higher-level controller (Deployment/StatefulSet/DaemonSet), edit the controller so all containers set CPU and memory requests (run on any machine with kubectl access). Example for a Deployment:

    kubectl -n <namespace> edit deployment <deployment-name>

    In each container under spec.template.spec.containers[], add or update:

    resources:
      requests:
        cpu: "100m"
        memory: "128Mi"

    Choose values appropriate for your workload.

  3. For a non-compliant pod created directly (no controller owner), edit the Pod spec (run on any machine with kubectl access). Note this will delete and recreate the Pod:

    kubectl -n <namespace> get pod <pod-name> -o yaml > /tmp/pod-<namespace>-<pod-name>.yaml

    # Edit the file
    vi /tmp/pod-<namespace>-<pod-name>.yaml

    Under each spec.containers[], add or update:

    resources:
      requests:
        cpu: "100m"
        memory: "128Mi"

    Then recreate:

    kubectl -n <namespace> delete pod <pod-name>
    kubectl -n <namespace> apply -f /tmp/pod-<namespace>-<pod-name>.yaml

  4. For workloads managed via manifests or GitOps, update the source YAML so changes persist (run on your manifest/IaC repository, then apply from any machine with kubectl access):

    # edit your deployment/statefulset/daemonset yaml in the repo
    # ensure each container has:
    resources:
      requests:
        cpu: "100m"
        memory: "128Mi"

    Then apply:

    kubectl apply -f <path-to-updated-manifest>.yaml

  5. Wait for updated workloads to roll out and pods to be recreated (run on any machine with kubectl access):

    kubectl -n <namespace> rollout status deployment/<deployment-name>
    kubectl -n <namespace> get pods -o wide

  6. Verify compliance (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
      | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
      | .metadata as $m
      | (.spec.nodeName // "") as $node
      | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
      | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
      | (.spec.containers // [])[]
      | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok
      | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
        + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
        + (if $node   == ""   then "" else " node=\($node)" end)
        + (if $labels == ""   then "" else " labels=\($labels)" end)
        + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
        + " container=\(.name) image=\(.image)"
        + " requestsCpu=\(.resources.requests.cpu // "unset") requestsMemory=\(.resources.requests.memory // "unset")"
        + " is_compliant=\(if $ok then "true" else "false" end)"
      ] as $rows
      | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

    Ensure either the output is is_compliant=true or every listed container shows is_compliant=true.

Using kubectl

On any machine with kubectl access:

  1. Identify non‑compliant pods and their controllers
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.containers // [])[]
  | ((.resources.requests.cpu == null) or (.resources.requests.memory == null)) as $bad
  | select($bad)
  | "ns=\($m.namespace) pod=\($m.name) ownerKind=\($own.kind // "Pod") ownerName=\($own.name // $m.name)"
  ][]'

Focus on the ownerKind/ownerName (Deployment, StatefulSet, Job, etc.). Edit the controller, not the live pod.

  1. Edit a controller to add requests (example: Deployment)
kubectl -n <namespace> get deploy <deployment-name> -o yaml > /tmp/deploy-with-requests.yaml

Edit /tmp/deploy-with-requests.yaml and, for each container under spec.template.spec.containers, add a resources.requests block, for example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <deployment-name>
  namespace: <namespace>
spec:
  template:
    spec:
      containers:
        - name: <container-name>
          image: <image>
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            # optional but recommended
            # limits:
            #   cpu: "500m"
            #   memory: "512Mi"

Apply the updated manifest:

kubectl apply -f /tmp/deploy-with-requests.yaml

This will roll pods for that Deployment as the template changes.

  1. Edit other owner types similarly (examples)

StatefulSet:

kubectl -n <namespace> get statefulset <sts-name> -o yaml > /tmp/sts-with-requests.yaml
# edit containers[].resources.requests as above
kubectl apply -f /tmp/sts-with-requests.yaml

DaemonSet:

kubectl -n <namespace> get daemonset <ds-name> -o yaml > /tmp/ds-with-requests.yaml
# edit containers[].resources.requests as above
kubectl apply -f /tmp/ds-with-requests.yaml

CronJob (template is nested):

kubectl -n <namespace> get cronjob <cronjob-name> -o yaml > /tmp/cronjob-with-requests.yaml

Edit:

spec:
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: <container-name>
              resources:
                requests:
                  cpu: "100m"
                  memory: "128Mi"

Apply:

kubectl apply -f /tmp/cronjob-with-requests.yaml

For standalone Pods created directly (ownerKind=Pod), either:

  • Edit in place (ephemeral; lost if pod is recreated by external system):

    kubectl -n <namespace> get pod <pod-name> -o yaml > /tmp/pod-with-requests.yaml
    # edit containers[].resources.requests
    kubectl delete pod -n <namespace> <pod-name>
    kubectl apply -f /tmp/pod-with-requests.yaml

  • Or better, manage them via a controller manifest going forward.

Operational impact: changing a controller’s pod template will trigger a rollout and recreate pods; ensure this is acceptable and coordinate if needed.

  1. Verification

Run the benchmark audit command again from any machine with kubectl access:

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.nodeName // "") as $node
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
  | (.spec.containers // [])[]
  | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok
  | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $node   == ""   then "" else " node=\($node)" end)
    + (if $labels == ""   then "" else " labels=\($labels)" end)
    + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
    + " container=\(.name) image=\(.image)"
    + " requestsCpu=\(.resources.requests.cpu // "unset") requestsMemory=\(.resources.requests.memory // "unset")"
    + " is_compliant=\(if $ok then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'
Automation
#!/usr/bin/env bash
#
# Ensure every non-system Pod container has CPU and memory requests set.
# Platform: Oracle OKE (or any standard Kubernetes cluster)
#
# Behavior:
# - Skips kube-system, kube-public, kube-node-lease
# - Patches only Pods/containers where requests.cpu or requests.memory is unset
# - Default requests: cpu=100m, memory=128Mi (adjust below if needed)
# - Idempotent: safe to re-run
#
# Requirements:
# - Run on any machine with kubectl and jq installed and access to the cluster
# - kubectl must be configured with appropriate permissions

set -euo pipefail

DEFAULT_CPU_REQUEST="100m"
DEFAULT_MEM_REQUEST="128Mi"

echo "Discovering non-compliant Pods/containers..."

NON_COMPLIANT_JSON=$(kubectl get pods --all-namespaces -o json | jq -c '
  .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | {
      ns: .metadata.namespace,
      name: .metadata.name,
      containers: (
        [ .spec.containers[]
          | {
              name,
              hasCpu: (.resources.requests.cpu != null),
              hasMem: (.resources.requests.memory != null)
            }
          | select(.hasCpu == false or .hasMem == false)
        ]
      )
    }
  | select(.containers | length > 0)
')

if [ -z "$NON_COMPLIANT_JSON" ]; then
  echo "No non-compliant Pods found."
else
  echo "Patching non-compliant Pods to set CPU and memory requests..."
fi

# Iterate over each non-compliant Pod
echo "$NON_COMPLIANT_JSON" | jq -c '.' | while read -r POD; do
  NS=$(echo "$POD" | jq -r '.ns')
  NAME=$(echo "$POD" | jq -r '.name')

  echo "Processing Pod ${NS}/${NAME}..."

  # Build a strategic merge patch for this Pod
  # We need to inspect the current Pod spec to avoid overwriting existing values
  POD_JSON=$(kubectl get pod "$NAME" -n "$NS" -o json)

  # Construct new containers array with ensured requests
  NEW_CONTAINERS=$(echo "$POD_JSON" | jq --arg cpu "$DEFAULT_CPU_REQUEST" --arg mem "$DEFAULT_MEM_REQUEST" '
    .spec.containers
    | map(
        .resources.requests = (
          (.resources.requests // {})
          | if (.cpu == null) then .cpu = $cpu else . end
          | if (.memory == null) then .memory = $mem else . end
        )
      )
  ')

  # Create patch JSON containing only the updated containers spec
  PATCH=$(jq -n --argjson containers "$NEW_CONTAINERS" '{spec: {containers: $containers}}')

  echo "$PATCH" | kubectl patch pod "$NAME" -n "$NS" --type merge -p "$(cat)"

done

echo "Verifying compliance..."

kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
  | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (.spec.containers // [])[]
  | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok
  | select($ok | not)
  ] as $rows
  | if ($rows | length) == 0
    then "All non-system Pods have cpu and memory requests set (is_compliant=true)."
    else "Non-compliant containers remain (is_compliant=false). Re-run or inspect manually."
    end'