> ## Documentation Index > Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Containers Should Set CPU And Memory Requests ### More Info: Verifies every container sets resources.requests so the scheduler can place the pod correctly and QoS is not BestEffort. ### Risk Level Low ### Address Security ### Compliance Standards * Cloudanix Best Practice ### Triage and Remediation ### Remediation 1. List all non-exempt pods and identify offenders (run on any machine with kubectl access): ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.containers // [])[] | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | select($ok | not) | "ns=\($m.namespace) pod=\($m.name) container=\(.name)" ] | unique[]' ``` 2. For a pod managed by a higher-level controller (Deployment/ReplicaSet/StatefulSet/DaemonSet/Job/CronJob), get and edit the owning object’s manifest (example for a Deployment; run on any machine with kubectl access): ```bash theme={null} kubectl -n NAMESPACE get deployment DEPLOYMENT_NAME -o yaml > /tmp/deploy.yaml vi /tmp/deploy.yaml ``` Under each container in `spec.template.spec.containers`, add or update: ```yaml theme={null} resources: requests: cpu: "100m" memory: "128Mi" ``` (Choose values appropriate for the application.) 3. Apply the updated controller manifest (run on any machine with kubectl access): ```bash theme={null} kubectl apply -f /tmp/deploy.yaml ``` Repeat steps 2–3 for other controller types (`kubectl get statefulset`, `kubectl get daemonset`, `kubectl get job`, `kubectl get cronjob`) that own non-compliant pods. 4. For stand-alone Pods (no controller ownerReference), edit the Pod spec directly (run on any machine with kubectl access): ```bash theme={null} kubectl -n NAMESPACE edit pod POD_NAME ``` In each `spec.containers[]`, ensure: ```yaml theme={null} resources: requests: cpu: "100m" memory: "128Mi" ``` Note: editing certain fields of a running Pod may be restricted; if so, recreate the Pod with a corrected manifest: ```bash theme={null} kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod.yaml # edit /tmp/pod.yaml: add resources.requests and remove fields under metadata.status, status, and cluster-assigned fields like metadata.uid, resourceVersion vi /tmp/pod.yaml kubectl -n NAMESPACE delete pod POD_NAME kubectl -n NAMESPACE apply -f /tmp/pod.yaml ``` 5. For workloads managed by AKS add-ons or third-party operators, adjust their Helm chart values or operator configuration so that the generated Pod templates include `resources.requests.cpu` and `resources.requests.memory` for all containers, then redeploy using the add-on’s/Helm’s normal process. 6. Verify compliance for all pods (run on any machine with kubectl access): ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.containers // [])[] | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | select($ok | not) ] as $rows | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end' ``` The output should be `is_compliant=true`. On any machine with kubectl access: 1. Identify non-compliant pods and their controllers ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own | (.spec.containers // [])[] | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | select($ok | not) | "ns=\($m.namespace) pod=\($m.name) ownerKind=\($own.kind // "Pod") ownerName=\($own.name // "")" ][]' ``` Focus on the owning workload (Deployment, StatefulSet, Job, etc.), not the pod itself. 2. Export the owning workload manifest (example for a Deployment) ```bash theme={null} kubectl -n NAMESPACE get deploy DEPLOYMENT_NAME -o yaml > /tmp/deploy-with-requests.yaml ``` 3. Edit containers to add `resources.requests` (CPU and memory) in the manifest In `/tmp/deploy-with-requests.yaml`, under each container in `spec.template.spec.containers`, ensure a block like: ```yaml theme={null} resources: requests: cpu: "100m" memory: "128Mi" ``` Adjust values to match application requirements. Do this for every container that currently lacks requests or has only one of CPU/memory. 4. Apply the updated manifest ```bash theme={null} kubectl apply -f /tmp/deploy-with-requests.yaml ``` Repeat steps 2–4 for each affected controller type (e.g., `kubectl get statefulset`, `kubectl get job`, etc.). 5. For standalone Pods managed directly (no ownerReferences) Export, edit, and re-create (pods are immutable): ```bash theme={null} kubectl -n NAMESPACE get pod POD_NAME -o yaml > /tmp/pod-with-requests.yaml ``` Edit each container to add: ```yaml theme={null} resources: requests: cpu: "100m" memory: "128Mi" ``` Remove runtime-only fields before re-creating: ```bash theme={null} yq 'del(.metadata.resourceVersion, .metadata.uid, .metadata.creationTimestamp, .metadata.managedFields, .status)' \ /tmp/pod-with-requests.yaml > /tmp/pod-with-requests-clean.yaml kubectl -n NAMESPACE delete pod POD_NAME kubectl apply -f /tmp/pod-with-requests-clean.yaml ``` 6. Verification Run the original audit and confirm all listed rows have `is_compliant=true` (or that it prints just `is_compliant=true`): ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.nodeName // "") as $node | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own | (.spec.containers // [])[] | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1" + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end) + (if $node == "" then "" else " node=\($node)" end) + (if $labels == "" then "" else " labels=\($labels)" end) + (if $own == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end) + " container=\(.name) image=\(.image)" + " requestsCpu=\(.resources.requests.cpu // "unset") requestsMemory=\(.resources.requests.memory // "unset")" + " is_compliant=\(if $ok then "true" else "false" end)" ] as $rows | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end' ``` ```bash theme={null} #!/usr/bin/env bash # # Fix CBP C1.9: Ensure every container sets CPU and memory requests # Target: any machine with kubectl access to the AKS cluster # # This script: # - Scans all non-system namespaces for Pods whose containers lack CPU and/or memory requests # - For workload-backed Pods (Deployment/ReplicaSet/StatefulSet/DaemonSet/Job/CronJob), # patches the owning workload to add default requests where missing # - Skips bare Pods (no controller ownerReferences) — these must be fixed manually # - Re-runs the audit to verify # # NOTE: # - You MUST review and customize the default request values below for your cluster. # - Changing workload specs will create new Pods; expect rolling restarts. # set -euo pipefail #----------------------------- # Configurable defaults #----------------------------- # Default requests to apply where missing. Adjust per your standards. DEFAULT_CPU_REQUEST="100m" DEFAULT_MEM_REQUEST="128Mi" # Namespaces to exclude from modification (system namespaces) EXCLUDED_NAMESPACES_REGEX='^(kube-system|kube-public|kube-node-lease)$' #----------------------------- # Preconditions #----------------------------- command -v kubectl >/dev/null 2>&1 || { echo "ERROR: kubectl not found in PATH" >&2 exit 1 } command -v jq >/dev/null 2>&1 || { echo "ERROR: jq not found in PATH" >&2 exit 1 } # Ensure we can talk to the cluster if ! kubectl version --request-timeout=10s >/dev/null 2>&1; then echo "ERROR: kubectl cannot reach the cluster. Check context and network." >&2 exit 1 fi #----------------------------- # Helper functions #----------------------------- # Patch a workload's pod template to ensure all containers have requests. # Arguments: # 1: namespace # 2: workload kind (Deployment/StatefulSet/DaemonSet/Job/CronJob/ReplicaSet) # 3: workload name patch_workload() { local ns="$1" local kind="$2" local name="$3" echo "Processing ${kind}/${ns}/${name} ..." # Get current spec template local tmpl if ! tmpl="$(kubectl -n "${ns}" get "${kind}" "${name}" -o json)"; then echo " WARN: Could not fetch ${kind}/${ns}/${name}, skipping" >&2 return fi # Build a patched pod template with ensured requests for all containers local patched patched="$(echo "${tmpl}" | jq --arg cpu "${DEFAULT_CPU_REQUEST}" --arg mem "${DEFAULT_MEM_REQUEST}" ' .spec.template.spec as $spec | .spec.template.spec.containers |= (map( .resources.requests |= ( if . == null then {cpu: $cpu, memory: $mem} else .cpu |= (if . == null then $cpu else . end) | .memory |= (if . == null then $mem else . end) end ) )) ')" # Apply patch via strategic merge (idempotent) # Extract only the template piece to avoid unintended changes local patch patch="$(echo "${patched}" | jq '{spec: {template: .spec.template}}')" echo "${patch}" | kubectl -n "${ns}" patch "${kind}" "${name}" --type merge -p "$(cat)" >/dev/null echo " Patched ${kind}/${ns}/${name}" } # Determine patchable owner kind (top-level) from ownerReferences # Takes JSON of ownerReferences[]; prints KIND NAMESPACE NAME on success, nothing otherwise get_owner_ref() { local pod_json="$1" # Owner kind we will patch (only controller owners) echo "${pod_json}" | jq -r ' .metadata as $m | ([($m.ownerReferences // [])[] | select(.controller == true)] | first) as $own | if $own == null then empty else "\($own.kind) \($m.namespace) \($own.name)" end ' } #----------------------------- # MAIN #----------------------------- echo "Scanning for non-compliant containers (missing CPU and/or memory requests)..." # Get all Pods in non-system namespaces pods_json="$(kubectl get pods --all-namespaces -o json)" # Collect non-compliant Pods non_compliant_pods="$(echo "${pods_json}" | jq -c ' .items[] | select(.metadata.namespace | test("'"${EXCLUDED_NAMESPACES_REGEX}"'") | not) | . as $pod | (.spec.containers // [])[] as $c | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | select($ok | not) | $pod ' | jq -s '.')" if [ "$(echo "${non_compliant_pods}" | jq 'length')" -eq 0 ]; then echo "No non-compliant pods found; nothing to do." else echo "Found non-compliant pods; identifying owning workloads..." # Map of owner (kind/ns/name) to any (dummy) value, to deduplicate # We'll feed this through jq to extract unique owners owners="$(echo "${non_compliant_pods}" | jq -c ' .[] | . as $p | $p.metadata.namespace as $ns | ($p.metadata.ownerReferences // []) as $owners | [ $owners[] | select(.controller == true) ] | first as $own | if $own == null then empty else { kind: $own.kind, namespace: $ns, name: $own.name } end ' | jq -s 'unique_by(.kind, .namespace, .name)')" # Log bare Pods (no controller) bare_pods="$(echo "${non_compliant_pods}" | jq -r ' .[] | .metadata as $m | [($m.ownerReferences // [])[] | select(.controller == true)] | first as $own | select($own == null) | "Bare Pod: ns=\($m.namespace) name=\($m.name)" ')" if [ -n "${bare_pods}" ]; then echo "NOTICE: The following non-compliant Pods are not controlled by a workload and must be fixed manually:" echo "${bare_pods}" echo fi # Patch each unique owner echo "${owners}" | jq -c '.[]' | while read -r o; do kind="$(echo "${o}" | jq -r '.kind')" ns="$(echo "${o}" | jq -r '.namespace')" name="$(echo "${o}" | jq -r '.name')" case "${kind}" in Deployment|StatefulSet|DaemonSet|Job|ReplicaSet) patch_workload "${ns}" "${kind}" "${name}" ;; CronJob) # For CronJob, the Pod template is at spec.jobTemplate.spec.template echo "Processing CronJob/${ns}/${name} ..." cj_json="$(kubectl -n "${ns}" get CronJob "${name}" -o json)" || { echo " WARN: Could not fetch CronJob/${ns}/${name}, skipping" >&2 continue } patched_cj="$(echo "${cj_json}" | jq --arg cpu "${DEFAULT_CPU_REQUEST}" --arg mem "${DEFAULT_MEM_REQUEST}" ' .spec.jobTemplate.spec.template.spec.containers |= (map( .resources.requests |= ( if . == null then {cpu: $cpu, memory: $mem} else .cpu |= (if . == null then $cpu else . end) | .memory |= (if . == null then $mem else . end) end ) )) ')" patch="$(echo "${patched_cj}" | jq '{spec: {jobTemplate: {spec: {template: .spec.jobTemplate.spec.template}}}}')" echo "${patch}" | kubectl -n "${ns}" patch CronJob "${name}" --type merge -p "$(cat)" >/dev/null echo " Patched CronJob/${ns}/${name}" ;; *) echo "WARN: Owner kind ${kind}/${ns}/${name} is not handled automatically; fix its manifest manually." >&2 ;; esac done fi echo echo "Waiting 10 seconds for workloads to roll out new Pods..." sleep 10 echo echo "Re-running compliance audit..." kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.nodeName // "") as $node | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own | (.spec.containers // [])[] | ((.resources.requests.cpu != null) and (.resources.requests.memory != null)) as $ok | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1" + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end) + (if $node == "" then "" else " node=\($node)" end) + (if $labels == "" then "" else " labels=\($labels)" end) + (if $own == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end) + " container=\(.name) image=\(.image)" + " requestsCpu=\(.resources.requests.cpu // "unset") requestsMemory=\(.resources.requests.memory // "unset")" + " is_compliant=\(if $ok then "true" else "false" end)" ] as $rows | if ($rows | map(select(. | test("is_compliant=false"))) | length) == 0 then "is_compliant=true" else $rows[] end ' ```