> ## Documentation Index > Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Sensitive Values Should Not Be Passed As Literal Env Vars ### More Info: Verifies secret-like env vars are not set as literal values. Literal values land in the pod manifest, logs and kubectl describe. ### Risk Level Medium ### Address Security ### Compliance Standards * Cloudanix Best Practice ### Triage and Remediation ### Remediation 1. Identify offending Pods and env vars (any machine with kubectl access) ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.containers // [] + .spec.initContainers // [])[] | .name as $c | (.env // [])[] | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i"))) | "ns=\($m.namespace) pod=\($m.name) container=\($c) env=\(.name)" ][]' ``` 2. For one violating Pod, capture its manifest (any machine with kubectl access)\ Replace `NAMESPACE` and `POD_NAME` with values from step 1. ```bash theme={null} kubectl get pod POD_NAME -n NAMESPACE -o yaml > /tmp/pod-POD_NAME.yaml ``` 3. Create a Secret containing the sensitive value (any machine with kubectl access)\ Choose a Secret name and key, then run: ```bash theme={null} kubectl create secret generic app-secret-POD_NAME \ -n NAMESPACE \ --from-literal=DB_PASSWORD='REPLACE_WITH_ACTUAL_PASSWORD' ``` Repeat with additional `--from-literal=KEY='VALUE'` flags for each sensitive env var. 4. Edit the Pod’s manifest to use `valueFrom.secretKeyRef` (any machine with kubectl access)\ Open the file from step 2 and, under the relevant container’s `env:` section, replace: ```yaml theme={null} - name: DB_PASSWORD value: "REPLACE_WITH_ACTUAL_PASSWORD" ``` with: ```yaml theme={null} - name: DB_PASSWORD valueFrom: secretKeyRef: name: app-secret-POD_NAME key: DB_PASSWORD ``` Do this for each sensitive variable, ensuring `key:` matches what you stored in the Secret. 5. Recreate the Pod so it uses the Secret-based env vars (any machine with kubectl access)\ For Pods managed by a higher-level controller (e.g., Deployment, StatefulSet), edit the controller instead of the live Pod. Example for a Deployment: ```bash theme={null} kubectl edit deployment DEPLOYMENT_NAME -n NAMESPACE ``` Apply the same `env:` changes to the Deployment spec, then let Kubernetes roll out new Pods.\ If it is a standalone Pod (no controller), delete and recreate it from the edited manifest: ```bash theme={null} kubectl delete pod POD_NAME -n NAMESPACE kubectl apply -f /tmp/pod-POD_NAME.yaml ``` 6. Verify the cluster is compliant (any machine with kubectl access) ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.containers // [] + .spec.initContainers // [])[] | .name as $c | (.env // [])[] | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i"))) | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1" + " container=\($c) env=\(.name) is_compliant=false" ] as $rows | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end' ``` Confirm the output is `is_compliant=true` or that no lines for the fixed Pods remain. On any machine with kubectl access: 1. Identify the violating Pod and env var (from the audit output): * Example: * Namespace: `default` * Pod name: `my-app-abc123` * Container: `app` * Offending env: `DB_PASSWORD` 2. Create a Secret that will hold the sensitive value (one-time, per app/namespace): ```bash theme={null} kubectl -n default create secret generic my-app-secret \ --from-literal=DB_PASSWORD='REPLACE_WITH_REAL_PASSWORD' ``` 3. Export the current Pod manifest, edit it to use `valueFrom.secretKeyRef`, and apply it via its controller (Deployment, StatefulSet, etc.).\ If the Pod is standalone (no controller), you must recreate it from a manifest. a) Get the owning controller kind/name from the audit output (`owner=` field) or via: ```bash theme={null} kubectl -n default get pod my-app-abc123 -o jsonpath='{.metadata.ownerReferences[0].kind}{" "}{.metadata.ownerReferences[0].name}{"\n"}' ``` Assume it is a Deployment named `my-app`. b) Export the Deployment manifest: ```bash theme={null} kubectl -n default get deploy my-app -o yaml > my-app-deploy.yaml ``` c) Edit `my-app-deploy.yaml`: Locate the container and replace the literal `value:` with `valueFrom.secretKeyRef`: ```yaml theme={null} apiVersion: apps/v1 kind: Deployment metadata: name: my-app namespace: default spec: template: spec: containers: - name: app env: - name: DB_PASSWORD valueFrom: secretKeyRef: name: my-app-secret key: DB_PASSWORD ``` Remove any previous `value: ...` line for `DB_PASSWORD`. d) Apply the updated manifest: ```bash theme={null} kubectl apply -f my-app-deploy.yaml ``` This will roll out new Pods using the Secret-based env var. 4. Verification: Run the audit command again and ensure there is no line for this Pod/env and that you see `is_compliant=true` when all violations are fixed: ```bash theme={null} kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.nodeName // "") as $node | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own | ((.spec.containers // []) + (.spec.initContainers // []))[] | .name as $c | (.env // [])[] | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i"))) | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1" + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end) + (if $node == "" then "" else " node=\($node)" end) + (if $labels == "" then "" else " labels=\($labels)" end) + (if $own == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end) + " container=\($c) env=\(.name) is_compliant=false" ] as $rows | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end' ``` ```bash theme={null} #!/usr/bin/env bash # # Remediate CBP C4.1 for Pods in AKS: # - Detect env vars with secret-like names that use a literal `value:` # - Create/patch a Secret per Pod+container # - Rewrite Pod spec to use valueFrom.secretKeyRef # # IMPORTANT: # - Runs from any machine with kubectl access and jq installed. # - Only touches namespaced Pods outside kube-system/kube-public/kube-node-lease. # - Handles immutable Pods (e.g., controlled by Deployments) by patching the controller instead. # - Safe to re-run: uses deterministic Secret names and key names; re-runs become no-ops. # # REQUIREMENTS: # - kubectl configured for the AKS cluster # - jq # # LIMITATIONS: # - Cannot recover the original secret value from Git/IaC; it uses the currently running literal # value to build a Secret. You should later rotate these Secrets from a secure source of truth. set -euo pipefail # Configurable regex for "secret-like" env var names SENSITIVE_ENV_REGEX='PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY' # Namespace ignore list (system namespaces) IGNORE_NAMESPACES=('kube-system' 'kube-public' 'kube-node-lease') # Check prerequisites command -v kubectl >/dev/null 2>&1 || { echo "kubectl is required"; exit 1; } command -v jq >/dev/null 2>&1 || { echo "jq is required"; exit 1; } is_ignored_ns() { local ns="$1" for n in "${IGNORE_NAMESPACES[@]}"; do if [[ "$n" == "$ns" ]]; then return 0 fi done return 1 } # Sanitize names for Secret/keys (DNS-1123 subdomain for Secret; key can be broader but we keep simple) sanitize_name() { # Lowercase, replace invalid chars with '-', trim leading/trailing '-' echo "$1" \ | tr '[:upper:]' '[:lower:]' \ | sed -E 's/[^a-z0-9.-]+/-/g; s/^[^a-z0-9]+//; s/[^a-z0-9]+$//' } create_or_patch_secret() { local namespace="$1" local secret_name="$2" local key="$3" local value="$4" # Check if secret exists if kubectl -n "$namespace" get secret "$secret_name" >/dev/null 2>&1; then # Check if key exists with same value (to keep idempotent) local existing existing="$(kubectl -n "$namespace" get secret "$secret_name" -o json \ | jq -r --arg k "$key" '.data[$k] // empty')" local new_b64 new_b64="$(printf '%s' "$value" | base64 -w0 2>/dev/null || printf '%s' "$value" | base64)" if [[ "$existing" == "$new_b64" ]]; then echo "Secret ${namespace}/${secret_name} already has key ${key} with same value; skip update" return fi echo "Patching Secret ${namespace}/${secret_name} to add/update key ${key}" kubectl -n "$namespace" patch secret "$secret_name" \ --type merge \ -p "$(jq -n --arg k "$key" --arg v "$new_b64" '{data: {($k): $v}}')" else echo "Creating Secret ${namespace}/${secret_name} with key ${key}" kubectl -n "$namespace" create secret generic "$secret_name" \ --from-literal "${key}=${value}" fi } patch_pod_env_to_secretref() { local namespace="$1" local pod="$2" local container="$3" local env_name="$4" local secret_name="$5" local secret_key="$6" echo "Patching Pod ${namespace}/${pod} container=${container} env=${env_name} -> secretKeyRef" # Use strategic merge patch on the Pod kubectl -n "$namespace" patch pod "$pod" --type merge -p "$( jq -n \ --arg c "$container" \ --arg en "$env_name" \ --arg sn "$secret_name" \ --arg sk "$secret_key" \ '{ spec: { containers: [ { name: $c, env: [ { name: $en, valueFrom: { secretKeyRef: { name: $sn, key: $sk } } } ] } ] } }' )" } patch_controller_env_to_secretref() { local kind="$1" # Deployment, StatefulSet, DaemonSet, Job, CronJob, ReplicaSet, ReplicationController local namespace="$2" local name="$3" local container="$4" local env_name="$5" local secret_name="$6" local secret_key="$7" echo "Patching ${kind} ${namespace}/${name} container=${container} env=${env_name} -> secretKeyRef" # Map kind to pod spec path local podspec_path case "$kind" in Deployment|StatefulSet|DaemonSet|ReplicaSet|ReplicationController) podspec_path='.spec.template.spec' ;; Job) podspec_path='.spec.template.spec' ;; CronJob) # For batch/v1 CronJob in AKS podspec_path='.spec.jobTemplate.spec.template.spec' ;; *) echo "Unsupported controller kind: $kind; skipping" >&2 return ;; esac # Build patch dynamically to target the matching container # We use a JSON patch instead of strategic to surgically replace this env entry # 1. Fetch existing pod template local tmpl tmpl="$(kubectl -n "$namespace" get "$kind" "$name" -o json)" # 2. Build new containers array with this env changed local new_containers new_containers="$(echo "$tmpl" | jq \ --arg c "$container" \ --arg en "$env_name" \ --arg sn "$secret_name" \ --arg sk "$secret_key" \ "$podspec_path.containers | map( if .name == \$c then .env |= (map( if .name == \$en then {name: \$en, valueFrom:{secretKeyRef:{name:\$sn,key:\$sk}}} else . end )) else . end )" )" # 3. Apply patch kubectl -n "$namespace" patch "$kind" "$name" --type merge -p "$( echo "$tmpl" | jq \ --argjson nc "$new_containers" \ "$podspec_path.containers = \$nc | {spec:.spec}" )" } # Main remediation echo "Discovering Pods with secret-like env vars using literal values..." pods_json="$(kubectl get pods --all-namespaces -o json)" # Iterate over Pods echo "$pods_json" | jq -c '.items[]' | while read -r pod; do ns="$(echo "$pod" | jq -r '.metadata.namespace')" if is_ignored_ns "$ns"; then continue fi pod_name="$(echo "$pod" | jq -r '.metadata.name')" # Check both containers and initContainers for field in containers initContainers; do echo "$pod" | jq -c --arg field "$field" ".spec[$field] // [] | .[]" | while read -r ctr; do container_name="$(echo "$ctr" | jq -r '.name')" echo "$ctr" | jq -c --arg re "$SENSITIVE_ENV_REGEX" '.env // [] | .[] | select((.value != null) and (.name | test($re;"i")))' | while read -r env; do env_name="$(echo "$env" | jq -r '.name')" env_val="$(echo "$env" | jq -r '.value')" # Determine controller owner (if any) owner_kind="$(echo "$pod" | jq -r '(.metadata.ownerReferences // []) | map(select(.controller)) | first?.kind // ""')" owner_name="$(echo "$pod" | jq -r '(.metadata.ownerReferences // []) | map(select(.controller)) | first?.name // ""')" # Derive Secret name and key # Secret name: secret-- (sanitized) # Key: sanitized env name secret_name_raw="secret-${pod_name}-${container_name}" secret_name="$(sanitize_name "$secret_name_raw")" secret_key="$(sanitize_name "$env_name")" if [[ -z "$secret_name" || -z "$secret_key" ]]; then echo "Failed to derive Secret name/key for ${ns}/${pod_name} env=${env_name}; skipping" >&2 continue fi create_or_patch_secret "$ns" "$secret_name" "$secret_key" "$env_val" if [[ -n "$owner_kind" && -n "$owner_name" ]]; then # Patch controller so future Pods are fixed patch_controller_env_to_secretref "$owner_kind" "$ns" "$owner_name" "$container_name" "$env_name" "$secret_name" "$secret_key" else # Patch the Pod directly (e.g., naked Pod) patch_pod_env_to_secretref "$ns" "$pod_name" "$container_name" "$env_name" "$secret_name" "$secret_key" fi done done done done echo echo "Re-running compliance check to verify..." kubectl get pods --all-namespaces -o json | jq -r ' [ .items[] | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not) | .metadata as $m | (.spec.nodeName // "") as $node | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own | ((.spec.containers // []) + (.spec.initContainers // []))[] | .name as $c | (.env // [])[] | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i"))) | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1" + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end) + (if $node == "" then "" else " node=\($node)" end) + (if $labels == "" then "" else " labels=\($labels)" end) + (if $own == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end) + " container=\($c) env=\(.name) is_compliant=false" ] as $rows | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end' echo echo "If the output above is 'is_compliant=true', the remediation is successful." ```