Skip to main content

Sensitive Values Should Not Be Passed As Literal Env Vars

More Info:​

Verifies secret-like env vars are not set as literal values. Literal values land in the pod manifest, logs and kubectl describe.

Risk Level​

Medium

Address​

Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Manual Steps

  1. List offending Pods and identify the owning workload (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
        | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
        | .metadata as $m
        | (.spec.nodeName // "") as $node
        | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
        | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
        | ((.spec.containers // []) + (.spec.initContainers // []))[]
        | .name as $c
        | (.env // [])[]
        | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
        | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
          + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
          + (if $node   == ""   then "" else " node=\($node)" end)
          + (if $labels == ""   then "" else " labels=\($labels)" end)
          + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
          + " container=\($c) env=\(.name) is_compliant=false"
      ] as $rows
      | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

    From each line, note ns=..., owner=... (e.g., Deployment/default/my-app/...), container=..., and env=....

  2. For each offending env var, create or update a Secret holding the sensitive value (run on any machine with kubectl access, per namespace). Replace placeholders with the real values:

    kubectl -n default create secret generic my-app-secret \
      --from-literal=DB_PASSWORD='actual-password-value' \
      --dry-run=client -o yaml > /tmp/my-app-secret.yaml

    kubectl apply -f /tmp/my-app-secret.yaml

    Use one key per env var (e.g., DB_PASSWORD, API_TOKEN).

  3. Patch the owning workload manifest to use valueFrom.secretKeyRef instead of a literal value: (run on any machine with kubectl access). First, export the current manifest:

    kubectl -n default get deployment my-app -o yaml > /tmp/my-app-deploy.yaml

    Then edit /tmp/my-app-deploy.yaml:

    • Find the container with the offending env, e.g.: 
      env:
        - name: DB_PASSWORD
          value: "actual-password-value"
    • Replace with: 
      env:
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: my-app-secret
              key: DB_PASSWORD
    • Ensure you do this for every env var flagged by the audit.

  4. Apply the updated workload manifest so new Pods use the Secret (run on any machine with kubectl access):

    kubectl apply -f /tmp/my-app-deploy.yaml

    For controllers that don’t roll Pods automatically (e.g., some StatefulSets/Jobs), trigger a restart as appropriate, understanding this will restart the affected Pods.

  5. Confirm new Pods no longer contain literal sensitive values in their env definition (run on any machine with kubectl access):

    kubectl -n default get pods -l app=my-app -o yaml | \
      grep -A2 "name: DB_PASSWORD"

    You should see only a valueFrom: secretKeyRef: block and no value: line for the sensitive variable names.

  6. Re-run the benchmark audit to verify compliance (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
      [ .items[]
        | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
        | .metadata as $m
        | (.spec.nodeName // "") as $node
        | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
        | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
        | ((.spec.containers // []) + (.spec.initContainers // []))[]
        | .name as $c
        | (.env // [])[]
        | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
        | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
          + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
          + (if $node   == ""   then "" else " node=\($node)" end)
          + (if $labels == ""   then "" else " labels=\($labels)" end)
          + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
          + " container=\($c) env=\(.name) is_compliant=false"
      ] as $rows
      | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

    The cluster is compliant when the output shows is_compliant=true and no is_compliant=false rows.

Using kubectl

On any machine with kubectl access to the cluster:

  1. Identify the offending Pod and env var from the finding output, for example:

    • Namespace: app-namespace
    • Pod name: web-7c9c6c7d4b-abcde
    • Container: web
    • Env var: DB_PASSWORD

  2. Extract the current Pod spec (for reference only; do not apply this Pod directly because it is usually managed by a controller such as a Deployment):

kubectl get pod web-7c9c6c7d4b-abcde -n app-namespace -o yaml > /tmp/web-pod.yaml
  1. Determine the owning controller (e.g., Deployment) from the owner= field in the finding or via:
kubectl get pod web-7c9c6c7d4b-abcde -n app-namespace -o jsonpath='{.metadata.ownerReferences[0].kind}{" "}{.metadata.ownerReferences[0].name}{"\n"}'

Assume the result is Deployment web.

  1. Create a Secret that will hold the sensitive value (if one does not already exist). Replace the literal with the real value:
kubectl -n app-namespace create secret generic db-credentials \
  --from-literal=DB_PASSWORD='REPLACE_WITH_REAL_PASSWORD'
  1. Patch the owning Deployment to reference the Secret via valueFrom.secretKeyRef instead of a literal value:. First, fetch the Deployment manifest:
kubectl get deployment web -n app-namespace -o yaml > /tmp/web-deploy.yaml
  1. Edit /tmp/web-deploy.yaml:
    • In the relevant container under spec.template.spec.containers[], find the existing env entry:
        env:
          - name: DB_PASSWORD
            value: "REPLACE_WITH_REAL_PASSWORD"
  • Replace it with:
        env:
          - name: DB_PASSWORD
            valueFrom:
              secretKeyRef:
                name: db-credentials
                key: DB_PASSWORD
  1. Apply the updated Deployment manifest:
kubectl apply -f /tmp/web-deploy.yaml

This will recreate Pods managed by the Deployment with the Secret-based env var.

  1. Verification: rerun the audit (or a scoped variant) and confirm no is_compliant=false rows remain for this Pod/container/env:
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null) and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
      + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
      + (if $node   == ""   then "" else " node=\($node)" end)
      + (if $labels == ""   then "" else " labels=\($labels)" end)
      + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
      + " container=\($c) env=\(.name) is_compliant=false"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'
Automation
#!/usr/bin/env bash
#
# Fix CBP C4.1: Sensitive values should not be passed as literal env vars
#
# This script:
#   - Scans all non-system namespaces for Pods with secret-like env var NAMES
#     (PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY) that have
#     literal values set via `env[].value`.
#   - Emits guidance and manifests you can use to convert them to Secret
#     references (`valueFrom.secretKeyRef`).
#
# IMPORTANT:
#   - There is no fully automatic, safe way to invent Secrets and wire them
#     into apps without knowing the correct values and ownership model.
#   - This script is NON-DESTRUCTIVE: it never deletes or patches live Pods.
#   - Use the generated manifests and commands as a starting point, then
#     review and apply manually.
#
# Requirements (run on any machine with kubectl access):
#   - kubectl
#   - jq
#
# Re-runnable: it only re-generates report files and guidance.

set -euo pipefail

WORKDIR="${PWD}/cbp-c4-1-remediation"
REPORT="${WORKDIR}/literal-secret-envvars.jsonl"
GUIDE="${WORKDIR}/literal-secret-envvars-remediation.txt"

mkdir -p "${WORKDIR}"

echo "Scanning cluster for Pods with sensitive literal env vars..."
kubectl get pods --all-namespaces -o json | jq -r '
  [
    .items[]
    | select(.metadata.namespace as $n
             | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null)
             and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | {
        kind:       "Pod",
        namespace:  $m.namespace,
        pod:        $m.name,
        uid:        $m.uid,
        apiVersion: "v1",
        created:    ($m.creationTimestamp // ""),
        node:       $node,
        labels:     $labels,
        ownerKind:  ($own.kind // ""),
        ownerName:  ($own.name // ""),
        ownerUID:   ($own.uid // ""),
        container:  $c,
        envName:    .name,
        # DO NOT write the env value to disk, just record that it exists
        hasLiteralValue: true
      }
  ] as $rows
  | if ($rows | length) == 0
    then "[]"    # empty JSON array
    else $rows
    end
' > "${REPORT}"

if [ "$(jq 'length' "${REPORT}")" -eq 0 ]; then
  echo "No Pods with sensitive literal env vars found. Cluster appears compliant."
  echo "is_compliant=true"
  exit 0
fi

echo "Found $(jq 'length' "${REPORT}") offending env vars. Writing remediation guidance to:"
echo "  - ${REPORT} (structured JSONL-style array)"
echo "  - ${GUIDE}  (human-readable guidance)"

cat > "${GUIDE}" <<'EOF'
CBP C4.1 REMEDIATION GUIDE
==========================

Sensitive values must NOT be passed as literal env vars via `env[].value`.
Instead, use:

  - `env[].valueFrom.secretKeyRef`  (recommended), or
  - a mounted Secret volume.

For every row in literal-secret-envvars.jsonl:

1. Identify the owning controller (Deployment/StatefulSet/Job/etc.) using:
   - fields ownerKind, ownerName, namespace in the JSON.

2. On a machine with kubectl access, inspect the owning object, for example:
   - kubectl -n <namespace> get deployment <ownerName> -o yaml

3. For each container/envName pair:
   a. Retrieve the CURRENT value from a live Pod (one-time, to seed a Secret):

        POD="<pod_name_from_report>"
        NS="<namespace_from_report>"
        kubectl -n "${NS}" get pod "${POD}" -o json \
          | jq -r '
              (.spec.containers[] + (.spec.initContainers // []))[]
              | select(.name=="<container_from_report>")
              | .env[]
              | select(.name=="<envName_from_report>")
              | .value
            '

      NOTE: Run this only long enough to bootstrap the Secret, then delete
            any local copies. Treat the output as sensitive.

   b. Create (or reuse) a Secret to hold this key. Example:

        NS="<namespace_from_report>"
        SECRET_NAME="<app>-sensitive-env"
        kubectl -n "${NS}" create secret generic "${SECRET_NAME}" \
          --from-literal="<envName_from_report>=<VALUE_FROM_STEP_3A>"

      If the Secret already exists, update it instead:

        kubectl -n "${NS}" create secret generic "${SECRET_NAME}" \
          --from-literal="<envName_from_report>=<VALUE_FROM_STEP_3A>" \
          --dry-run=client -o yaml | kubectl apply -f -

   c. Edit the owning controller to replace `value:` with `valueFrom:`:

        kubectl -n <namespace_from_report> edit <ownerkind_from_report>/<ownerName_from_report>

      In the relevant container's env section, change:

        - name: <envName_from_report>
          value: "<literal_value>"

      TO:

        - name: <envName_from_report>
          valueFrom:
            secretKeyRef:
              name: <SECRET_NAME>
              key: <envName_from_report>

      Save and exit. The controller will roll out new Pods using the Secret.

4. Pods not managed by a controller (bare Pods):
   - Prefer to replace them with a Deployment/Job and follow the same pattern.
   - If you must keep a bare Pod, update its manifest in your IaC / manifests
     to use `valueFrom.secretKeyRef` and recreate the Pod.

5. Post-change verification for each namespace:

   - Re-run the audit:

        kubectl get pods --all-namespaces -o json | jq -r '
          [ .items[]
            | select(.metadata.namespace as $n
                     | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
            | .metadata as $m
            | (.spec.nodeName // "") as $node
            | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
            | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
            | ((.spec.containers // []) + (.spec.initContainers // []))[]
            | .name as $c
            | (.env // [])[]
            | select((.value != null)
                     and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
            | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
              + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
              + (if $node   == ""   then "" else " node=\($node)" end)
              + (if $labels == ""   then "" else " labels=\($labels)" end)
              + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
              + " container=\($c) env=\(.name) is_compliant=false"
          ] as $rows
          | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end
        '

   - You should ultimately see only:

        is_compliant=true

EOF

echo
echo "Next steps:"
echo "  1) Review ${REPORT} to see each violating Pod/container/envName."
echo "       jq -C . ${REPORT} | less -R"
echo "  2) Follow the step-by-step instructions in:"
echo "       ${GUIDE}"
echo
echo "When you have updated the owning controllers to use valueFrom.secretKeyRef,"
echo "re-run this script or the audit command to confirm you get: is_compliant=true"

echo
echo "Running verification audit now (read-only):"
kubectl get pods --all-namespaces -o json | jq -r '
  [ .items[]
    | select(.metadata.namespace as $n
             | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | (.spec.nodeName // "") as $node
    | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | ((.spec.containers // []) + (.spec.initContainers // []))[]
    | .name as $c
    | (.env // [])[]
    | select((.value != null)
             and (.name | test("PASSWORD|SECRET|TOKEN|APIKEY|API_KEY|CREDENTIAL|PRIVATE_KEY"; "i")))
    | "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
      + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
      + (if $node   == ""   then "" else " node=\($node)" end)
      + (if $labels == ""   then "" else " labels=\($labels)" end)
      + (if $own    == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
      + " container=\($c) env=\(.name) is_compliant=false"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end
'