Prefer Secrets Files Over Secrets Environment Variables
More Info:
Kubernetes supports mounting secrets as data volumes or as environment variables. Minimize the use of environment variable secrets.
Risk Level
High
Address
Security
Compliance Standards
- CIS Kubernetes
Triage and Remediation
- Remediation
Remediation
Manual Steps
-
List all Secrets and identify likely sensitive ones
Run on any machine withkubectlaccess:kubectl get secrets -ANote Secrets that likely contain credentials or keys (names including
password,token,key,cert,secret, etc.) for deeper review in the next steps. -
Identify Pods that consume Secrets as environment variables
Run:kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{range .spec.containers[*].env[*]}{" ENV: "}{.name}{" fromSecret="}{.valueFrom.secretKeyRef.name}{"\n"}{end}{end}' 2>/dev/null | grep 'fromSecret='This shows Pods/containers with
enventries sourced from Secrets. Capture the namespace, pod, container, env var name, and secret name. -
Identify Pods that use Secrets as files (preferred)
Run:kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{range .spec.volumes[*]}{" VOL: "}{.name}{" secret="}{.secret.secretName}{"\n"}{end}{end}' 2>/dev/null | grep 'secret='This shows Pods that mount Secrets as volumes. Use this as a reference “good pattern” when planning changes away from env var usage.
-
Review affected Deployments/Workloads and application behavior
For one affected Pod from step 2, find and inspect its controller (e.g., Deployment):# Example for a Deploymentkubectl get deploy -n NAMESPACEkubectl get deploy DEPLOYMENT_NAME -n NAMESPACE -o yaml > deployment-inspect.yamlIn
deployment-inspect.yaml, locate containers usingenv.valueFrom.secretKeyRef. Assess with the application owner whether the app can be modified to read these values from files (mounted Secret volume) instead of environment variables. -
Plan and implement manifest changes to use Secret volumes instead of env vars
For each agreed change:- Edit the workload manifest (Deployment/StatefulSet/Job, etc.) to:
- Add a
volumesentry referencing the Secret:volumes:- name: app-secretsecret:secretName: SECRET_NAME - Mount it into the container:
volumeMounts:- name: app-secretmountPath: /var/run/secrets/appreadOnly: true
- Remove or phase out
enventries that reference the same Secret, once the application has been updated to read from/var/run/secrets/app/.... Apply the updated manifest from any machine withkubectl:
- Add a
kubectl apply -f UPDATED_MANIFEST.yaml - Edit the workload manifest (Deployment/StatefulSet/Job, etc.) to:
-
Verify reduced use of secrets in environment variables
After changes roll out, re-run the environment-variable usage check:kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{range .spec.containers[*].env[*]}{" ENV: "}{.name}{" fromSecret="}{.valueFrom.secretKeyRef.name}{"\n"}{end}{end}' 2>/dev/null | grep 'fromSecret=' || echo "No secrets used as env vars found"Confirm that pods for the updated workloads no longer appear, and that the corresponding pods show Secret volume mounts (step 3) instead.
Using kubectl
Using kubectl
1. List all pods that use environment variables from Secrets
Run on: any machine with kubectl access.
kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{range .spec.containers[*]}{" container: "}{.name}{"\n"}{range .env[*]}{" env: "}{.name}{" fromSecret="}{.valueFrom.secretKeyRef.name}{"\n"}{end}{range .envFrom[*]}{" envFrom: secretRef="}{.secretRef.name}{"\n"}{end}{end}{"\n"}{end}'
Indicates a problem:
Any env: or envFrom: line where fromSecret= or secretRef= is non-empty shows a container using a Secret via environment variables. These are candidates to convert to file-based mounts.
2. Identify workload types using Secret env vars (Deployments, StatefulSets, etc.)
kubectl get deploy,statefulset,daemonset,job,cronjob -A -o json | \
jq -r '
.items[] |
{
kind: .kind,
ns: .metadata.namespace,
name: .metadata.name,
containers: (
[.spec.template.spec.containers[]? |
{
name,
env: [.env[]? | select(.valueFrom.secretKeyRef != null)],
envFrom: [.envFrom[]? | select(.secretRef != null)]
}
] // []
)
} |
select([.containers[]?.env[], .containers[]?.envFrom[]] | length > 0) |
"KIND=\(.kind) NS=\(.ns) NAME=\(.name)\n" +
(
.containers[] |
" container: \(.name)\n" +
( .env[]? | " env: \(.name) fromSecret=\(.valueFrom.secretKeyRef.name)\n") +
( .envFrom[]? | " envFrom: secretRef=\(.secretRef.name)\n")
)'
Indicates a problem:
Any listed workload has at least one container using a Secret via env or envFrom. These should be reviewed and, where feasible, changed to use volume-mounted Secrets.
3. Compare with existing Secret volume mounts (to see if an alternative already exists)
kubectl get deploy,statefulset,daemonset,job,cronjob -A -o json | \
jq -r '
.items[] |
{
kind: .kind,
ns: .metadata.namespace,
name: .metadata.name,
containers: [.spec.template.spec.containers[]?],
volumes: [.spec.template.spec.volumes[]? | select(.secret != null)]
} |
select((.containers | length) > 0) |
"KIND=\(.kind) NS=\(.ns) NAME=\(.name)\n" +
(
.volumes[]? |
" volumeSecret: name=\(.name) secret=\(.secret.secretName)\n"
) +
(
.containers[] |
" container: \(.name)\n" +
( .env[]? | select(.valueFrom.secretKeyRef != null) |
" env: \(.name) fromSecret=\(.valueFrom.secretKeyRef.name)\n"
) +
( .envFrom[]? | select(.secretRef != null) |
" envFrom: secretRef=\(.secretRef.name)\n"
)
)'
Indicates a problem/opportunity:
If a workload shows both volumeSecret: and env/envFrom lines referring to the same Secret, it is already mounting the Secret as a file but still using it via env vars. This is a strong candidate to update the application code/config to read from the mounted file instead of the env var.
4. Focus on a specific namespace (optional, for detailed review)
NAMESPACE=default
kubectl get pods -n "$NAMESPACE" -o yaml | \
yq '.items[] |
{
kind: "Pod",
ns: .metadata.namespace,
name: .metadata.name,
containers: .spec.containers
}'
Manually review each container’s env and envFrom sections for secretKeyRef/secretRef usage versus any volumes and volumeMounts using secret:.
Indicates a problem:
Containers that consume Secrets only via env/envFrom and have no Secret volume mounts, or use both but still rely on env vars.
5. Verification step after any changes
After you refactor workloads to use file-based Secrets, re-run:
kubectl get deploy,statefulset,daemonset,job,cronjob -A -o json | \
jq -r '
.items[] |
{
kind: .kind,
ns: .metadata.namespace,
name: .metadata.name,
containers: (
[.spec.template.spec.containers[]? |
{
name,
env: [.env[]? | select(.valueFrom.secretKeyRef != null)],
envFrom: [.envFrom[]? | select(.secretRef != null)]
}
] // []
)
} |
select([.containers[]?.env[], .containers[]?.envFrom[]] | length > 0) |
"KIND=\(.kind) NS=\(.ns) NAME=\(.name)\n"'
Desired outcome:
The output is empty or only includes workloads that you have consciously decided must continue using Secret environment variables (document the justification).
Automation
#!/usr/bin/env bash
#
# Report deployments/statefulsets/daemonsets with env vars sourced from Secrets.
# Run on any machine with kubectl access and a current context.
#
# Requires: kubectl, jq
set -euo pipefail
if ! command -v kubectl >/dev/null 2>&1; then
echo "kubectl not found in PATH" >&2
exit 1
fi
if ! command -v jq >/dev/null 2>&1; then
echo "jq not found in PATH" >&2
exit 1
fi
echo "Scanning cluster for Secret-based environment variables..."
echo
# Common JSON snippet to extract env and envFrom secret references
read -r -d '' SECRET_ENV_JQ <<'EOF' || true
[
.spec.template.spec.containers[]? as $c
| {
container: $c.name,
env: (
($c.env // [])
| map(select(.valueFrom != null and .valueFrom.secretKeyRef != null)
| { name, secretKeyRef: .valueFrom.secretKeyRef })
),
envFrom: (
($c.envFrom // [])
| map(select(.secretRef != null)
| { secretRef: .secretRef })
)
}
]
| map(select((.env | length) > 0 or (.envFrom | length) > 0))
EOF
scan_kind() {
local kind="$1"
echo "=== ${kind}s with Secret-based environment variables ==="
# List all objects of this kind in all namespaces with env/envFrom from secrets
kubectl get "$kind" --all-namespaces -o json \
| jq -r --arg kind "$kind" '
.items[]
| .metadata as $m
| . as $obj
| '"$SECRET_ENV_JQ"' as $containers
| select($containers | length > 0)
| {
kind: $kind,
namespace: $m.namespace,
name: $m.name,
containers: $containers
}
' \
| jq -r '
# Pretty output
"NAMESPACE: \(.namespace)\nKIND: \(.kind)\nNAME: \(.name)\n"
+
( .containers[]
| " CONTAINER: \(.container)\n"
+ (if (.env | length) > 0 then
" env (from Secrets):\n"
+ ( .env[]
| " - name: \(.name) secret: \(.secretKeyRef.name) key: \(.secretKeyRef.key)"
| join("\n")
) + "\n"
else "" end)
+ (if (.envFrom | length) > 0 then
" envFrom (SecretRefs):\n"
+ ( .envFrom[]
| " - secret: \(.secretRef.name // "N/A") optional: \(.secretRef.optional // false)"
| join("\n")
) + "\n"
else "" end)
)
+ "\n-----------------------------\n"
' || echo " (none found or error parsing)"
echo
}
scan_kind deployment
scan_kind statefulset
scan_kind daemonset
echo "Scan complete."
cat <<'EOT'
Interpretation:
- Any listed resource is using Secrets as environment variables (env or envFrom).
- These are CANDIDATES for remediation under CIS 5.4.1.
Specifically:
- Lines under "env (from Secrets)" show individual env vars populated from Secret keys.
- Lines under "envFrom (SecretRefs)" show containers where all keys from a Secret
become environment variables.
Potential issues to review:
- Workloads where highly sensitive data (e.g., DB passwords, API keys, tokens)
is exposed as environment variables instead of mounted Secret files.
- Broad "envFrom" usage that automatically exposes many Secret keys as env vars.
Next steps (manual, per workload):
- For each candidate, decide whether Secrets can be consumed from mounted files
instead of env vars.
- Update application code and manifests accordingly where feasible.
EOT