More Info:
Storing secrets in an external, dedicated secrets manager can improve protection over native Kubernetes Secrets. Evaluate such options.Risk Level
InformationalAddress
SecurityCompliance Standards
- CIS Kubernetes
Triage and Remediation
- Remediation
Remediation
Manual Steps
Manual Steps
-
Inventory how Kubernetes Secrets are currently used
- On any machine with kubectl access, list all Secret objects and their types:
- Identify workloads that mount or reference these Secrets:
- Note any use of type
kubernetes.io/basic-auth,kubernetes.io/ssh-auth, database creds, API keys, tokens, etc., which are strong candidates for external storage.
- On any machine with kubectl access, list all Secret objects and their types:
-
Assess in-cluster Secret protection and sensitivity
- Check if Encryption at Rest is enabled for Secrets (control plane–specific; if you do not manage it, retrieve documentation/config from your provider or IaC):
- For self-managed clusters, on every control plane node:
- For self-managed clusters, on every control plane node:
- Classify secrets by sensitivity (e.g., “high” for long-lived credentials to production systems, “medium” for non-production, “low” for ephemeral tokens) and record which namespaces and apps depend on them.
- Check if Encryption at Rest is enabled for Secrets (control plane–specific; if you do not manage it, retrieve documentation/config from your provider or IaC):
-
Review available external secrets managers and cluster integration options
- From your cloud console or IaC, identify supported secrets managers and their Kubernetes integrations, for example (depending on environment):
- AWS: AWS Secrets Manager / SSM Parameter Store + External Secrets Operator or Secrets Store CSI Driver.
- GCP: Secret Manager + Secret Manager CSI driver / external-secrets.
- Azure: Key Vault + Secrets Store CSI Driver / external-secrets.
- Confirm organizational standards or regulatory requirements that may mandate a particular secrets manager or key management scheme (e.g., use of HSM-backed keys, centralized audit, rotation policies).
- From your cloud console or IaC, identify supported secrets managers and their Kubernetes integrations, for example (depending on environment):
-
Decide scope and pattern for migration to external secret storage
- Choose a preferred integration pattern (e.g., External Secrets Operator with CRDs like
ExternalSecret, or CSI driver volume mounts) based on your platform. - Select a pilot scope: one or a few high-sensitivity applications and their Secrets (from step 1) to migrate first.
- For those pilot apps, map each existing
Secretto a candidate external secret path/name in the chosen manager.
- Choose a preferred integration pattern (e.g., External Secrets Operator with CRDs like
-
Implement and test external secrets for a pilot workload
- Using cloud console or IaC, create corresponding secrets in the external manager for the pilot application, ensuring:
- Access control (IAM/RBAC) only allows the Kubernetes cluster/namespace/service account to read them.
- Rotation policy is defined where supported.
- On any machine with kubectl access, deploy the chosen integration (operator or CSI driver) according to its vendor documentation, then define a test manifest that consumes the external secret (example pattern only, adjust to your tool):
- Verify the pod(s) receive correct secret data and the original Kubernetes
Secretis no longer required for that workload.
- Using cloud console or IaC, create corresponding secrets in the external manager for the pilot application, ensuring:
-
Plan progressive rollout and deprecation of native Secrets where appropriate
- For remaining high- and medium-sensitivity Secrets, repeat the mapping and migration process, updating manifests in your IaC to source data from the external store instead of embedding values into Kubernetes Secrets.
- Verify after each migration that no workloads still depend on the old Secrets:
- Once confirmed, delete or minimize high-sensitivity native Secrets and document the new standard: which classes of secrets must be stored in the external manager, and which (if any) may remain as Kubernetes Secrets.
Using kubectl
Using kubectl
type: Opaquesecrets containing:- Long-lived application credentials (database users/passwords, API keys, TLS private keys not associated with short-lived certs).
- Cloud provider access keys, tokens, or passwords to external services.
- Secrets with generic names but clearly sensitive values, for example:
db-credentials,payment-gateway-key,smtp-password,aws-credentials,gcp-service-account,azure-sp-credentials.
- Secrets referenced broadly across many workloads (suggesting high blast radius).
env:orenvFrom:entries that reference sensitive secrets viavalueFrom.secretKeyRef.volumes:withsecret:sources that mount sensitive secrets to many pods.- Same secret name used across many different applications/namespaces (central, critical credential).
- Long-lived,
- Grant access to external or critical systems,
- Needed by many different workloads,
- TLS private keys or CA roots that are long-lived and reused across multiple services or ingresses; those are often better managed in a dedicated secrets/PKI system.
- Based on the above inspection, decide which secrets should remain as native Kubernetes Secrets (low-risk, internal-only, non-critical), and which should instead be sourced from an external secrets manager per your cloud provider or chosen third-party solution.
Automation
Automation
- Large counts of non-service-account Secrets in critical namespaces (e.g.,
default, app namespaces) suggest widespread storage of app credentials in native Secrets. - Many pods showing populated
ENV_SECRETREFS,ENVFROM_SECRETREFS, orVOLUME_SECRETScolumns mean extensive direct use of Kubernetes Secrets instead of an external manager. - ConfigMaps listed in the “ConfigMaps that embed data…” section likely contain secrets in plain text and are strong candidates for migration to an external secrets manager.
- Absence of any external-secrets-related CRDs or controllers plus heavy native secret use suggests the cluster relies solely on native Secrets and should be evaluated for adoption of an external secrets manager.

