Secrets Should Be Encrypted At Rest

More Info:

Advisory: EncryptionConfiguration with a KMS provider should be enabled for Secret resources so etcd does not store secrets in plaintext.

Risk Level

High

Address

Security

Compliance Standards

• Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Identify the cluster and its resource group

   • Run on: any machine with Azure CLI access.
   • Command:

     az aks list -o table

     Note the name and resourceGroup of the affected cluster.

2. Check whether customer‑managed key (CMK) + disk encryption set is configured (AKS cluster‑wide encryption)

   • Run on: any machine with Azure CLI access.
   • Command:

     az aks show \
       --name <AKS_CLUSTER_NAME> \
       --resource-group <RESOURCE_GROUP> \
       --query "securityProfile.azureKeyVaultKms" -o json

   • Review:
     • If this returns null or "enabled": false, then KMS/envelope encryption for Kubernetes secrets is not enabled.
     • If "enabled": true, confirm "keyId" is set to the expected Azure Key Vault key.

3. Review current encryption design and key management requirements

   • Manually verify with your security/ops teams:
     • Which Azure Key Vault and key (URI, versioning, rotation policy) must be used for AKS secrets.
     • Required regions, RBAC, and access policies for the AKS managed identity to use that key.
     • Compliance requirements for key ownership, rotation, and logging (Key Vault logging to Log Analytics/Storage).

4. Enable or correct AKV KMS integration for the AKS cluster (envelope encryption for secrets)

   • Preconditions: an Azure Key Vault with an RSA key exists, with access granted to the AKS cluster’s managed identity.
   • Run on: any machine with Azure CLI access.
   • Commands (example pattern; replace placeholders with actual values decided in step 3):

     # Get the AKS cluster's kubelet identity (or user-assigned identity if used)
     az aks show \
       --name <AKS_CLUSTER_NAME> \
       --resource-group <RESOURCE_GROUP> \
       --query "identityProfile.kubeletidentity.clientId" -o tsv
     
     # Grant Key Vault permissions (or adjust if you use RBAC for Key Vault)
     az keyvault set-policy \
       --name <KEYVAULT_NAME> \
       --resource-group <KEYVAULT_RG> \
       --spn <AKS_MANAGED_IDENTITY_CLIENT_ID> \
       --key-permissions get unwrapKey wrapKey
     
     # Enable AKV KMS on the AKS cluster
     az aks update \
       --name <AKS_CLUSTER_NAME> \
       --resource-group <RESOURCE_GROUP> \
       --enable-azure-keyvault-kms \
       --azure-keyvault-kms-key-id "https://<KEYVAULT_NAME>.vault.azure.net/keys/<KEY_NAME>/<KEY_VERSION>" \
       --azure-keyvault-kms-key-vault-network-access public

   • Operational impact: this updates control-plane configuration; the API server is reconfigured by Azure. Existing secrets are not automatically re‑encrypted; they will be encrypted as they are written/updated.

5. Re‑write critical secrets to ensure they are encrypted with KMS

   • Run on: any machine with kubectl access.
   • For each sensitive secret, force a rewrite:

     kubectl get secret <SECRET_NAME> -n <NAMESPACE> -o yaml > /tmp/secret.yaml
     kubectl delete secret <SECRET_NAME> -n <NAMESPACE>
     kubectl apply -f /tmp/secret.yaml

   • Repeat for all high‑sensitivity secrets or redeploy them via your IaC pipelines.

6. Verify encryption configuration is active and in use

   • Configuration check (repeat step 2):

     az aks show \
       --name <AKS_CLUSTER_NAME> \
       --resource-group <RESOURCE_GROUP> \
       --query "securityProfile.azureKeyVaultKms" -o json

     Confirm "enabled": true and "keyId" matches the intended Key Vault key.
   • Functional check (indirect, since etcd is not accessible in AKS):
     • Create a test secret and ensure no plaintext appears in API server logs or client‑side tooling beyond base64 encoding:

       kubectl create secret generic kms-test-secret \
         -n default \
         --from-literal=password="VerySensitiveTestValue123!"
       kubectl get secret kms-test-secret -n default -o yaml

     • Confirm the value is only visible as base64 and that the AKS KMS configuration remains enabled as above.

Using kubectl

kubectl cannot configure envelope/KMS encryption for Secrets on AKS, because this setting is only available in the AKS control plane and Azure-side configuration. To address this finding, use the Azure portal/CLI or your IaC (such as ARM/Bicep/Terraform) as described in the Manual Steps section.

Automation

#!/usr/bin/env bash
#
# AKS Secret Encryption - Automation Check (CBP C6.2)
#
# Requirements:
# - Azure CLI logged in (`az login`)
# - kubectl configured for each cluster context you want to check
#
# This script:
# 1) Lists AKS clusters in the given subscription(s)
# 2) Reports whether AKS secret encryption is enabled and which KMS key is used
# 3) Optionally runs a basic data‑plane check via kubectl for a named context
#
# USAGE EXAMPLES:
#   Check all clusters in current subscription:
#       ./check_aks_secret_encryption.sh
#
#   Check all clusters in specific subscriptions:
#       AZ_SUBSCRIPTIONS="subid1 subid2" ./check_aks_secret_encryption.sh
#
#   Also run a kubectl check for a specific context:
#       KUBECTL_CONTEXT="my-aks-context" ./check_aks_secret_encryption.sh
#
# INTERPRETING OUTPUT:
#   - Problem / needs review:
#       encryptionAtHost: false    (for node OS disks; advisory, not etcd)
#       enableRbac: false          (separate issue, not this control)
#       encryption:
#         enabled: false           (AKS secret encryption NOT enabled)
#       encryptionProfile: null    (no encryption profile)
#     or:
#       encryption:
#         keyVaultProperties: {}   (missing key or key URL)
#
#   - Desired state for this control:
#       encryption:
#         keyVaultProperties:
#           keyIdentifier: https://<kv-name>.vault.azure.net/keys/<key-name>/<version>
#
#   Data‑plane kubectl probe (optional):
#     This *cannot* confirm KMS, but helps detect obviously plaintext secrets
#     if etcd were ever directly accessible (normally not in AKS).
#

set -euo pipefail

# Colors for readability
RED="$(printf '\033[0;31m')"
GREEN="$(printf '\033[0;32m')"
YELLOW="$(printf '\033[0;33m')"
NC="$(printf '\033[0m')" # no color

# Get list of subscriptions to check
if [[ -n "${AZ_SUBSCRIPTIONS:-}" ]]; then
  SUBSCRIPTIONS=${AZ_SUBSCRIPTIONS}
else
  SUBSCRIPTIONS=$(az account list --query '[].id' -o tsv)
fi

if [[ -z "${SUBSCRIPTIONS}" ]]; then
  echo "No Azure subscriptions found. Run 'az login' and try again." >&2
  exit 1
fi

echo "Checking AKS secret encryption configuration in subscriptions:"
echo "${SUBSCRIPTIONS}" | sed 's/^/  - /'

for SUB in ${SUBSCRIPTIONS}; do
  echo
  echo "=================================================================="
  echo "Subscription: ${SUB}"
  echo "=================================================================="

  az account set -s "${SUB}"

  # List all AKS clusters in this subscription
  CLUSTERS_JSON=$(az aks list -o json)
  CLUSTER_COUNT=$(echo "${CLUSTERS_JSON}" | jq 'length')

  if [[ "${CLUSTER_COUNT}" -eq 0 ]]; then
    echo "No AKS clusters found in this subscription."
    continue
  fi

  echo "Found ${CLUSTER_COUNT} AKS cluster(s)."

  # Iterate clusters
  echo "${CLUSTERS_JSON}" | jq -c '.[]' | while read -r CL; do
    NAME=$(echo "${CL}" | jq -r '.name')
    RG=$(echo "${CL}"   | jq -r '.resourceGroup')
    LOCATION=$(echo "${CL}" | jq -r '.location')

    echo
    echo "------------------------------------------------------------------"
    echo "Cluster: ${NAME}"
    echo "Resource Group: ${RG}"
    echo "Location: ${LOCATION}"
    echo "------------------------------------------------------------------"

    # Get full cluster details including encryption profile
    AKS_DETAIL=$(az aks show -g "${RG}" -n "${NAME}" -o json)

    # Extract encryption profile (managed control-plane secret encryption)
    ENCRYPTION_PROFILE=$(echo "${AKS_DETAIL}" | jq '.encryptionProfile // .securityProfile.encryptionAtHost // null')

    # Newer AKS API (recommended): .encryptionProfile
    # Older / alternative fields may not exist. We'll try both patterns.

    # Attempt to read top-level encryption configuration
    # Newer API: .encryptionProfile
    HAS_ENCRYPTION_PROFILE=$(echo "${AKS_DETAIL}" | jq 'has("encryptionProfile")')
    if [[ "${HAS_ENCRYPTION_PROFILE}" == "true" ]]; then
      echo "Raw encryptionProfile:"
      echo "${AKS_DETAIL}" | jq '.encryptionProfile'
    else
      echo "No 'encryptionProfile' field present in this AKS cluster (API version may differ)."
    fi

    # Look for keyVaultProperties if present
    KEY_ID=$(echo "${AKS_DETAIL}" | jq -r '.encryptionProfile.keyVaultProperties.keyIdentifier // empty')
    if [[ -n "${KEY_ID}" ]]; then
      echo "${GREEN}Secret encryption with customer-managed key appears ENABLED.${NC}"
      echo "Key Vault Key Identifier: ${KEY_ID}"
    else
      echo "${RED}Secret encryption with customer-managed key appears NOT configured or not visible in this API version.${NC}"
      echo "This requires manual review in the Azure Portal or IaC definitions."
    fi

    echo
    echo "Summary (for this cluster):"
    echo "  - encryptionProfile present: ${HAS_ENCRYPTION_PROFILE}"
    echo "  - keyVaultProperties.keyIdentifier: ${KEY_ID:-<none>}"

  done
done

# Optional: basic kubectl-based probe for one context (informational only)
if [[ -n "${KUBECTL_CONTEXT:-}" ]]; then
  echo
  echo "=================================================================="
  echo "Optional kubectl probe for context: ${KUBECTL_CONTEXT}"
  echo "=================================================================="
  echo "NOTE: This does NOT prove KMS/etcd encryption state; it only inspects"
  echo "      Kubernetes API behavior and ensures secrets are not obviously"
  echo "      exposed in unexpected ways."
  echo

  # Machine: any machine with kubectl access
  set +e
  kubectl --context "${KUBECTL_CONTEXT}" get ns >/dev/null 2>&1
  if [[ $? -ne 0 ]]; then
    echo "${RED}kubectl cannot access context '${KUBECTL_CONTEXT}'. Skipping kubectl probe.${NC}"
    exit 0
  fi
  set -e

  TEST_NS="encryption-check-$(date +%s)"
  echo "Creating test namespace: ${TEST_NS}"
  kubectl --context "${KUBECTL_CONTEXT}" create namespace "${TEST_NS}"

  echo "Creating a test secret in ${TEST_NS}..."
  kubectl --context "${KUBECTL_CONTEXT}" -n "${TEST_NS}" create secret generic kms-test-secret \
    --from-literal=password='CorrectHorseBatteryStaple123!' >/dev/null

  echo "Reading the secret via API to ensure it is base64-encoded (expected):"
  kubectl --context "${KUBECTL_CONTEXT}" -n "${TEST_NS}" get secret kms-test-secret -o yaml

  echo
  echo "Deleting test resources..."
  kubectl --context "${KUBECTL_CONTEXT}" delete ns "${TEST_NS}" --wait=true

  echo
  echo "Interpretation of kubectl probe:"
  echo "  - Seeing base64-encoded data in the Secret is NORMAL and does NOT"
  echo "    indicate whether etcd encryption is enabled."
  echo "  - To verify KMS-based etcd encryption, rely on AKS control-plane"
  echo "    configuration (portal/CLI/IaC) as reported above, not kubectl."
fi