Encrypting Kubernetes Secrets At Rest In Etcd

More Info:

Encrypt Kubernetes secrets at rest in etcd using a master encryption key from the OCI Vault (KMS) service. Cross-tenancy access requires policies allowing the cluster to use the key.

Risk Level

High

Address

Security

Compliance Standards

CIS OKE

Triage and Remediation

Remediation

Remediation

Manual Steps

Confirm cluster KMS key configuration (ClusterTenancy)
- On any machine with oci configured for the ClusterTenancy, run:
```
oci ce cluster list --compartment-id <CLUSTER_TENANCY_COMPARTMENT_OCID> \
  --all \
  --query "data[].{name:name,id:id,\"kms-key-id\":\"options.kms-key-id\"}" \
  --output table
```
- For each relevant cluster, check whether kms-key-id is set. If it is empty, secrets are not encrypted with OCI Vault.

If kms-key-id is unset, identify / create a Vault key (KeyTenancy)

In the KeyTenancy (or same tenancy if not cross‑tenancy), list candidate keys:

oci kms management key list \
  --compartment-id <KEY_TENANCY_VAULT_COMPARTMENT_OCID> \
  --endpoint <KMS_MANAGEMENT_ENDPOINT> \
  --query "data[].{name:\"display-name\",id:id,lifecycle_state:\"lifecycle-state\"}" \
  --output table

Choose an id of a suitable, enabled key (ocid1.key.oc1...) to use as the master encryption key.

Review / create cross‑tenancy policies in KeyTenancy (if using a key from another tenancy)

In KeyTenancy, collect OCIDs for: ClusterTenancy, OKEAdminGroup, OKEAdminDynGroup, and any dynamic group representing clusters:
```
oci iam tenancy get
oci iam group list --all
oci iam dynamic-group list --all
```

In the KeyTenancy root compartment, create or review a policy with statements adapted to your OCIDs, e.g.:

Define tenancy OKE_Tenancy as <CLUSTER_TENANCY_OCID>
Define dynamic-group RemoteOKEClusterDynGroup as <OKE_ADMIN_DYN_GROUP_OCID>
Define group RemoteOKEAdminGroup as <OKE_ADMIN_GROUP_OCID>
Admit dynamic-group RemoteOKEClusterDynGroup of tenancy OKE_Tenancy to use keys in tenancy where target.key.id = '<KEY_OCID>'
Admit group RemoteOKEAdminGroup of tenancy OKE_Tenancy to use keys in tenancy where target.key.id = '<KEY_OCID>'

Apply via Console or, from KeyTenancy, with something like:

oci iam policy create \
  --name "oke-cross-kms-keytenancy" \
  --compartment-id <KEY_TENANCY_ROOT_COMPARTMENT_OCID> \
  --statements "[\"Define tenancy OKE_Tenancy as <CLUSTER_TENANCY_OCID>\",
  \"Define dynamic-group RemoteOKEClusterDynGroup as <OKE_ADMIN_DYN_GROUP_OCID>\",
  \"Define group RemoteOKEAdminGroup as <OKE_ADMIN_GROUP_OCID>\",
  \"Admit dynamic-group RemoteOKEClusterDynGroup of tenancy OKE_Tenancy to use keys in tenancy where target.key.id = '<KEY_OCID>'\",
  \"Admit group RemoteOKEAdminGroup of tenancy OKE_Tenancy to use keys in tenancy where target.key.id = '<KEY_OCID>'\"]"

Review / create cross‑tenancy policies in ClusterTenancy

In ClusterTenancy, gather OCIDs for: KeyTenancy, OKEAdminGroup, OKEAdminDynGroup:
```
oci iam tenancy get
oci iam group list --all
oci iam dynamic-group list --all
```

In the ClusterTenancy root compartment, create or review a policy with statements adapted to your OCIDs, e.g.:

Define tenancy KMS_Tenancy as <KEY_TENANCY_OCID>
Endorse group OKEAdminGroup to use keys in tenancy KMS_Tenancy
Endorse dynamic-group OKEAdminDynGroup to use keys in tenancy KMS_Tenancy
Allow dynamic-group OKEAdminDynGroup to use keys in tenancy where target.key.id = '<KEY_OCID>'

Apply via Console or, from ClusterTenancy, with:

oci iam policy create \
  --name "oke-cross-kms-clustertenancy" \
  --compartment-id <CLUSTER_TENANCY_ROOT_COMPARTMENT_OCID> \
  --statements "[\"Define tenancy KMS_Tenancy as <KEY_TENANCY_OCID>\",
  \"Endorse group OKEAdminGroup to use keys in tenancy KMS_Tenancy\",
  \"Endorse dynamic-group OKEAdminDynGroup to use keys in tenancy KMS_Tenancy\",
  \"Allow dynamic-group OKEAdminDynGroup to use keys in tenancy where target.key.id = '<KEY_OCID>'\"]"

Create a new cluster with the KMS key (required if existing cluster lacks kms-key-id)
- Encryption at rest for etcd secrets must be specified at cluster creation; you cannot retroactively attach a KMS key to an existing OKE cluster.
- From any machine with oci configured for ClusterTenancy, run:
```
oci ce cluster create \
  --name <NEW_CLUSTER_NAME> \
  --kubernetes-version <K8S_VERSION> \
  --vcn-id <VCN_OCID> \
  --service-lb-subnet-ids '["<SUBNET_OCID>"]' \
  --compartment-id <CLUSTER_TENANCY_COMPARTMENT_OCID> \
  --kms-key-id <KEY_OCID>
```
- Plan for migration of workloads from the old cluster to this new, encrypted cluster.
Verify encryption configuration (ClusterTenancy)
- After the cluster is ACTIVE, confirm that the KMS key is associated:
```
oci ce cluster get --cluster-id <NEW_CLUSTER_OCID> \
  --query "data.{name:name,id:id,\"kms-key-id\":\"options.kms-key-id\"}" \
  --output table
```
- The kms-key-id field must show the expected ocid1.key.oc1... from Vault; document this as evidence that Kubernetes secrets are configured to be encrypted at rest in etcd.

Using kubectl

kubectl cannot configure encryption of Kubernetes secrets at rest in etcd on OCI; this is controlled entirely by the managed control plane and OCI KMS/Vault configuration via the OCI Console, CLI, or IaC. To address this finding, follow the guidance in the Manual Steps section for setting up Vault keys, cross-tenancy policies (if needed), and creating or updating the cluster with a kms-key-id.

Automation

#!/usr/bin/env bash
#
# Check OKE clusters for Secrets encryption at rest in etcd (CIS OKE 5.3.1)
#
# Requirements:
#   - OCI CLI configured with a profile that can list clusters and compartments
#   - jq
#
# Usage:
#   ./check_oke_secrets_encryption.sh \
#       --compartment-ids "ocid1.compartment.oc1..aaaa,ocid1.compartment.oc1..bbbb" \
#       [--profile MYPROFILE] \
#       [--region eu-frankfurt-1]

set -euo pipefail

COMPARTMENT_IDS=""
OCI_PROFILE=""
OCI_REGION=""

while [[ $# -gt 0 ]]; do
  case "$1" in
    --compartment-ids)
      COMPARTMENT_IDS="$2"
      shift 2
      ;;
    --profile)
      OCI_PROFILE="--profile $2"
      shift 2
      ;;
    --region)
      OCI_REGION="--region $2"
      shift 2
      ;;
    *)
      echo "Unknown argument: $1" >&2
      exit 1
      ;;
  esac
done

if [[ -z "${COMPARTMENT_IDS}" ]]; then
  echo "ERROR: --compartment-ids is required (comma-separated list)" >&2
  exit 1
fi

if ! command -v oci >/dev/null 2>&1; then
  echo "ERROR: oci CLI not found in PATH" >&2
  exit 1
fi

if ! command -v jq >/dev/null 2>&1; then
  echo "ERROR: jq not found in PATH" >&2
  exit 1
fi

IFS=',' read -r -a COMPARTMENTS <<< "${COMPARTMENT_IDS}"

echo "=== OKE Secrets Encryption at Rest Report (CIS OKE 5.3.1) ==="
echo "Timestamp: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
echo

for COMP_ID in "${COMPARTMENTS[@]}"; do
  COMP_ID_TRIMMED="$(echo "${COMP_ID}" | xargs)"
  echo "Compartment: ${COMP_ID_TRIMMED}"
  echo "-----------------------------------------------------------------"

  # List all clusters in this compartment
  CLUSTERS_JSON="$(oci ce cluster list \
      --compartment-id "${COMP_ID_TRIMMED}" \
      ${OCI_PROFILE} ${OCI_REGION} \
      --all 2>/dev/null || true)"

  CLUSTER_COUNT="$(echo "${CLUSTERS_JSON}" | jq '.data | length')"

  if [[ -z "${CLUSTER_COUNT}" || "${CLUSTER_COUNT}" -eq 0 ]]; then
    echo "  No clusters found."
    echo
    continue
  fi

  echo "${CLUSTERS_JSON}" | jq -c '.data[]' | while read -r CL; do
    CL_OCID="$(echo "${CL}" | jq -r '.id')"
    CL_NAME="$(echo "${CL}" | jq -r '.name')"
    CL_LIFECYCLE_STATE="$(echo "${CL}" | jq -r '.lifecycle-state')"

    # Get full cluster details (includes kms-key-id)
    CL_DESC_JSON="$(oci ce cluster get \
        --cluster-id "${CL_OCID}" \
        ${OCI_PROFILE} ${OCI_REGION} 2>/dev/null || true)"

    KMS_KEY_ID="$(echo "${CL_DESC_JSON}" | jq -r '.data."kms-key-id" // empty')"

    echo "  Cluster: ${CL_NAME}"
    echo "    OCID: ${CL_OCID}"
    echo "    State: ${CL_LIFECYCLE_STATE}"

    if [[ -z "${KMS_KEY_ID}" || "${KMS_KEY_ID}" == "null" ]]; then
      echo "    KMS key: NONE CONFIGURED  <== REVIEW: Secrets likely NOT encrypted at rest"
    else
      echo "    KMS key: ${KMS_KEY_ID}    (Secrets encryption at rest ENABLED)"
    fi
    echo
  done
done

cat <<'EOF'

Interpretation:

- For each cluster:
  - "KMS key: <ocid1.key...> (Secrets encryption at rest ENABLED)"
      => Cluster is configured to use an OCI Vault (KMS) key for etcd secrets encryption.
         Next manual step: verify that cross-tenancy policies (if applicable) are correctly
         defined in both ClusterTenancy and KeyTenancy as per CIS OKE 5.3.1 remediation.

  - "KMS key: NONE CONFIGURED  <== REVIEW: Secrets likely NOT encrypted at rest"
      => Cluster was created without --kms-key-id and Secrets are likely stored
         unencrypted in etcd. This is a FINDING.
         Manual remediation:
           * For new clusters: create them with --kms-key-id pointing to a Vault key.
           * For cross-tenancy keys: define the required cross-tenancy policies in both
             tenancies, then recreate or newly create clusters with the desired key.

This script cannot automatically:
  - Create or rotate Vault keys
  - Configure or change the kms-key-id on existing clusters
  - Create the required cross-tenancy policies

Those must be designed and applied manually using OCI Console/CLI/IaC, following
the benchmark's guidance.
EOF

More Info:​

Risk Level​

Address​

Compliance Standards​

Triage and Remediation​

Remediation​