Skip to main content

Ensure Audit Policy Covers Key Security Concerns

More Info:​

Ensure that the audit policy created for the cluster covers key security concerns.

Risk Level​

High

Address​

Security

Compliance Standards​

  • CIS Kubernetes

Triage and Remediation​

Remediation​

Manual Steps

  1. Locate and identify the audit policy file

    • On every control plane node, inspect the API server manifest to find the --audit-policy-file path: 
      sudo grep -E -- '--audit-policy-file=' /etc/kubernetes/manifests/kube-apiserver.yaml
    • Note the file path (for example /etc/kubernetes/audit-policy.yaml). If the flag is missing or the file does not exist, plan to create/enable an audit policy file at a standard path (e.g. /etc/kubernetes/audit-policy.yaml).

  2. Collect the current audit policy for review

    • On every control plane node, display the current policy file (replace the path with the one found in step 1): 
      sudo cat /etc/kubernetes/audit-policy.yaml
    • Save a copy for change control: 
      sudo cp /etc/kubernetes/audit-policy.yaml /etc/kubernetes/audit-policy.yaml.backup.$(date +%F_%H%M%S)

  3. Review coverage for Secrets, ConfigMaps, and TokenReviews (metadata-only)

    • In the policy file, manually check for rules that match:
      • resources: ["secrets", "configmaps"] and group: "" (core API group), and
      • resources: ["tokenreviews"] and group: "authentication.k8s.io",
        with level: Metadata (or higher) and not logging the full object (omitStages/omitManagedFields as appropriate to avoid sensitive content).
    • If missing or overly broad (e.g. level: RequestResponse for these resources), edit the policy file with a root-capable editor on the control plane node, for example: 
      sudo vi /etc/kubernetes/audit-policy.yaml
      and add or adjust rules so that these resources are covered at least at Metadata level while avoiding sensitive payload logging.

  4. Review coverage for Pod/Deployment modifications

    • In the same policy file, manually check for rules that:
      • Target verbs like create, update, patch, delete, deletecollection, and
      • Apply to resources such as pods (group "") and deployments (group apps).
    • Ensure these rules have at least level: Metadata (or higher, according to your risk tolerance) so that changes to workloads are auditable. If absent, edit /etc/kubernetes/audit-policy.yaml to add such rules.

  5. Review coverage for exec/portforward/proxy usage

    • In the policy file, check for rules that match:
      • resources: ["pods/exec", "pods/portforward", "pods/proxy"] in the core group "", and
      • resources: ["services/proxy"] in the core group "",
        with verbs typically including create and get as applicable, and level: Metadata or higher.
    • If these subresources are not explicitly covered or are only matched by a very generic low-level rule (e.g. level: None), update /etc/kubernetes/audit-policy.yaml to add specific rules that log at least metadata for these actions.

  6. Apply and validate the updated audit policy

    • Saving changes to /etc/kubernetes/audit-policy.yaml on a control plane node that runs the API server as a static pod under /etc/kubernetes/manifests will cause the kubelet to automatically restart the kube-apiserver container to pick up the updated policy; expect a brief control-plane disruption.
    • After a minute, confirm the API server has restarted and is using the policy file: 
      sudo crictl ps | grep kube-apiserver
      sudo grep -E -- '--audit-policy-file=' /etc/kubernetes/manifests/kube-apiserver.yaml
    • Optionally, trigger a covered action (for example, creating a test Pod or reading a Secret) and inspect the audit log file configured via --audit-log-path on the control plane node: 
      sudo grep -E '"resource":"(secrets|pods|deployments|pods/exec|pods/portforward|pods/proxy|services/proxy)"' /var/log/kubernetes/audit.log | head
      to verify entries are recorded at least at the metadata level for the required resources and operations.
Using kubectl

kubectl cannot modify the API server’s audit policy or its manifest at /etc/kubernetes/manifests/kube-apiserver.yaml, because those are host-level files and flags on the control plane nodes. To address this finding, make the changes directly on every control plane node as described in the Manual Steps section.

Automation
#!/usr/bin/env bash
# Purpose: Summarize Kubernetes API audit policy coverage for key security concerns.
# Scope:   Run on ANY MACHINE with:
#            - kubectl access to the cluster
#            - ssh access to EVERY CONTROL PLANE NODE (for policy files and flags)
#
# NOTE: This script DOES NOT fix anything. It only reports current state for review.

set -euo pipefail

# -----------------------------
# Helper: print section header
# -----------------------------
sec() {
  printf '\n==== %s ====\n' "$*"
}

# ---------------------------------------------------
# 1. Discover control plane nodes via kubectl labels
# ---------------------------------------------------
sec "Discovering control plane nodes"

CONTROL_PLANE_NODES=$(kubectl get nodes -l node-role.kubernetes.io/control-plane= -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true)

if [ -z "$CONTROL_PLANE_NODES" ]; then
  # Older clusters may use the master label
  CONTROL_PLANE_NODES=$(kubectl get nodes -l node-role.kubernetes.io/master= -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true)
fi

if [ -z "$CONTROL_PLANE_NODES" ]; then
  echo "No control plane nodes discovered via labels."
  echo "You may need to specify them manually in this script."
  exit 1
fi

echo "Control plane nodes:"
echo "$CONTROL_PLANE_NODES"

# ----------------------------------------------------
# 2. For each control plane node, inspect kube-apiserver
# ----------------------------------------------------
for NODE in $CONTROL_PLANE_NODES; do
  sec "Node: $NODE"

  # Assumes SSH access via the same name; adjust ssh target format if needed.
  SSH_TARGET="$NODE"

  # 2.1 Confirm kube-apiserver manifest path and audit flags
  echo "-- kube-apiserver manifest and flags --"
  ssh "$SSH_TARGET" 'sudo test -f /etc/kubernetes/manifests/kube-apiserver.yaml && echo "Found /etc/kubernetes/manifests/kube-apiserver.yaml" || echo "MISSING: /etc/kubernetes/manifests/kube-apiserver.yaml"' || true

  ssh "$SSH_TARGET" '
    if [ -f /etc/kubernetes/manifests/kube-apiserver.yaml ]; then
      echo
      echo "kube-apiserver command and audit-related flags:"
      # Print container command and all args
      yq e ".spec.containers[] | select(.name==\"kube-apiserver\") | .command, .args[]" /etc/kubernetes/manifests/kube-apiserver.yaml 2>/dev/null || \
      python - <<PY 2>/dev/null
import yaml
from pathlib import Path
p = Path("/etc/kubernetes/manifests/kube-apiserver.yaml")
if p.is_file():
    data = yaml.safe_load(p.read_text())
    for c in data.get("spec", {}).get("containers", []):
        if c.get("name") == "kube-apiserver":
            for v in c.get("command", []):
                print(v)
            for v in c.get("args", []):
                print(v)
PY

      echo
      echo "Filtered audit flags:"
      yq e ".spec.containers[] | select(.name==\"kube-apiserver\") | .args[]" /etc/kubernetes/manifests/kube-apiserver.yaml 2>/dev/null \
        | grep -E -- "--audit-(log-path|policy-file|maxage|maxbackup|maxsize|webhook-config-file|webhook-mode|webhook-batch-max-wait|webhook-batch-max-size)" || true
    else
      echo "kube-apiserver manifest not found; cannot detect audit configuration on this node."
    fi
  '

  # 2.2 Extract audit-policy-file path from manifest
  AUDIT_POLICY_PATH=$(
    ssh "$SSH_TARGET" '
      if [ -f /etc/kubernetes/manifests/kube-apiserver.yaml ]; then
        yq e ".spec.containers[] | select(.name==\"kube-apiserver\") | .args[]" /etc/kubernetes/manifests/kube-apiserver.yaml 2>/dev/null \
          | grep -E "^--audit-policy-file=" | head -n1 | sed "s/^--audit-policy-file=//"
      fi
    ' 2>/dev/null || true
  )

  if [ -z "$AUDIT_POLICY_PATH" ]; then
    echo
    echo "WARNING: No --audit-policy-file flag detected on $NODE."
    echo "This likely means audit is not governed by a custom policy file on this node."
    continue
  fi

  echo
  echo "Detected audit policy file on $NODE: $AUDIT_POLICY_PATH"

  # 2.3 Show high-level policy info: existence and top-level rules count
  ssh "$SSH_TARGET" "
    if [ -f '$AUDIT_POLICY_PATH' ]; then
      echo
      echo 'Audit policy file exists. Top-level rules count:'
      (yq e '.rules | length' '$AUDIT_POLICY_PATH' 2>/dev/null || python - <<PY 2>/dev/null
import yaml, sys
p = '$AUDIT_POLICY_PATH'
try:
    with open(p) as f:
        d = yaml.safe_load(f)
    print(len(d.get('rules', []) or []))
except Exception as e:
    print('ERROR parsing policy:', e, file=sys.stderr)
PY
      ) || true
    else
      echo
      echo 'WARNING: Audit policy file path configured but file not found: $AUDIT_POLICY_PATH'
    fi
  "

  # 2.4 Inspect coverage of key security concerns
  if [ -n "$AUDIT_POLICY_PATH" ]; then
    ssh "$SSH_TARGET" "
      if [ -f '$AUDIT_POLICY_PATH' ]; then
        echo
        echo '--- Policy excerpts: SECRETS, CONFIGMAPS, TOKENREVIEWS (should log at least metadata, avoid full object) ---'
        echo
        echo 'Rules referencing secrets/configmaps/tokenreviews:'
        yq e '.rules[] | select((.resources[]?.resources[]? == \"secrets\") or (.resources[]?.resources[]? == \"configmaps\") or (.resources[]?.resources[]? == \"tokenreviews\"))' '$AUDIT_POLICY_PATH' 2>/dev/null || python - <<PY 2>/dev/null
import yaml
from pprint import pprint
p = '$AUDIT_POLICY_PATH'
with open(p) as f:
    d = yaml.safe_load(f)
for r in d.get('rules', []):
    for rs in r.get('resources', []) or []:
        for res in rs.get('resources', []) or []:
            if res in ('secrets', 'configmaps', 'tokenreviews'):
                pprint(r)
                print('---')
                break
PY

        echo
        echo '--- Policy excerpts: MODIFICATION of pods & deployments ---'
        echo
        echo 'Rules that match pods/deployments with verb create/update/patch/delete:'
        yq e '.rules[] | select((.resources[]?.resources[]? == \"pods\" or .resources[]?.resources[]? == \"deployments\") and (.verbs[]? == \"create\" or .verbs[]? == \"update\" or .verbs[]? == \"patch\" or .verbs[]? == \"delete\"))' '$AUDIT_POLICY_PATH' 2>/dev/null || python - <<PY 2>/dev/null
import yaml
from pprint import pprint
p = '$AUDIT_POLICY_PATH'
with open(p) as f:
    d = yaml.safe_load(f)
interesting = {'pods', 'deployments'}
verbs = {'create','update','patch','delete'}
for r in d.get('rules', []):
    has_res = False
    for rs in r.get('resources', []) or []:
        if any(res in interesting for res in (rs.get('resources') or [])):
            has_res = True
    if not has_res:
        continue
    if r.get('verbs') and any(v in verbs for v in r['verbs']):
        pprint(r)
        print('---')
PY

        echo
        echo '--- Policy excerpts: pods/exec, pods/portforward, pods/proxy, services/proxy ---'
        echo
        echo 'Rules matching subresources exec/portforward/proxy:'
        yq e '.rules[] | select(.resources[]?.resources[]? == \"pods\" or .resources[]?.resources[]? == \"services\") | select(.resources[]?.subresources[]? == \"exec\" or .resources[]?.subresources[]? == \"portforward\" or .resources[]?.subresources[]? == \"proxy\")' '$AUDIT_POLICY_PATH' 2>/dev/null || python - <<PY 2>/dev/null
import yaml
from pprint import pprint
p = '$AUDIT_POLICY_PATH'
with open(p) as f:
    d = yaml.safe_load(f)
interesting_res = {'pods', 'services'}
interesting_sub = {'exec','portforward','proxy'}
for r in d.get('rules', []):
    found = False
    for rs in r.get('resources', []) or []:
        if any(res in interesting_res for res in (rs.get('resources') or [])):
            subs = set(rs.get('subresources') or [])
            if subs & interesting_sub:
                found = True
    if found:
        pprint(r)
        print('---')
PY
      else
        echo
        echo 'Cannot inspect policy; file missing: $AUDIT_POLICY_PATH'
      fi
    "
  fi
done

sec "INTERPRETING THIS OUTPUT (what indicates a PROBLEM)"

cat <<'EOF'
For each control plane node:

1. Missing or misconfigured audit policy
   - Problem if:
     - kube-apiserver manifest is missing, OR
     - There is NO --audit-policy-file flag, OR
     - The configured audit policy file path does not exist.
   - Impact: API requests may not be audited according to a policy.

2. Secrets / ConfigMaps / TokenReviews coverage
   - Problem if:
     - No rules are printed that mention resources: ["secrets"], ["configmaps"], or ["tokenreviews"], OR
     - The matching rules log at Level: None (i.e., effectively not logged).
   - Risk: Access to sensitive objects is not visible in audit logs.
   - Additional risk: If Level is Request, RequestResponse, or includes ResponseBody, sensitive payloads may be logged rather than just metadata.

3. Modification of Pods and Deployments
   - Problem if:
     - No rules are printed that include resources: ["pods"] or ["deployments"] AND verbs including create, update, patch, or delete.
   - Risk: Changes to workload definitions are not audited.

4. pods/exec, pods/portforward, pods/proxy, services/proxy
   - Problem if:
     - No rules are printed referencing:
         - resource: pods with subresources: exec, portforward, proxy
         - resource: services with subresource: proxy
   - Risk: Interactive and lateral-movement style access paths are not recorded.

5. Logging level
   - General recommendation from the benchmark:
     - For most requests, minimally log at the Metadata level.
   - Problems include:
     - Level: None for the above key areas (no auditing).
     - Very verbose levels capturing response bodies for Secrets/ConfigMaps/TokenReviews (potential exposure of sensitive data).

This script only surfaces the current configuration; you must manually review the printed rules to ensure they align with the benchmark guidance and your organization's requirements.
EOF

Additional Reading:​