An Admission Policy Engine Should Enforce Workload Policy

More Info:

Advisory: an admission controller (Pod Security Admission, Kyverno, or OPA Gatekeeper) should enforce workload best practices at admission time, not only detect them after the fact.

Risk Level

Medium

Address

Security

Compliance Standards

• Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

1. Decide admission controller strategy (design choice)

   • On any machine with gcloud, list cluster details and note Kubernetes version and whether Autopilot is used:

     gcloud container clusters describe CLUSTER_NAME \
       --region=REGION \
       --format="yaml(name,location,releaseChannel,autopilot,verticalPodAutoscaling,ipAllocationPolicy,networkConfig)"

   • Decide whether to:
     • Use built‑in Pod Security Admission (PSA) profiles (recommended baseline: restricted), and/or
     • Deploy a policy engine (Kyverno or Gatekeeper) via Terraform/Deployment Manager.

2. Review Pod Security Admission (PSA) configuration (if using PSA)

   • On any machine with gcloud, check if a default Pod Security profile is set at cluster level (only via GKE APIs/IaC; this will often be unset):

     gcloud container clusters describe CLUSTER_NAME \
       --region=REGION \
       --format="yaml(binaryAuthorization,workloadIdentityConfig,securityPostureConfig)"

     (GKE currently has no dedicated top‑level PSA field; you must enforce PSA via namespace labels or policy engine.)
   • On any machine with kubectl access, list namespaces and their PSA labels to see if restrictive profiles are enforced at admission:

     kubectl get ns \
       -o=custom-columns=NAME:.metadata.name, \
       enforce:.metadata.labels.pod-security\.kubernetes\.io/enforce, \
       enforce-version:.metadata.labels.pod-security\.kubernetes\.io/enforce-version

   • If critical namespaces (application namespaces, not kube-system) lack enforce=restricted (or at least baseline), plan to manage these labels via Terraform/Deployment Manager (not kubectl for long‑term drift‑free config).

3. Assess whether a policy engine (Kyverno / Gatekeeper) is already in use

   • On any machine with kubectl access, check for controllers and policies (discovery only; final config should be via IaC):

     kubectl get ns | egrep 'kyverno|gatekeeper'
     kubectl api-resources | egrep 'ClusterPolicy|ClusterConstraintTemplate|Policy'
     kubectl get clusterpolicies.kyverno.io -A 2>/dev/null
     kubectl get k8sverylongnamedconstraints -A 2>/dev/null || true
     kubectl get constrainttemplates.templates.gatekeeper.sh 2>/dev/null || true

   • If no Kyverno/Gatekeeper controllers or policies exist, you currently do not have an admission policy engine enforcing workload best practices.

4. Map existing/desired policies to best practices C1–C5

   • On any machine with kubectl access, inspect at least a sample of current policies for enforcement vs. audit:

     # Kyverno example
     kubectl get clusterpolicies.kyverno.io -o yaml
     
     # Gatekeeper example
     kubectl get constraints -A -o yaml 2>/dev/null || true

   • Manually compare what’s enforced to the C1–C5 topics (e.g., non‑root, minimal capabilities, resource limits, disallow hostPath/privileged, require secure probes/config).
   • Decide which of these should be block‑on‑admission vs. audit only, and document that decision for implementation in your IaC (Terraform/Deployment Manager modules or GKE fleet policy if applicable).

5. Implement or strengthen admission enforcement via GKE‑appropriate IaC/controls

   • If using PSA: define standard namespace labels for your environments in Terraform/Deployment Manager (example conceptual snippet for Terraform, not a direct command):

     resource "kubernetes_namespace" "prod" {
       metadata {
         name = "prod"
         labels = {
           "pod-security.kubernetes.io/enforce"         = "restricted"
           "pod-security.kubernetes.io/enforce-version" = "latest"
         }
       }
     }

   • If using Kyverno or Gatekeeper: use Helm/Terraform/Config Sync to install the controller and define policies/constraints that enforce the C1–C5 rules (e.g., deny privileged pods, require resource limits). Ensure policies use deny/enforce modes, not just audit.
   • For Autopilot clusters, prefer PSA labels and Gatekeeper policies integrated via Config Sync/fleet‑level policy.

6. Verify enforcement is active, not just detection

   • On any machine with kubectl access, attempt to create a workload that intentionally violates your intended policy (e.g., privileged pod without resource limits):

     cat > /tmp/violating-pod.yaml << 'EOF'
     apiVersion: v1
     kind: Pod
     metadata:
       name: violating-pod
       namespace: TEST_NAMESPACE
     spec:
       containers:
       - name: c
         image: gcr.io/google-containers/pause:3.2
         securityContext:
           privileged: true
     EOF
     
     kubectl apply -f /tmp/violating-pod.yaml

   • Confirm that the request is rejected at admission time with an error from PSA/Kyverno/Gatekeeper (HTTP 4xx and a message referencing the policy/PodSecurity) instead of being created successfully. If it is allowed, revisit steps 2–5 to tighten enforcement.

Using kubectl

kubectl cannot configure admission policy engines or GKE control‑plane features; those are managed via the Google Cloud console, gcloud CLI, or your IaC definitions. Review the Manual Steps section for how to enable and configure Pod Security Admission or a third‑party admission controller (Kyverno or Gatekeeper) at the cloud provider level.

Automation

#!/usr/bin/env bash
# Report admission policy engines and workload policy enforcement on a GKE cluster
# Run on: any machine with kubectl access and context set to the target cluster

set -euo pipefail

echo "=== 1. Cluster version and features (GKE) ==="
echo "# GKE control-plane version:"
kubectl version --short || true
echo

echo "# Pod Security Admission (PSA) usage: namespaces with pod-security labels"
echo "# Namespaces SHOULD define pod-security.kubernetes.io/enforce labels at level baseline or restricted."
echo "# Missing or 'privileged' = weak or no enforcement."
kubectl get ns --show-labels | sed 's/,/\n          /g'
echo

echo "=== 2. PodSecurityPolicy (legacy) status ==="
echo "# PodSecurityPolicy is deprecated in GKE; if still present, investigate but do not rely on it for future enforcement."
kubectl get psp 2>/dev/null || echo "No PodSecurityPolicy resources found (or API disabled)."
echo

echo "=== 3. Kyverno status (if installed) ==="
echo "# Check if Kyverno admission controller is present and ready."
kubectl get ns kyverno >/dev/null 2>&1 && {
  echo "# Kyverno namespace found:"
  kubectl get pods -n kyverno -o wide
  echo
  echo "# Kyverno ClusterPolicies:"
  kubectl get clusterpolicies.kyverno.io -o wide 2>/dev/null || echo "No Kyverno ClusterPolicies found."
  echo
  echo "# Kyverno Policies (namespaced):"
  kubectl get policies.kyverno.io -A -o wide 2>/dev/null || echo "No namespaced Kyverno Policies found."
  echo
  echo "# Kyverno policy enforcement summary (count by validation mode):"
  kubectl get clusterpolicies.kyverno.io -o json 2>/dev/null \
    | jq -r '.items[]
      | {name: .metadata.name,
         validationFailureAction: .spec.validationFailureAction,
         background: .spec.background}
      | @tsv' 2>/dev/null \
    || echo "jq not available or no Kyverno ClusterPolicies."
} || {
  echo "Kyverno namespace not found. Kyverno is likely not installed."
}
echo

echo "=== 4. OPA Gatekeeper status (if installed) ==="
echo "# Check if Gatekeeper admission controller is present and ready."
kubectl get ns gatekeeper-system >/dev/null 2>&1 && {
  echo "# Gatekeeper namespace found:"
  kubectl get pods -n gatekeeper-system -o wide
  echo
  echo "# Gatekeeper ConstraintTemplates:"
  kubectl get constrainttemplates.templates.gatekeeper.sh -o wide 2>/dev/null || echo "No ConstraintTemplates found."
  echo
  echo "# Gatekeeper Constraints (all kinds):"
  kubectl api-resources --api-group='constraints.gatekeeper.sh' -o name 2>/dev/null | while read -r kind; do
    echo "## ${kind}:"
    kubectl get "${kind}" -A -o wide 2>/dev/null || echo "  None"
    echo
  done
} || {
  echo "gatekeeper-system namespace not found. OPA Gatekeeper is likely not installed."
}
echo

echo "=== 5. Workload policy enforcement indicators ==="
echo "# 5.1 PSA: namespaces WITHOUT enforce labels (potentially no pod security enforcement)"
echo "# Namespaces listed below do NOT have pod-security.kubernetes.io/enforce* labels:"
kubectl get ns -o json \
  | jq -r '
    .items[]
    | select(.metadata.labels["pod-security.kubernetes.io/enforce"] == null
             and .metadata.labels["pod-security.kubernetes.io/enforce-level"] == null)
    | .metadata.name' 2>/dev/null \
  || echo "jq not available; manually inspect namespace labels above."
echo

echo "# 5.2 Kyverno: policies that are 'audit' only (do not block non-compliant workloads)"
kubectl get clusterpolicies.kyverno.io -o json 2>/dev/null \
  | jq -r '
    .items[]
    | select(.spec.validationFailureAction == "audit" or .spec.validationFailureAction == null)
    | .metadata.name' 2>/dev/null \
  && echo "# Above Kyverno ClusterPolicies are not enforcing (audit or default). Review if they should be enforce." \
  || echo "No Kyverno ClusterPolicies found or jq not available."
echo

echo "# 5.3 Gatekeeper: constraints configured in 'dryrun' mode (do not block non-compliant workloads)"
kubectl api-resources --api-group='constraints.gatekeeper.sh' -o name 2>/dev/null | while read -r kind; do
  kubectl get "${kind}" -A -o json 2>/dev/null \
    | jq -r --arg kind "${kind}" '
        .items[]
        | select(.spec.enforcementAction == "dryrun" or .spec.enforcementAction == null)
        | "\($kind)\t\(.metadata.namespace // "-")\t\(.metadata.name)\t\(.spec.enforcementAction // "default")"
      ' 2>/dev/null \
    || true
done
echo "# Lines above (if any) are Gatekeeper constraints that do not enforce; they only audit."
echo

echo "=== Interpretation Guide ==="
cat <<'EOF'
This script does NOT automatically fix anything. Use the output to review:

- If NO Kyverno and NO Gatekeeper components are detected AND most namespaces lack
  pod-security.kubernetes.io/enforce* labels, then the cluster likely does NOT
  have an admission policy engine enforcing workload best practices (C6.3 issue).

- Pod Security Admission:
  - Problem indicator: many application namespaces appear in the list of
    'namespaces WITHOUT enforce labels', or labels are set to 'privileged'.
  - Healthy indicator: application namespaces have enforce labels set to
    'baseline' or 'restricted', optionally with versions.

- Kyverno:
  - Problem indicator: no Kyverno namespace or policies, or policies exist but
    validationFailureAction is 'audit' (or omitted) for policies meant to enforce
    C1–C5 best practices.
  - Healthy indicator: key security policies use validationFailureAction:
    'enforce' (or 'enforce' via global default) and cover relevant workloads.

- OPA Gatekeeper:
  - Problem indicator: no gatekeeper-system namespace or constraints, or most
    constraints have enforcementAction 'dryrun' (or defaulting to non-blocking).
  - Healthy indicator: constraints with enforcementAction 'deny' (or equivalent)
    that cover C1–C5 best practices for your workloads.

Use this report alongside your GKE configuration (e.g., Pod Security settings,
add-ons, and any admission webhooks deployed via IaC) to decide how and where
to enable enforcing admission policies. There is no single automated remediation;
you must design policies appropriate for your environment and then enable
enforcement in your GKE configuration.
EOF