> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure Image Provenance Using ImagePolicyWebhook Admission Controller

### More Info:

The ImagePolicyWebhook admission controller can enforce image provenance so that only trusted, verified images are admitted to the cluster.

### Risk Level

High

### Address

Security

### Compliance Standards

* CIS Kubernetes

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Manual Steps" defaultOpen="true">
        1. **Review current admission configuration for ImagePolicyWebhook**
           * On any machine with kubectl access:
             ```bash theme={null}
             # AdmissionConfiguration is usually passed via --admission-control-config-file or --admission-control-config
             # Try to discover it from the kube-apiserver static pod manifest
             sudo cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep -E 'admission-control-config'
             ```
           * If you find a config file path (for example `/etc/kubernetes/admission-config.yaml`), inspect it:
             ```bash theme={null}
             sudo cat /etc/kubernetes/admission-config.yaml
             ```
           * Confirm whether there is a `ImagePolicyWebhook` plugin section configured; if not, the feature is not in use.

        2. **Determine the desired image provenance mechanism and trust policy**
           * Decide what constitutes a “trusted image” in your environment (e.g., signed by a particular key, coming from a specific registry/project, or verified by an external attestation service).
           * Choose or design the webhook backend that will enforce this policy (custom admission webhook service, in-cluster or external, or a supported third‑party image verification solution).
           * Document:
             * The verification method (signing/attestation technology).
             * The registries/namespaces allowed.
             * Failure policy (reject on verification error vs allow).

        3. **Design or validate the ImagePolicyWebhook AdmissionConfiguration**
           * Draft an `AdmissionConfiguration` manifest including the `ImagePolicyWebhook` plugin that points to your webhook backend and enforces your policy decisions. For example (adjust URLs, timeouts, and CABundle to your environment):
             ```yaml theme={null}
             apiVersion: apiserver.k8s.io/v1alpha1
             kind: AdmissionConfiguration
             plugins:
               - name: ImagePolicyWebhook
                 configuration:
                   imagePolicy:
                     kubeConfigFile: "/etc/kubernetes/image-policy-webhook.kubeconfig"
                     allowTTL: 50
                     denyTTL: 50
                     retryBackoff: 500
                     defaultAllow: false
             ```
           * Ensure the referenced `kubeconfig` (or other connection method your Kubernetes version supports) exists and points to your policy webhook service.
           * Do **not** apply yet; validate it against Kubernetes version compatibility and your webhook implementation docs.

        4. **Stage and test the webhook service and policy in a non‑production environment**
           * Deploy the admission webhook service (if not already running) in a test or staging cluster using manifests:
             ```bash theme={null}
             kubectl apply -f /path/to/image-policy-webhook-deployment.yaml
             kubectl apply -f /path/to/image-policy-webhook-service.yaml
             ```
           * Configure the apiserver in that environment to use your drafted `AdmissionConfiguration` and restart the apiserver as required by your distribution.
           * Test with sample workloads:
             ```bash theme={null}
             kubectl run unsigned-test --image=UNTRUSTED_IMAGE
             kubectl run signed-test   --image=TRUSTED_IMAGE
             ```
           * Confirm that untrusted images are rejected and trusted ones are admitted, and refine policy if behavior is not as intended.

        5. **Promote the configuration to production and monitor**
           * Copy the validated `AdmissionConfiguration` file and supporting artifacts (kubeconfig, CA certs) to the production control plane nodes, then update the `kube-apiserver` manifest (or equivalent) to reference it via the appropriate flag. This is done outside kubectl (host-level change) and will restart the apiserver.
           * After the apiserver restarts, verify that workloads using untrusted images are rejected and trusted images succeed:
             ```bash theme={null}
             kubectl run provenance-test-bad --image=UNTRUSTED_IMAGE
             kubectl run provenance-test-good --image=TRUSTED_IMAGE
             ```
           * Review apiserver and webhook logs for errors or excessive denials and adjust policy if needed.

        6. **Document, periodically review, and re‑verify**
           * Record: where the `AdmissionConfiguration` file lives, what the policy rules are, and who maintains the webhook service.
           * On a recurring basis, re‑check configuration and effectiveness:
             ```bash theme={null}
             # Confirm the ImagePolicyWebhook plugin remains configured
             sudo cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep -E 'admission-control-config'

             # Confirm the webhook endpoint is reachable from the cluster
             kubectl get pods,svc -n NAMESPACE_OF_WEBHOOK
             ```
           * Re-run functional tests (trusted vs untrusted images) after significant cluster or policy changes to ensure image provenance enforcement remains active.
      </Accordion>

      <Accordion title="Using kubectl">
        ```bash theme={null}
        # 1) Check if any ImagePolicyWebhook configuration exists
        # Run on: any machine with kubectl access

        kubectl get validatingwebhookconfigurations,mutatingwebhookconfigurations \
          -o wide
        ```

        **What to look for (problem indicators)**

        * No webhook configuration related to image policy or provenance (for example names not mentioning `image-policy`, `imageprovenance`, `cosign`, `notary`, `policy`, etc.).
        * This usually means no image provenance enforcement is configured at all.

        ***

        ```bash theme={null}
        # 2) Inspect likely image-policy related ValidatingWebhookConfiguration(s)

        # Adjust the name(s) based on the output of the previous command
        kubectl get validatingwebhookconfigurations <webhook-name> -o yaml
        ```

        **What to look for (problem indicators)**

        * `webhooks[].rules` do **not** include `apiGroups: [""]` and `resources: ["pods"]` or other workload types (`deployments`, `replicasets`, `statefulsets`, etc.) — then pod/image creation may bypass the policy.
        * `failurePolicy` is `Ignore` instead of `Fail` — unverified images may still be admitted.
        * `sideEffects` is unset or incorrect (should typically be `None`).
        * `clientConfig.service` points to a non‑existent namespace/service, or wrong `path`/`port`.
        * `namespaceSelector` or `objectSelector` exclude important namespaces (e.g., default application namespaces), leaving workloads unenforced.
        * No mention in annotations or configuration of trusted registries, signatures, or verification policy.

        ***

        ```bash theme={null}
        # 3) Check if any MutatingWebhookConfiguration is being used for image policy
        kubectl get mutatingwebhookconfigurations -o yaml
        ```

        **What to look for (problem indicators)**

        * No mutating webhook that adjusts image references to a verified form (if your design expects this).
        * Any mutating webhook that weakens image references (e.g., stripping digests or pinning to `:latest`) instead of strengthening provenance.

        ***

        ```bash theme={null}
        # 4) Discover and inspect policy engine components (if any)
        # Common namespaces; adapt or add your org’s namespaces as needed
        kubectl get pods -A | egrep -i 'image|policy|cosign|notary|kyverno|connaisseur|gatekeeper'

        # Example: inspect a policy engine namespace
        kubectl get all -n <policy-namespace>
        kubectl describe deployment <policy-deployment> -n <policy-namespace>
        kubectl get configmap,secret -n <policy-namespace> -o wide
        ```

        **What to look for (problem indicators)**

        * No policy engine or image-verification service deployed at all.
        * Configuration (in ConfigMaps/Secrets) that does **not** reference trusted registries, keys, or signature verification settings.
        * Policy engine pods in `CrashLoopBackOff` or `Error` state, indicating admission requests may fail open depending on `failurePolicy`.

        ***

        ```bash theme={null}
        # 5) Exercise the current configuration with a test pod
        # Replace <untrusted/image:tag> and <trusted/image@sha256:...> with images appropriate for your environment

        # Test A: attempt to create a pod using an image that should FAIL provenance checks
        kubectl apply -f - << 'EOF'
        apiVersion: v1
        kind: Pod
        metadata:
          name: image-provenance-test-untrusted
        spec:
          containers:
          - name: test
            image: untrusted/image:tag
            command: ["sleep", "3600"]
          restartPolicy: Never
        EOF

        # Test B: attempt to create a pod using an image that should PASS provenance checks
        kubectl apply -f - << 'EOF'
        apiVersion: v1
        kind: Pod
        metadata:
          name: image-provenance-test-trusted
        spec:
          containers:
          - name: test
            image: trusted/image@sha256:deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef
            command: ["sleep", "3600"]
          restartPolicy: Never
        EOF

        # Check results
        kubectl get pods image-provenance-test-untrusted image-provenance-test-trusted
        kubectl describe pod image-provenance-test-untrusted
        kubectl describe pod image-provenance-test-trusted
        ```

        **What to look for (problem indicators)**

        * The “untrusted” pod is **created and runs successfully**; this strongly suggests image provenance is **not** enforced.
        * No admission webhook denial messages in the pod events (`kubectl describe pod ...`) for the untrusted image.
        * The “trusted” pod is rejected with provenance-related errors (misconfigured policy) or both pods are treated identically (no provenance enforcement).

        ***

        ```bash theme={null}
        # 6) Check API server flags for admission webhooks (if accessible via config in-cluster)
        # Some managed clusters expose a ConfigMap or API object with apiserver flags.
        # Example (adapt as needed):
        kubectl get pods -n kube-system -l component=kube-apiserver -o yaml
        ```

        **What to look for (problem indicators)**

        * No reference to `ImagePolicyWebhook` in `--enable-admission-plugins` **if** your design relies on the legacy built‑in ImagePolicyWebhook admission plugin (on self‑managed control planes).
        * For managed control planes (EKS/AKS/GKE/OKE), control plane flags are typically not visible; in that case, absence of any webhooks/configs as above is the main indicator.

        ***

        All of the above commands only surface the current state. Any decision to introduce, tighten, or relax image provenance policies, and the exact implementation (ImagePolicyWebhook vs. external policy engines), requires human review against your organization’s threat model and Kubernetes documentation.
      </Accordion>

      <Accordion title="Automation">
        ```bash theme={null}
        #!/usr/bin/env bash
        #
        # audit-image-policy-webhook.sh
        #
        # Purpose:
        #   Report whether ImagePolicyWebhook-based image provenance is configured
        #   and enforced across the cluster.
        #
        # Run on:
        #   Any machine with kubectl access and correct kubeconfig context.
        #
        # Requirements:
        #   - bash
        #   - kubectl in PATH
        #   - cluster-admin or equivalent access (to list webhooks, namespaces, and workloads)

        set -euo pipefail

        echo "=== Image Provenance / ImagePolicyWebhook Cluster Audit ==="
        echo "Kubeconfig context: $(kubectl config current-context)"
        echo

        ###############################################################################
        # 1. Check for presence of ImagePolicyWebhook admission configuration object
        ###############################################################################
        echo "1) AdmissionConfiguration objects that may define ImagePolicyWebhook:"
        kubectl get cm -A \
          --field-selector metadata.name=admission-configuration \
          -o wide 2>/dev/null || true
        echo

        echo "   Attempting to locate AdmissionConfiguration in kube-system namespace:"
        ADMISSION_CM_NAME=$(kubectl -n kube-system get cm \
          --no-headers 2>/dev/null | awk '/admission/{print $1}' || true)

        if [ -n "${ADMISSION_CM_NAME:-}" ]; then
          echo "   Found potential AdmissionConfiguration ConfigMap(s) in kube-system:"
          kubectl -n kube-system get cm "${ADMISSION_CM_NAME}" -o yaml
        else
          echo "   No obvious AdmissionConfiguration ConfigMap found in kube-system."
        fi
        echo

        ###############################################################################
        # 2. Check for Mutating/ValidatingWebhookConfiguration that enforce image policy
        ###############################################################################
        echo "2) ValidatingWebhookConfiguration objects related to image policy:"
        kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io \
          -o custom-columns=NAME:.metadata.name \
          --no-headers 2>/dev/null | grep -i "image\|policy" || echo "   (none matching 'image' or 'policy' in name)"
        echo

        echo "   Full list of ValidatingWebhookConfiguration objects (for manual review):"
        kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io -o wide || true
        echo

        echo "   Details for likely image-policy related webhooks:"
        for w in $(kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io \
                      -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null \
                    | grep -i "image\|policy" || true); do
          echo "----- ValidatingWebhookConfiguration: $w -----"
          kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io "$w" -o yaml
          echo
        done

        echo "2b) MutatingWebhookConfiguration objects related to image policy (less common):"
        kubectl get mutatingwebhookconfigurations.admissionregistration.k8s.io \
          -o custom-columns=NAME:.metadata.name \
          --no-headers 2>/dev/null | grep -i "image\|policy" || echo "   (none matching 'image' or 'policy' in name)"
        echo

        ###############################################################################
        # 3. Check for per-namespace admission configuration hints/annotations
        ###############################################################################
        echo "3) Namespaces with annotations that may indicate image policy enforcement:"
        kubectl get ns -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations}{"\n"}{end}' \
          2>/dev/null | grep -Ei 'image|policy|admission' || echo "   (no matching annotations found)"
        echo

        ###############################################################################
        # 4. Inventory workloads and their image registries (for provenance review)
        ###############################################################################
        echo "4) Workloads and container images per namespace:"
        echo "   NOTE: This does NOT prove image provenance is enforced; it supports review."

        namespaces=$(kubectl get ns -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}')
        for ns in $namespaces; do
          echo "----- Namespace: $ns -----"
          kubectl get pods -n "$ns" -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.containers[*]}{.image}{" "}{end}{"\n"}{end}' 2>/dev/null \
            || echo "   (no pods or access denied)"
          echo
        done

        ###############################################################################
        # 5. High-level summary hints
        ###############################################################################
        echo "=== Interpretation Guide (manual review required) ==="
        cat <<'EOF'
        This script does NOT determine compliance automatically. Use these indicators:

        1) ImagePolicyWebhook configuration:
           - Problem indicators:
             * No AdmissionConfiguration object referencing 'ImagePolicyWebhook' or an
               external image policy service.
             * No cluster-wide documentation/config in ConfigMap or kube-apiserver
               flags describing ImagePolicyWebhook.
           - Healthy indicators:
             * AdmissionConfiguration (via ConfigMap or kube-apiserver config file)
               defines an 'imagePolicy' or 'pluginConfig' entry of type
               'ImagePolicyWebhook' with a reachable backend (e.g., HTTPS endpoint).

        2) Webhook configurations:
           - Problem indicators:
             * No ValidatingWebhookConfiguration whose name, rules, or clientConfig
               clearly correspond to an image policy / provenance service.
             * Webhook exists but:
               - 'failurePolicy' is 'Ignore' (weak enforcement).
               - 'namespaceSelector' or 'objectSelector' excludes critical workloads.
               - 'rules' do NOT include create/update of core workload types
                 (pods, deployments, statefulsets, daemonsets, jobs, cronjobs).
           - Healthy indicators:
             * A ValidatingWebhookConfiguration that:
               - Targets pod and workload creation/updates in relevant API groups.
               - Uses 'failurePolicy: Fail' for enforcement (as appropriate).
               - Points to a trusted image policy service.

        3) Workload images:
           - Problem indicators:
             * Workloads pulling from untrusted or public registries without a clear
               corresponding image policy configuration.
           - Healthy indicators:
             * Images consistently sourced from approved registries, with a clearly
               documented policy on how provenance is validated by the webhook.

        Next steps:
           - If you do not find any clear ImagePolicyWebhook or equivalent admission
             webhook enforcing image provenance, treat this as a gap and design a
             policy and webhook deployment following Kubernetes documentation.
        EOF
        ```
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>