> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Audit Logging Should Be Enabled And Shipped Off-Cluster

### More Info:

Advisory: Kubernetes API audit logging should be enabled and forwarded to an external, tamper-resistant store so control-plane activity is retained independently of the cluster.

### Risk Level

Medium

### Address

Security

### Compliance Standards

* Cloudanix Best Practice

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Manual Steps" defaultOpen="true">
        1. **Identify your environment and provider-specific audit feature**
           * Determine whether this is EKS, AKS, GKE, OKE, or another managed service, and locate that provider’s Kubernetes API/audit logging feature in its docs or console (e.g., “control plane logging”, “master API audit logs”, “Kubernetes audit logs”).
           * If you use IaC (Terraform/CloudFormation/ARM/Deployment Manager), locate the module or stack that defines the cluster.

        2. **Check if API/audit logging is enabled (console and IaC)**
           * In the cloud console, open the cluster’s control-plane/monitoring/logging settings and verify whether Kubernetes API/audit logs are enabled and which log types are selected (API server, audit, controller manager, scheduler, etc.).
           * In your IaC code, verify the logging blocks match the console and ensure audit/API logs are enabled there as the source of truth (for example, look for fields like `enable_kubernetes_audit_logs`, `cluster_logging`, `control_plane_logging`, or similar).
           * If audit/API logging is disabled or only partially enabled, plan to enable full API/audit logging via the same mechanism you normally use (preferably IaC, then reconcile manually in the console).

        3. **Verify logs are being produced by the control plane**
           * From any machine with access to your logging backend (or via the provider log viewer), confirm that new Kubernetes API/audit events appear when you perform actions in the cluster (e.g., list pods, create/delete a test resource).
           * Example test: on any machine with `kubectl` access, run:
             ```bash theme={null}
             kubectl get pods -A >/dev/null
             ```
             Then in the logging backend or log viewer for the control-plane/audit logs, search for recent entries involving the `get` verb on `pods` from your user or IP.

        4. **Assess whether logs are shipped to an external, tamper-resistant store**
           * Determine the current storage location: confirm whether logs are only in an in-console log viewer or short-lived provider storage, or if they are exported to an external system (e.g., SIEM, separate log account/project, object storage with write-once / immutability / restricted access).
           * In the provider or logging service UI/CLI, review any configured log sinks/exports (e.g., “log sinks”, “subscriptions”, “streams”, “export rules”) to see where Kubernetes audit/API logs are being sent.

        5. **Configure or strengthen off-cluster forwarding and retention**
           * If Kubernetes API/audit logs are not forwarded off-cluster, create or update a log export/sink/stream to send them to an external, centralized, and access-controlled destination (e.g., dedicated logging account/project, immutable bucket, or SIEM).
           * In that destination, configure:
             * Retention to meet your policy.
             * Restricted write-only/append-only access for the logging pipeline.
             * Read access limited to appropriate security/operations roles.
           * Ensure this configuration is captured in your IaC (e.g., log sink resources, IAM/policy definitions, storage lifecycle rules).

        6. **Re‑verify end-to-end and document**
           * Trigger a new API action (e.g., create/delete a test namespace with `kubectl`) and confirm that a corresponding audit/API log record appears in the external store, not just the provider console.
           * Record in your runbook: how audit logging is enabled, where logs are exported, retention settings, and who has access; schedule periodic review to ensure the configuration and exports remain in place.
      </Accordion>

      <Accordion title="Using kubectl">
        kubectl cannot enable or configure Kubernetes API audit logging, nor can it direct logs to an external store; these settings are only available in your cloud provider’s managed control-plane configuration (console, CLI, or IaC). Refer to the Manual Steps section for provider-specific guidance on enabling audit logging and shipping it off-cluster.
      </Accordion>

      <Accordion title="Automation">
        ```bash theme={null}
        #!/usr/bin/env bash
        #
        # audit-api-audit-logging.sh
        #
        # Purpose:
        #   Gather evidence about Kubernetes API audit logging and whether
        #   audit logs are being shipped off-cluster, for manual review.
        #
        # Run on:
        #   Any machine with:
        #     - kubectl configured for the cluster
        #     - access to cloud CLI (optional but recommended: aws / az / gcloud / oci)
        #
        # Usage:
        #   chmod +x audit-api-audit-logging.sh
        #   ./audit-api-audit-logging.sh > audit-api-audit-logging-$(date +%F).txt

        set -euo pipefail

        echo "=== Audit: Kubernetes API Audit Logging & Off-Cluster Shipping ==="
        echo "Timestamp: $(date -Iseconds)"
        echo

        echo "Cluster info (for context)"
        echo "------------------------------------------------------------"
        kubectl version --short 2>/dev/null || echo "kubectl version: unable to query (check kubeconfig)"
        echo
        kubectl cluster-info 2>/dev/null || echo "kubectl cluster-info: unable to query (check permissions)"
        echo

        ###############################################################################
        # 1. Managed-control-plane detection (best-effort, for operator context)
        ###############################################################################

        echo "Control plane type detection (best effort)"
        echo "------------------------------------------------------------"
        # Try common hints but do NOT assume result is authoritative.
        PROVIDER_HINT="unknown"

        if kubectl get ns kube-system >/dev/null 2>&1; then
          if kubectl get configmap -n kube-system aws-auth >/dev/null 2>&1; then
            PROVIDER_HINT="EKS"
          elif kubectl get nodes -o jsonpath='{.items[0].metadata.labels.eks\.amazonaws\.com/nodegroup}' >/dev/null 2>&1; then
            PROVIDER_HINT="EKS"
          elif kubectl get nodes -o jsonpath='{.items[0].metadata.labels.cloud\.google\.com/gke-nodepool}' >/dev/null 2>&1; then
            PROVIDER_HINT="GKE"
          elif kubectl get nodes -o jsonpath='{.items[0].metadata.labels.agentpool}' >/dev/null 2>&1; then
            PROVIDER_HINT="AKS"
          fi
        fi

        echo "Provider hint (heuristic, verify manually): ${PROVIDER_HINT}"
        echo

        ###############################################################################
        # 2. In-cluster evidence of API audit logging usage
        ###############################################################################

        echo "In-cluster evidence related to API audit logging"
        echo "------------------------------------------------------------"

        echo
        echo "2.1. Check for namespaces and components commonly used for log shipping"
        echo "-----------------------------------------------------------------"
        kubectl get ns 2>/dev/null | sed 's/^/  /' || echo "  Unable to list namespaces"

        echo
        echo "2.2. Check for logging / audit shipping DaemonSets (cluster-wide)"
        echo "-----------------------------------------------------------------"
        kubectl get daemonset -A 2>/dev/null | sed 's/^/  /' || echo "  Unable to list DaemonSets"

        echo
        echo "2.3. Check for logging / audit shipping Deployments (cluster-wide)"
        echo "-----------------------------------------------------------------"
        kubectl get deploy -A 2>/dev/null | sed 's/^/  /' || echo "  Unable to list Deployments"

        echo
        echo "2.4. Look for components suggesting API server audit ingestion"
        echo "-----------------------------------------------------------------"
        echo "  (grep for 'audit' in Deployments/DaemonSets names)"
        {
          kubectl get deploy -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name --no-headers 2>/dev/null
          kubectl get daemonset -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name --no-headers 2>/dev/null
        } | grep -i "audit" || echo "  No Deployments/DaemonSets with 'audit' in the name found (this may be OK)."

        ###############################################################################
        # 3. Provider-specific hints (manual review required)
        ###############################################################################
        # NOTE:
        #   For managed control planes, *enabling API audit logging and off-cluster
        #   shipping* is configured in the cloud provider console / API / IaC.
        #   The following commands only help you discover current logging-related
        #   config; they do not guarantee that audit logging is properly enabled.
        ###############################################################################

        echo
        echo "Provider-specific evidence (you must review & interpret manually)"
        echo "------------------------------------------------------------"

        case "${PROVIDER_HINT}" in
          "EKS")
            echo "Detected EKS (heuristic)."
            echo
            echo "3.1. List EKS clusters and logging configuration via AWS CLI"
            echo "     (Run these on a machine with 'aws' configured and proper IAM perms)"
            cat <<'EOF'
        # List clusters
        aws eks list-clusters

        # For each cluster, show enabled control-plane logs, including audit
        aws eks describe-cluster \
          --name <CLUSTER_NAME> \
          --query 'cluster.logging.clusterLogging[*].{types:types, enabled:enabled}'
        EOF
            echo
            echo "Interpretation (problem indicators):"
            echo "  - If 'audit' is NOT present in 'types' with 'enabled: true',"
            echo "    then API audit logging is not enabled for that EKS cluster."
            echo "  - Even when enabled, confirm that CloudWatch Logs or another"
            echo "    external, tamper-resistant destination retains logs according"
            echo "    to your policy (verify in AWS console/IaC)."
            ;;
          "GKE")
            echo "Detected GKE (heuristic)."
            echo
            echo "3.1. List GKE clusters and logging config via gcloud"
            echo "     (Run these on a machine with 'gcloud' configured and proper perms)"
            cat <<'EOF'
        # List clusters with logging details
        gcloud container clusters list \
          --format="table(name,location,loggingService,loggingConfig)"

        # For a specific cluster, get full logging config (includes audit logs)
        gcloud container clusters describe <CLUSTER_NAME> \
          --region <REGION> \
          --format="yaml(loggingConfig,loggingService)"
        EOF
            echo
            echo "Interpretation (problem indicators):"
            echo "  - If 'loggingService' is 'none', control-plane logs (including audit)"
            echo "    are not being exported."
            echo "  - If 'loggingConfig.componentConfig.enableComponents' does NOT include"
            echo "    'APISERVER', then API server logs (including audits) may not be"
            echo "    collected."
            echo "  - Confirm logs are sent to Cloud Logging with retention and access"
            echo "    controls that make them tamper-resistant."
            ;;
          "AKS")
            echo "Detected AKS (heuristic)."
            echo
            echo "3.1. List AKS clusters and diagnostic settings via Azure CLI"
            echo "     (Run these on a machine with 'az' configured and proper perms)"
            cat <<'EOF'
        # List AKS clusters
        az aks list -o table

        # For a given cluster, show diagnostic settings on the managed resource
        AKS_RG="<RESOURCE_GROUP>"
        AKS_NAME="<CLUSTER_NAME>"

        # Get the underlying managed resource id
        AKS_ID=$(az aks show -g "$AKS_RG" -n "$AKS_NAME" --query id -o tsv)

        # List diagnostic settings (includes control-plane / audit categories if enabled)
        az monitor diagnostic-settings list --resource "$AKS_ID" -o json
        EOF
            echo
            echo "Interpretation (problem indicators):"
            echo "  - If there are NO diagnostic settings for the AKS resource, audit"
            echo "    logs are not being shipped off-cluster."
            echo "  - In the diagnostic settings JSON, if categories like 'kube-audit'"
            echo "    or equivalent are absent or disabled, API audit logging is not"
            echo "    being exported."
            echo "  - Confirm destination is Log Analytics / Event Hub / storage with"
            echo "    appropriate tamper-resistance and retention."
            ;;
          *)
            echo "Provider unknown from heuristics."
            echo
            echo "3.1. MANUAL: Determine your control-plane provider and review:"
            echo "  - Platform documentation for 'Kubernetes API audit logging'"
            echo "  - Control-plane logging / diagnostics settings in the provider console"
            echo "  - IaC definitions (Terraform, CloudFormation, ARM/Bicep, etc.)"
            echo
            echo "Example manual evidence commands (generic; adjust for your platform):"
            cat <<'EOF'
        # If using Terraform:
        grep -Rni "audit" . | head

        # If using Helm or other IaC for logging stacks:
        grep -Rni "audit" logging/ manifests/ | head
        EOF
            ;;
        esac

        ###############################################################################
        # 4. What output indicates a problem?
        ###############################################################################

        echo
        echo "Interpretation summary: what indicates a potential PROBLEM"
        echo "------------------------------------------------------------"
        cat <<'EOF'
        Flag as needing remediation if you observe ANY of the following:

        1) EKS:
           - In 'aws eks describe-cluster ...':
             - 'audit' is missing from 'cluster.logging.clusterLogging[*].types'
               OR
             - 'audit' exists but 'enabled' is false.
           - No clear CloudWatch Logs / external destination configured for control-plane logs.

        2) GKE:
           - 'loggingService' is 'none' for the cluster.
           - 'loggingConfig.componentConfig.enableComponents' does NOT include 'APISERVER'.
           - Logs are not visible in Cloud Logging or retention is too short / not protected.

        3) AKS:
           - 'az monitor diagnostic-settings list --resource <AKS_ID>' returns no settings.
           - Diagnostic settings do not include categories for API server / audit logs
             (e.g., 'kube-audit' or equivalent).
           - Destination (Log Analytics, Event Hub, Storage) is missing or clearly
             not tamper-resistant.

        4) Any provider:
           - No evidence that API-level audit logs are being produced by the control plane.
           - No evidence that such logs are being shipped to an external, independent,
             tamper-resistant log store.
           - IaC / configuration does not define audit logging or external log sinks.

        NOTE:
          This script only gathers evidence; it does NOT and CANNOT fully
          determine compliance automatically. You must review the outputs
          and your provider's configuration to decide if:
            - Kubernetes API audit logging is enabled, AND
            - Audit logs are being forwarded to an external, tamper-resistant store.
        EOF

        echo
        echo "=== End of audit ==="
        ```
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>