Skip to main content

Audit Logging Should Be Enabled And Shipped Off-Cluster

More Info:​

Advisory: Kubernetes API audit logging should be enabled and forwarded to an external, tamper-resistant store so control-plane activity is retained independently of the cluster.

Risk Level​

Medium

Address​

Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Manual Steps

  1. Identify the OKE cluster and compartment

    • On any machine with OCI CLI configured: 
      oci ce cluster list --compartment-id <OCID_OF_COMPARTMENT> --all
    • Note the id of the target cluster (for example, ocid1.cluster.oc1...).

  2. Check whether Kubernetes API audit logging is enabled for the cluster

    • Using OCI CLI on any machine: 
      oci ce cluster get --cluster-id <OCID_OF_CLUSTER> --query 'data."options"' --output json
    • In the JSON output, review any kubernetesAudit / apiServer / logging-related fields.
    • If the console exposes an “Audit logs” or “API server logs” toggle for OKE in the cluster’s details page, verify whether Kubernetes API audit logging is enabled there.

  3. Verify cluster logs are being shipped to OCI Logging

    • Using the OCI Console:
      • Go to Observability & Management → Logging → Log Groups.
      • In the compartment that contains the cluster, look for an OKE-related log group (for example, named with the cluster or node pool).
      • Open it and verify there is a log for control-plane or audit events (for example, “Kubernetes API Server Logs” or similar).
    • Or via OCI CLI on any machine: 
      oci logging log-group list --compartment-id <OCID_OF_COMPARTMENT> --all
      oci logging log list --log-group-id <OCID_OF_LOG_GROUP> --all
    • Confirm that at least one log corresponds to API server / audit events.

  4. Confirm logs are exported off-cluster to a tamper-resistant destination

    • In the OCI Console, for each relevant OKE control-plane / audit log:
      • Go to Logging → Logs → <the log> → Actions / Configure Log.
      • Check for an active Service Connector or Log Rule exporting to:
        • OCI Logging Analytics, or
        • OCI Object Storage bucket with restricted access, or
        • OCI Logging to external SIEM / third-party sink.
    • With OCI CLI on any machine, list service connectors: 
      oci sch service-connector list --compartment-id <OCID_OF_COMPARTMENT> --all
    • Inspect each connector’s source to confirm it pulls from the OKE log(s), and target to confirm it writes to an external, controlled, and access-logged destination.

  5. If audit logging or export is missing, configure it in OCI

    • In the OCI Console, navigate to the OKE cluster’s details page and:
      • Enable Kubernetes API / audit logging if an option exists for the cluster.
    • Then, under Observability & Management → Logging:
      • Create or identify a log group collecting the OKE control-plane / audit logs.
    • Under Service Connectors:
      • Create a new service connector with:
        • Source: the OKE audit/control-plane log(s).
        • Target: an OCI Object Storage bucket or Logging Analytics / external sink with write-only from logging services and tightly controlled read access.
    • If managing via IaC (Terraform), ensure corresponding oci_logging_log, oci_sch_service_connector, and, where supported, OKE cluster options for audit logging are defined and applied.

  6. Re-verify logging and export after configuration

    • Generate a few Kubernetes API events (for example, kubectl get pods -A) from any machine with kubectl access to the cluster.
    • In OCI Logging:
      • Open the OKE control-plane / audit log and confirm new entries appear matching the recent API calls.
    • In the configured external destination (Object Storage bucket, Logging Analytics, or external SIEM):
      • Confirm that newly generated audit log entries are present and that access controls on the destination prevent unauthorised modification or deletion.
Using kubectl

kubectl cannot enable or configure Kubernetes API audit logging for Oracle OKE clusters, because this setting is managed entirely in the Oracle Cloud Infrastructure (OCI) / OKE control-plane configuration and associated logging services. To address this finding, use the OCI Console, CLI, or IaC (Terraform/Resource Manager) as described in the Manual Steps section.

Automation
#!/usr/bin/env bash
#
# audit-oke-api-logging.sh
#
# Purpose:
#   Help review whether Oracle OKE clusters have API server audit logging
#   enabled and shipped off-cluster.
#
# Scope:
#   - Runs on: any machine with:
#       * OCI CLI configured (with permissions to list & inspect OKE clusters,
#         logging configs, log groups, and service connectors)
#       * jq
#
# Notes:
#   - This does NOT change any configuration.
#   - This is a best‑effort helper for a MANUAL control; you still must review
#     the output and make a judgment.
#
# Usage:
#   export OCI_COMPARTMENT_OCID="ocid1.compartment.oc1..xxxxx"
#   ./audit-oke-api-logging.sh
#
# Optional filters:
#   export OKE_NAME_FILTER="prod"   # only clusters whose display-name matches this regex
#
set -euo pipefail

if ! command -v oci >/dev/null 2>&1; then
  echo "ERROR: oci CLI not found in PATH" >&2
  exit 1
fi

if ! command -v jq >/dev/null 2>&1; then
  echo "ERROR: jq not found in PATH" >&2
  exit 1
fi

: "${OCI_COMPARTMENT_OCID:?ERROR: Set OCI_COMPARTMENT_OCID to the OCID of the compartment to scan}"

OKE_NAME_FILTER="${OKE_NAME_FILTER:-}"

echo "=== OKE API Audit Logging Review ==="
echo "Compartment: ${OCI_COMPARTMENT_OCID}"
[ -n "${OKE_NAME_FILTER}" ] && echo "Name filter (regex): ${OKE_NAME_FILTER}"
echo

# Helper: describe verdict text
explain_verdict() {
  cat <<'EOF'
Verdict meanings:
  - NO_CONTROL_PLANE_LOGS:
      No Logging configuration of type CONTROL_PLANE or APISERVER found.
      => API audit logging is very likely NOT enabled.
  - CONTROL_PLANE_LOGS_NO_DESTINATION:
      Cluster has control-plane logs configured, but no evidence of
      shipping to an external destination (no related service connectors).
      => Logs may be staying only in OCI Logging; assess if this meets
         your "external, tamper-resistant" requirement.
  - CONTROL_PLANE_LOGS_WITH_DESTINATION:
      Cluster has control-plane logs AND a service connector appears to
      route them to another sink (e.g., Object Storage, Logging Analytics,
      Stream).
      => Manual review still required to confirm this meets policy (e.g.,
         immutability, retention, access controls).
EOF
}

# Cache log groups and service connectors for the compartment (and child compartments)
echo "Fetching log groups and service connectors for analysis..." >&2

LOG_GROUPS_JSON="$(oci logging log-group list \
  --compartment-id "${OCI_COMPARTMENT_OCID}" \
  --all \
  --compartment-id-in-subtree true \
  --query 'data[].{id:id, name:"display-name", compartment:"compartment-id"}' \
  --raw-output)"

SERVICE_CONNECTORS_JSON="$(oci sch service-connector list \
  --compartment-id "${OCI_COMPARTMENT_OCID}" \
  --all \
  --compartment-id-in-subtree true \
  --query 'data[].{
    id:id,
    name:"display-name",
    state: "lifecycle-state",
    sourceType: "source.type",
    sourceLogGroupId: "source.logSources[0].logGroupId",
    targetKind: "target.kind",
    targetDescription: "target.description"
  }' \
  --raw-output || echo '[]')"

echo
echo "=== Clusters in compartment (and subcompartments) ==="
CLUSTERS_JSON="$(oci ce cluster list \
  --compartment-id "${OCI_COMPARTMENT_OCID}" \
  --all \
  --compartment-id-in-subtree true \
  --query 'data[].{
    id:id,
    name:"name",
    lifecycle:"lifecycle-state",
    endpointConfig:"kubernetes-endpoint-config"
  }' \
  --raw-output)"

if [ -n "${OKE_NAME_FILTER}" ]; then
  CLUSTERS_JSON="$(echo "${CLUSTERS_JSON}" | jq --arg re "${OKE_NAME_FILTER}" '[.[] | select(.name|test($re))]')"
fi

if [ "$(echo "${CLUSTERS_JSON}" | jq 'length')" -eq 0 ]; then
  echo "No clusters found for the given criteria."
  exit 0
fi

echo "${CLUSTERS_JSON}" | jq -r '.[] | "\(.name) (\(.id)) - state: \(.lifecycle)"'
echo

echo "=== Detailed API logging review per cluster ==="
echo

echo "${CLUSTERS_JSON}" | jq -c '.[]' | while read -r CL; do
  CL_ID="$(echo "${CL}" | jq -r '.id')"
  CL_NAME="$(echo "${CL}" | jq -r '.name')"
  CL_STATE="$(echo "${CL}" | jq -r '.lifecycle')"

  echo "-----"
  echo "Cluster: ${CL_NAME}"
  echo "OCID   : ${CL_ID}"
  echo "State  : ${CL_STATE}"

  # Skip non-active clusters but still show basic info
  if [ "${CL_STATE}" != "ACTIVE" ]; then
    echo "Verdict: SKIPPED_NON_ACTIVE (cluster not ACTIVE; inspect manually if needed)"
    echo
    continue
  fi

  # Get Logging configurations associated with this cluster
  LOGGING_CFGS="$(oci logging log list \
    --compartment-id "${OCI_COMPARTMENT_OCID}" \
    --all \
    --compartment-id-in-subtree true \
    --query "data[?contains(\"defined-tags\".\"Oracle-Kubernetes\".\"cluster-id\", '${CL_ID}')][]" \
    --raw-output || echo '[]')"

  CONTROL_PLANE_LOGS="$(echo "${LOGGING_CFGS}" | jq '[.[] | select(.type=="SERVICE" and (.\"configuration\".source.service=="kubernetes" or .\"configuration\".source.service=="oke"))]')"

  CP_COUNT="$(echo "${CONTROL_PLANE_LOGS}" | jq 'length')"

  if [ "${CP_COUNT}" -eq 0 ]; then
    echo "Control-plane / API-related logs: NONE detected for this cluster."
    echo "Verdict: NO_CONTROL_PLANE_LOGS"
    echo "Explanation: API audit logging is likely NOT enabled for this cluster."
    echo
    continue
  fi

  echo "Control-plane / API-related log objects:"
  echo "${CONTROL_PLANE_LOGS}" | jq -r '.[] | "- Log: \(.\"display-name\") | id: \(.id) | group: \(.\"log-group-id\") | state: \(.\"lifecycle-state\")"'

  # For each relevant log group, see if there is a service connector shipping it elsewhere
  DESTINATIONS_FOUND=0
  echo
  echo "Checking service connectors for these log groups..."

  echo "${CONTROL_PLANE_LOGS}" | jq -r '.[]."log-group-id"' | sort -u | while read -r LG_ID; do
    [ -z "${LG_ID}" ] && continue
    LG_INFO="$(echo "${LOG_GROUPS_JSON}" | jq --arg id "${LG_ID}" '[.[] | select(.id==$id)][0]')"
    LG_NAME="$(echo "${LG_INFO}" | jq -r '.name // "UNKNOWN"' 2>/dev/null || echo "UNKNOWN")"

    echo "  Log Group: ${LG_NAME} (${LG_ID})"

    # Find service connectors using this log group as source
    SC_MATCH="$(echo "${SERVICE_CONNECTORS_JSON}" | jq --arg lg "${LG_ID}" '[.[] | select(.sourceType=="logging" and .sourceLogGroupId==$lg and .state=="ACTIVE")]')"
    SC_COUNT="$(echo "${SC_MATCH}" | jq 'length')"

    if [ "${SC_COUNT}" -eq 0 ]; then
      echo "    -> NO active service connector found for this log group."
    else
      DESTINATIONS_FOUND=1
      echo "    -> Active service connectors:"
      echo "${SC_MATCH}" | jq -r '.[] | "       - \(.name) (\(.id)) | targetKind: \(.targetKind) | desc: \(.targetDescription)"'
    fi
  done

  if [ "${DESTINATIONS_FOUND}" -eq 0 ]; then
    echo
    echo "Verdict: CONTROL_PLANE_LOGS_NO_DESTINATION"
    echo "Explanation: Control-plane / API logs exist in OCI Logging, but"
    echo "             no active service connector appears to forward them"
    echo "             to an external/tamper-resistant destination."
  else
    echo
    echo "Verdict: CONTROL_PLANE_LOGS_WITH_DESTINATION"
    echo "Explanation: Control-plane / API logs are configured AND there"
    echo "             is at least one service connector forwarding the"
    echo "             log group(s) to another sink (e.g., Object Storage,"
    echo "             Logging Analytics, Stream). Manually verify that this"
    echo "             meets your 'external, tamper-resistant' requirement."
  fi

  echo
done

echo "==== Review Guidance ===="
explain_verdict

How to interpret output (what indicates a problem)

Run on any machine with oci and jq:

export OCI_COMPARTMENT_OCID="ocid1.compartment.oc1..xxxxx"
./audit-oke-api-logging.sh

Problem indicators to investigate and manually remediate:

  • Verdict: NO_CONTROL_PLANE_LOGS
    → The script found no OKE-related control-plane/API logs. API audit logging is likely not enabled.

  • Verdict: CONTROL_PLANE_LOGS_NO_DESTINATION
    → Control-plane/API logs exist only inside OCI Logging, and no active Service Connector was found that exports them. This usually means logs are not shipped to an independent, tamper-resistant store.

CONTROL_PLANE_LOGS_WITH_DESTINATION is better, but still requires manual review of the destination (e.g., Object Storage bucket policies, retention, access controls) to decide whether it satisfies “external, tamper-resistant” for your environment.