Skip to main content

GCP Enable Linux Audit Logging

More Info:​

Run The Auditd Logging Daemon To Obtain Verbose Operating System Logs From Gke Nodes Running Container-Optimized Os (Cos).

Risk Level​

Medium

Address​

Security

Compliance Standards​

  • CIS GKE
  • Securities and Exchange Board of India (SEBI) - Cloud Security Adoption Framework

Triage and Remediation​

Remediation​

Manual Steps

  1. Review current logging/OS and decide if auditd applies

    • Confirm your nodes run Container-Optimized OS (COS):
      • On any machine with kubectl access: 
        kubectl get nodes -o wide
      • Check the OS-IMAGE column for COS. If nodes are not COS, this specific remediation may not be applicable.

  2. Check whether an auditd DaemonSet is already deployed

    • On any machine with kubectl access: 
      kubectl get daemonsets -A -o json \
      | jq '.items[] | select (.spec.template.spec.containers[].image | contains ("gcr.io/stackdriver-agents/stackdriver-logging-agent"))' \
      | jq '{name: .metadata.name, annotations: .metadata.annotations."kubernetes.io/description", namespace: .metadata.namespace, status: .status}'
    • If this returns a DaemonSet that is running and clearly annotated/described as COS auditd logging (or equivalent) in all relevant node pools, you may consider this control satisfied. Otherwise, proceed.

  3. Decide namespace, labels, and any customizations for audit logging

    • Determine:
      • Namespace you want to use for the auditd logging DaemonSet (default from example is cos-auditd).
      • Any label selectors needed if you do not want auditd on every node (e.g., only certain node pools).
    • This decision is environmental and policy-driven; document it for future audits.

  4. Download and review the reference manifests

    • On any machine with kubectl access: 
      curl https://raw.githubusercontent.com/GoogleCloudPlatform/k8s-node-tools/master/os-audit/cos-auditd-logging.yaml > cos-auditd-logging.yaml
    • Open and review cos-auditd-logging.yaml to confirm:
      • Namespace is what you decided in step 3.
      • DaemonSet tolerations, node selectors, and resource requests/limits fit your node configuration.
      • Log destination and integration with your existing logging pipeline meet your compliance and retention needs.
    • Adjust the manifest as needed to align with your environment and policies.

  5. Deploy or update the auditd logging DaemonSet

    • On any machine with kubectl access: 
      kubectl apply -f cos-auditd-logging.yaml
    • This will create/update the namespace, service account, RBAC, and DaemonSet that runs the auditd logging agent on COS nodes.

  6. Verify auditd logging is running and healthy

    • On any machine with kubectl access (adjust namespace if you changed it): 
      kubectl get pods --namespace=cos-auditd -o wide
    • Confirm there is one running pod per COS node (or per targeted nodes if you customized selectors) and that pods are in Running state with READY containers.
    • Optionally re-run the audit query from step 2 to ensure the DaemonSet is now present and healthy.
Using kubectl

kubectl cannot change this setting because it is managed at the cloud provider / managed control plane configuration layer, not via direct Kubernetes API edits. To address this finding, follow the guidance in the Manual Steps section and make the required changes through your cloud provider configuration and associated manifests.

Automation
#!/usr/bin/env bash
# Check status of COS auditd logging across the cluster
# Run on: any machine with kubectl access and jq installed

set -euo pipefail

echo "== Checking for COS auditd logging DaemonSets (cos-auditd) =="

# 1. List all namespaces and DaemonSets that look like COS auditd logging
echo
echo "-- Potential COS auditd DaemonSets (by name/namespace) --"
kubectl get daemonsets -A -o wide | \
  awk 'NR==1 || $2 ~ /cos-audit|auditd|os-audit|cos-auditd/'

# 2. Find DaemonSets that use the documented COS auditd manifest images
#    (update image patterns here if you use a custom image)
echo
echo "-- DaemonSets using COS auditd / node audit logging images --"
kubectl get daemonsets -A -o json | jq -r '
  .items[]
  | {
      ns: .metadata.namespace,
      name: .metadata.name,
      images: [ .spec.template.spec.containers[].image ],
      desired: .status.desiredNumberScheduled,
      ready: .status.numberReady
    }
  | select(
      ([.images[] | tostring]
        | map(
            test("cos-auditd-logging|os-audit|auditd-logging|node-audit"; "i")
          )
        | any)
    )
  | "namespace=\(.ns)\tname=\(.name)\timages=\(.images | join(","))\tdesired=\(.desired)\tready=\(.ready)"
'

# 3. Show node OS images to help scope where COS auditd is expected
echo
echo "-- Node OS summary (to identify Container-Optimized OS nodes) --"
kubectl get nodes -o json | jq -r '
  .items[]
  | {
      name: .metadata.name,
      osImage: .status.nodeInfo.osImage
    }
  | "node=\(.name)\tosImage=\(.osImage)"
' | sort

# 4. Correlate COS nodes with auditd DaemonSets actually scheduled there
echo
echo "-- Pods from auditd-style DaemonSets on COS nodes --"
kubectl get pods -A -o json | jq -r '
  def looks_like_auditd:
    (.metadata.ownerReferences // [])
    | map(select(.kind=="DaemonSet"))
    | map(.name)
    | map(test("cos-audit|auditd|os-audit|cos-auditd"; "i"))
    | any;

  .items[]
  | select(looks_like_auditd)
  | {
      ns: .metadata.namespace,
      name: .metadata.name,
      node: .spec.nodeName,
      phase: .status.phase,
      reason: .status.reason,
      osImage: ( .spec.nodeName as $n
        | input_filename as $f
        | ""
      )
    }
' 2>/dev/null || true

echo
echo "== INTERPRETING RESULTS =="
cat <<'EOF'
Potential problems that need human review:

1) No matching COS auditd DaemonSet:
   - If the section "DaemonSets using COS auditd / node audit logging images" is EMPTY,
     then auditd logging is likely NOT deployed cluster-wide.
   - Compare this to the CIS GKE example manifest (cos-auditd-logging.yaml). If you use a
     different name or image, update the image/name patterns in this script accordingly.

2) DaemonSet not running on all nodes:
   - In the DaemonSet summary, if 'desired' != 'ready', some nodes are not running
     the auditd logging pod. Investigate with:
       kubectl describe daemonset <name> -n <namespace>
       kubectl get pods -n <namespace> -o wide

3) Missing logging on COS nodes:
   - From the node OS summary, identify nodes running Container-Optimized OS.
   - Ensure each COS node has a running pod from the auditd DaemonSet (kubectl get pods -A -o wide).
   - If COS nodes exist but no auditd pods are scheduled there, this is a gap vs. the benchmark.

Because this is a MANUAL control, you must decide whether your current logging solution
(mechanism, images, and coverage) is equivalent to or better than the reference
cos-auditd-logging.yaml and document that decision.
EOF

Additional Reading:​