Skip to main content

Event Record Qps Argument Is Set To 0 Level Which Ensures

More Info:

Security relevant information should be captured. The eventRecordQPS on the Kubelet configuration can be used to limit the rate at which events are gathered and sets the maximum event creations per second. Setting this too low could result in relevant events not being logged, however the unlimited setting of 0 could result in a denial of service on the kubelet.

Risk Level

Medium

Address

Security

Compliance Standards

  • APRA CPS 234 (Australia)
  • BSI C5 (Germany)
  • Brazil LGPD
  • CCPA / CPRA (California)
  • CIS Critical Security Controls v8
  • CIS EKS
  • CMMC 2.0
  • CSA Cloud Controls Matrix v4
  • DPDPA
  • Digital Operational Resilience Act (EU)
  • ISO/IEC 27017
  • ISO/IEC 27018
  • ISO/IEC 27701
  • KSA PDPL
  • MAS Technology Risk Management (Singapore)
  • MITRE ATT&CK (Cloud)
  • NIS2 Directive
  • NIST SP 800-171
  • NYDFS 23 NYCRR 500
  • SWIFT Customer Security Controls Framework
  • Sarbanes-Oxley IT General Controls
  • UK NCSC Cyber Assessment Framework

Triage and Remediation

Remediation

Manual Steps
  1. On every worker node, back up the current Kubelet config file:

    sudo cp /var/lib/kubelet/config.yaml /var/lib/kubelet/config.yaml.bak.$(date +%F-%H%M%S)
  2. Edit /var/lib/kubelet/config.yaml and set eventRecordQPS to an appropriate non-zero level (example: 50). If the field does not exist, add it under the top-level KubeletConfiguration:

    sudo sed -i 's/^eventRecordQPS: .*$/eventRecordQPS: 50/' /var/lib/kubelet/config.yaml || \
    echo 'eventRecordQPS: 50' | sudo tee -a /var/lib/kubelet/config.yaml

    Review the file afterward to ensure eventRecordQPS: 50 is present only once and correctly indented for valid YAML:

    sudo cat /var/lib/kubelet/config.yaml
  3. If your kubelet is also configured via systemd arguments, verify that no conflicting --eventRecordQPS flag is present in the drop-in unit (adjust if needed):

    sudo sed -n '1,160p' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

    If you find a --eventRecordQPS flag there and you are using the config file, remove that flag from the line and save the file with a text editor (for example sudo vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf).

  4. Reload systemd and restart kubelet on that worker node (this will temporarily disrupt workloads on the node while kubelet restarts):

    sudo systemctl daemon-reload
    sudo systemctl restart kubelet.service
  5. Verify the kubelet is running with the desired configuration on that worker node by re-checking the process and confirming there is no conflicting --eventRecordQPS flag:

    /bin/ps -fC kubelet

    If the flag appears on the command line with an unexpected value, adjust the systemd unit as in step 3 and restart kubelet again.

  6. Repeat steps 1–5 on every worker node in the cluster to ensure consistent configuration.

Using kubectl

kubectl cannot modify kubelet host-level settings such as eventRecordQPS in /var/lib/kubelet/config.yaml or the kubelet systemd unit on worker nodes. To remediate this finding, make the changes directly on each worker node as described in the Manual Steps section.

Automation
#!/usr/bin/env bash
#
# Automation: Ensure kubelet eventRecordQPS is set to a safe, non-zero value
# Scope: run on every worker node (as root)
#
# This script:
# - Updates /var/lib/kubelet/config.yaml to set eventRecordQPS: 50 (adjust if needed)
# - Preserves file permissions/ownership
# - Reloads systemd and restarts kubelet
# - Verifies the setting via kubelet config dump
#
# Idempotent: safe to re-run; it will keep eventRecordQPS at 50.

set -euo pipefail

EVENT_QPS_VALUE=50
KUBELET_CONFIG="/var/lib/kubelet/config.yaml"
SYSTEMD_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"

echo "==> Running on worker node: $(hostname)"

if [[ "$EUID" -ne 0 ]]; then
echo "ERROR: run this script as root." >&2
exit 1
fi

if [[ ! -f "$KUBELET_CONFIG" ]]; then
echo "ERROR: kubelet config file not found at $KUBELET_CONFIG" >&2
echo "This node may be using only command-line flags; adjust manually in $SYSTEMD_DROPIN." >&2
exit 1
fi

echo "==> Backing up existing kubelet config"
cp -p "$KUBELET_CONFIG" "${KUBELET_CONFIG}.$(date +%Y%m%d%H%M%S).bak"

echo "==> Ensuring eventRecordQPS: $EVENT_QPS_VALUE in $KUBELET_CONFIG"

# If eventRecordQPS exists, replace its value; otherwise, add it under the top-level section.
if grep -qE '^[[:space:]]*eventRecordQPS:' "$KUBELET_CONFIG"; then
# Replace existing line (independent of indentation or previous value)
sed -i -E "s/^[[:space:]]*eventRecordQPS:.*/eventRecordQPS: ${EVENT_QPS_VALUE}/" "$KUBELET_CONFIG"
else
# Insert near the top, after any leading comments/shebang; keeps YAML valid
tmpfile="$(mktemp)"
inserted=0
while IFS='' read -r line || [[ -n "$line" ]]; do
if [[ $inserted -eq 0 ]] && [[ ! "$line" =~ ^[[:space:]]*# ]] && [[ -n "$line" ]]; then
echo "eventRecordQPS: ${EVENT_QPS_VALUE}" >> "$tmpfile"
inserted=1
fi
echo "$line" >> "$tmpfile"
done < "$KUBELET_CONFIG"

if [[ $inserted -eq 0 ]]; then
# File was empty or only comments; just append
echo "eventRecordQPS: ${EVENT_QPS_VALUE}" >> "$tmpfile"
fi

mv "$tmpfile" "$KUBELET_CONFIG"
fi

echo "==> Reloading systemd and restarting kubelet (this will restart kubelet)"
systemctl daemon-reload
systemctl restart kubelet.service

echo "==> Waiting for kubelet to stabilize"
sleep 10

echo "==> Verification: dumping kubelet config and checking eventRecordQPS"
if command -v kubelet &>/dev/null && kubelet --help 2>&1 | grep -q 'config'; then
# Use kubelet config dump if available
if kubelet --config="$KUBELET_CONFIG" --help &>/dev/null; then
:
fi
fi

# Primary verification: inspect the effective config file
if grep -qE "^[[:space:]]*eventRecordQPS:[[:space:]]*${EVENT_QPS_VALUE}\b" "$KUBELET_CONFIG"; then
echo "SUCCESS: eventRecordQPS is set to ${EVENT_QPS_VALUE} in ${KUBELET_CONFIG}"
else
echo "ERROR: eventRecordQPS not correctly set in ${KUBELET_CONFIG}" >&2
exit 1
fi

# Secondary verification: confirm kubelet process is running and using the config file
echo "==> Verifying kubelet process and config usage"
/bin/ps -fC kubelet || {
echo "ERROR: kubelet process not found after restart" >&2
exit 1
}

echo "==> Done."

Additional Reading: