Skip to main content

Kubelet Streaming Connection Idle Timeout Not Set To 0

More Info:

The kubelet --streaming-connection-idle-timeout argument should not be set to 0. A zero timeout leaves idle streaming connections open indefinitely, enabling denial-of-service.

Risk Level

High

Address

Security

Compliance Standards

  • CIS OKE

Triage and Remediation

Remediation

Manual Steps
  1. On every worker node, check current kubelet process flags to confirm the issue:

    /bin/ps -fC kubelet

    Look for --streaming-connection-idle-timeout=0 in the output.

  2. On every worker node, open the kubelet systemd drop-in configuration for editing:

    sudo vi /etc/systemd/system/kubelet.service.d/00-default.conf
  3. In that file, locate the KUBELET_KUBECONFIG_ARGS, KUBELET_ARGS, or the ExecStart= line that contains kubelet flags. Ensure the --streaming-connection-idle-timeout flag is either:

    • removed entirely (to use the default), or
    • set to a non-zero duration, for example:
      --streaming-connection-idle-timeout=4h

    Save and exit the editor.

  4. If /etc/kubernetes/kubelet-config.json is referenced by kubelet (e.g. via --config=/etc/kubernetes/kubelet-config.json), ensure it does not explicitly set the timeout to 0. Edit on every worker node:

    sudo vi /etc/kubernetes/kubelet-config.json

    If present, change:

    "streamingConnectionIdleTimeout": "0s"

    to a non-zero duration, for example:

    "streamingConnectionIdleTimeout": "4h"

    or remove the line to rely on defaults. Save the file.

  5. On every worker node, reload systemd and restart kubelet (this will temporarily disrupt workloads handled by that node’s kubelet):

    sudo systemctl daemon-reload
    sudo systemctl restart kubelet.service
    sudo systemctl status kubelet -l
  6. Verify on every worker node that kubelet no longer uses a zero timeout:

    /bin/ps -fC kubelet

    Confirm there is no --streaming-connection-idle-timeout=0 flag and, if present, it is set to a non-zero value.

Using kubectl

kubectl cannot modify kubelet process flags or the /etc/kubernetes/kubelet-config.json file on worker nodes. To remediate this finding, you must update the kubelet systemd configuration on each worker node and restart kubelet as described in the Manual Steps section.

Automation
#!/usr/bin/env bash
#
# Fix CIS OKE 3.2.5: Ensure kubelet --streaming-connection-idle-timeout is not set to 0
#
# Run on: every worker node (as root)
# Safe to re-run: yes

set -euo pipefail

KUBELET_DROPIN_DIR="/etc/systemd/system/kubelet.service.d"
KUBELET_DROPIN_FILE="${KUBELET_DROPIN_DIR}/00-default.conf"
BACKUP_SUFFIX="$(date +%Y%m%d%H%M%S)"
DESIRED_TIMEOUT="4h" # non-zero; adjust if your standard is different

echo "[INFO] Starting kubelet streaming-connection-idle-timeout remediation"

if [[ $EUID -ne 0 ]]; then
echo "[ERROR] This script must be run as root." >&2
exit 1
fi

if ! systemctl list-unit-files | grep -q '^kubelet\.service'; then
echo "[ERROR] kubelet.service not found on this node." >&2
exit 1
fi

# Ensure drop-in directory exists
mkdir -p "${KUBELET_DROPIN_DIR}"

if [[ ! -f "${KUBELET_DROPIN_FILE}" ]]; then
echo "[WARN] ${KUBELET_DROPIN_FILE} not found, creating a minimal drop-in."
cat > "${KUBELET_DROPIN_FILE}" <<'EOF'
[Service]
# Additional kubelet args can be specified here with KUBELET_EXTRA_ARGS
# Example:
# Environment="KUBELET_EXTRA_ARGS="
EOF
fi

echo "[INFO] Backing up ${KUBELET_DROPIN_FILE} to ${KUBELET_DROPIN_FILE}.${BACKUP_SUFFIX}.bak"
cp -p "${KUBELET_DROPIN_FILE}" "${KUBELET_DROPIN_FILE}.${BACKUP_SUFFIX}.bak"

# Ensure KUBELET_EXTRA_ARGS is present in the drop-in
if ! grep -q '^Environment="KUBELET_EXTRA_ARGS=' "${KUBELET_DROPIN_FILE}"; then
echo "[INFO] Adding KUBELET_EXTRA_ARGS environment line to ${KUBELET_DROPIN_FILE}"
printf '\nEnvironment="KUBELET_EXTRA_ARGS="\n' >> "${KUBELET_DROPIN_FILE}"
fi

# Edit KUBELET_EXTRA_ARGS to enforce a non-zero --streaming-connection-idle-timeout
tmpfile="$(mktemp)"
timeout_regex='--streaming-connection-idle-timeout(=[^" ]*)?'

while IFS= read -r line; do
if [[ "$line" =~ ^Environment=\"KUBELET_EXTRA_ARGS= ]]; then
# Strip leading prefix and trailing quote
prefix='Environment="KUBELET_EXTRA_ARGS='
current="${line#${prefix}}"
current="${current%\"}"

# Remove existing --streaming-connection-idle-timeout occurrences
# shellcheck disable=SC2001
cleaned="$(sed -E "s/${timeout_regex}//g" <<<"${current}")"
cleaned="$(xargs <<<"${cleaned}" || true)" # normalize spaces

# Append desired timeout
if [[ -z "${cleaned}" ]]; then
newval="--streaming-connection-idle-timeout=${DESIRED_TIMEOUT}"
else
newval="${cleaned} --streaming-connection-idle-timeout=${DESIRED_TIMEOUT}"
fi

echo "Environment=\"KUBELET_EXTRA_ARGS=${newval}\"" >> "${tmpfile}"
else
echo "${line}" >> "${tmpfile}"
fi
done < "${KUBELET_DROPIN_FILE}"

mv "${tmpfile}" "${KUBELET_DROPIN_FILE}"

echo "[INFO] Reloading systemd and restarting kubelet (this will restart kubelet and may briefly impact node workloads)."
systemctl daemon-reload
systemctl restart kubelet.service

echo "[INFO] Verifying kubelet status"
if ! systemctl is-active --quiet kubelet; then
echo "[ERROR] kubelet is not active after restart." >&2
systemctl status kubelet -l || true
exit 1
fi

echo "[INFO] Verifying --streaming-connection-idle-timeout is set and non-zero in running kubelet process"
if ! /bin/ps -fC kubelet | grep -q -- '--streaming-connection-idle-timeout'; then
echo "[ERROR] kubelet is running but --streaming-connection-idle-timeout flag is not present." >&2
/bin/ps -fC kubelet || true
exit 1
fi

if /bin/ps -fC kubelet | grep -q -- '--streaming-connection-idle-timeout=0'; then
echo "[ERROR] kubelet still has --streaming-connection-idle-timeout=0 set." >&2
/bin/ps -fC kubelet || true
exit 1
fi

echo "[INFO] Verification succeeded. Current kubelet command line:"
/bin/ps -fC kubelet

echo "[INFO] Remediation completed successfully on this node."