Kubelet Streaming Connection Idle Timeout Not Set To 0
More Info:
The kubelet --streaming-connection-idle-timeout argument should not be set to 0. A zero timeout leaves idle streaming connections open indefinitely, enabling denial-of-service.
Risk Level
High
Address
Security
Compliance Standards
- CIS OKE
Triage and Remediation
- Remediation
Remediation
Manual Steps
-
On every worker node, check current kubelet process flags to confirm the issue:
/bin/ps -fC kubeletLook for
--streaming-connection-idle-timeout=0in the output. -
On every worker node, open the kubelet systemd drop-in configuration for editing:
sudo vi /etc/systemd/system/kubelet.service.d/00-default.conf -
In that file, locate the
KUBELET_KUBECONFIG_ARGS,KUBELET_ARGS, or theExecStart=line that contains kubelet flags. Ensure the--streaming-connection-idle-timeoutflag is either:- removed entirely (to use the default), or
- set to a non-zero duration, for example:
--streaming-connection-idle-timeout=4h
Save and exit the editor.
-
If
/etc/kubernetes/kubelet-config.jsonis referenced by kubelet (e.g. via--config=/etc/kubernetes/kubelet-config.json), ensure it does not explicitly set the timeout to 0. Edit on every worker node:sudo vi /etc/kubernetes/kubelet-config.jsonIf present, change:
"streamingConnectionIdleTimeout": "0s"to a non-zero duration, for example:
"streamingConnectionIdleTimeout": "4h"or remove the line to rely on defaults. Save the file.
-
On every worker node, reload systemd and restart kubelet (this will temporarily disrupt workloads handled by that node’s kubelet):
sudo systemctl daemon-reloadsudo systemctl restart kubelet.servicesudo systemctl status kubelet -l -
Verify on every worker node that kubelet no longer uses a zero timeout:
/bin/ps -fC kubeletConfirm there is no
--streaming-connection-idle-timeout=0flag and, if present, it is set to a non-zero value.
Using kubectl
kubectl cannot modify kubelet process flags or the /etc/kubernetes/kubelet-config.json file on worker nodes. To remediate this finding, you must update the kubelet systemd configuration on each worker node and restart kubelet as described in the Manual Steps section.
Automation
#!/usr/bin/env bash
#
# Fix CIS OKE 3.2.5: Ensure kubelet --streaming-connection-idle-timeout is not set to 0
#
# Run on: every worker node (as root)
# Safe to re-run: yes
set -euo pipefail
KUBELET_DROPIN_DIR="/etc/systemd/system/kubelet.service.d"
KUBELET_DROPIN_FILE="${KUBELET_DROPIN_DIR}/00-default.conf"
BACKUP_SUFFIX="$(date +%Y%m%d%H%M%S)"
DESIRED_TIMEOUT="4h" # non-zero; adjust if your standard is different
echo "[INFO] Starting kubelet streaming-connection-idle-timeout remediation"
if [[ $EUID -ne 0 ]]; then
echo "[ERROR] This script must be run as root." >&2
exit 1
fi
if ! systemctl list-unit-files | grep -q '^kubelet\.service'; then
echo "[ERROR] kubelet.service not found on this node." >&2
exit 1
fi
# Ensure drop-in directory exists
mkdir -p "${KUBELET_DROPIN_DIR}"
if [[ ! -f "${KUBELET_DROPIN_FILE}" ]]; then
echo "[WARN] ${KUBELET_DROPIN_FILE} not found, creating a minimal drop-in."
cat > "${KUBELET_DROPIN_FILE}" <<'EOF'
[Service]
# Additional kubelet args can be specified here with KUBELET_EXTRA_ARGS
# Example:
# Environment="KUBELET_EXTRA_ARGS="
EOF
fi
echo "[INFO] Backing up ${KUBELET_DROPIN_FILE} to ${KUBELET_DROPIN_FILE}.${BACKUP_SUFFIX}.bak"
cp -p "${KUBELET_DROPIN_FILE}" "${KUBELET_DROPIN_FILE}.${BACKUP_SUFFIX}.bak"
# Ensure KUBELET_EXTRA_ARGS is present in the drop-in
if ! grep -q '^Environment="KUBELET_EXTRA_ARGS=' "${KUBELET_DROPIN_FILE}"; then
echo "[INFO] Adding KUBELET_EXTRA_ARGS environment line to ${KUBELET_DROPIN_FILE}"
printf '\nEnvironment="KUBELET_EXTRA_ARGS="\n' >> "${KUBELET_DROPIN_FILE}"
fi
# Edit KUBELET_EXTRA_ARGS to enforce a non-zero --streaming-connection-idle-timeout
tmpfile="$(mktemp)"
timeout_regex='--streaming-connection-idle-timeout(=[^" ]*)?'
while IFS= read -r line; do
if [[ "$line" =~ ^Environment=\"KUBELET_EXTRA_ARGS= ]]; then
# Strip leading prefix and trailing quote
prefix='Environment="KUBELET_EXTRA_ARGS='
current="${line#${prefix}}"
current="${current%\"}"
# Remove existing --streaming-connection-idle-timeout occurrences
# shellcheck disable=SC2001
cleaned="$(sed -E "s/${timeout_regex}//g" <<<"${current}")"
cleaned="$(xargs <<<"${cleaned}" || true)" # normalize spaces
# Append desired timeout
if [[ -z "${cleaned}" ]]; then
newval="--streaming-connection-idle-timeout=${DESIRED_TIMEOUT}"
else
newval="${cleaned} --streaming-connection-idle-timeout=${DESIRED_TIMEOUT}"
fi
echo "Environment=\"KUBELET_EXTRA_ARGS=${newval}\"" >> "${tmpfile}"
else
echo "${line}" >> "${tmpfile}"
fi
done < "${KUBELET_DROPIN_FILE}"
mv "${tmpfile}" "${KUBELET_DROPIN_FILE}"
echo "[INFO] Reloading systemd and restarting kubelet (this will restart kubelet and may briefly impact node workloads)."
systemctl daemon-reload
systemctl restart kubelet.service
echo "[INFO] Verifying kubelet status"
if ! systemctl is-active --quiet kubelet; then
echo "[ERROR] kubelet is not active after restart." >&2
systemctl status kubelet -l || true
exit 1
fi
echo "[INFO] Verifying --streaming-connection-idle-timeout is set and non-zero in running kubelet process"
if ! /bin/ps -fC kubelet | grep -q -- '--streaming-connection-idle-timeout'; then
echo "[ERROR] kubelet is running but --streaming-connection-idle-timeout flag is not present." >&2
/bin/ps -fC kubelet || true
exit 1
fi
if /bin/ps -fC kubelet | grep -q -- '--streaming-connection-idle-timeout=0'; then
echo "[ERROR] kubelet still has --streaming-connection-idle-timeout=0 set." >&2
/bin/ps -fC kubelet || true
exit 1
fi
echo "[INFO] Verification succeeded. Current kubelet command line:"
/bin/ps -fC kubelet
echo "[INFO] Remediation completed successfully on this node."