Skip to main content

Kubelet Streaming Connection Idle Timeout Not Set To Zero

More Info:​

The kubelet streamingConnectionIdleTimeout should not be 0, which disables the timeout and leaves idle streaming connections open indefinitely.

Risk Level​

High

Address​

Security

Compliance Standards​

  • CIS AKS

Triage and Remediation​

Remediation​

Manual Steps
  1. Check current kubelet config (per worker node)
    Run on every worker node:

    ps -fC kubelet

    Note whether --config is used (config file) and whether --streaming-connection-idle-timeout=0 appears as a flag.

  2. If using kubelet config file, set a non‑zero timeout
    Run on every worker node (only if /var/lib/kubelet/config.yaml is in use):

    sudo sed -i 's/^streamingConnectionIdleTimeout: 0$/streamingConnectionIdleTimeout: 5m/' /var/lib/kubelet/config.yaml

    If the key does not exist, append it:

    sudo sh -c 'echo "streamingConnectionIdleTimeout: 5m" >> /var/lib/kubelet/config.yaml'
  3. If using command‑line flags, set a non‑zero timeout
    Run on every worker node (only if kubelet is configured via systemd flags):
    Edit the drop‑in unit:

    sudo nano /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

    In the KUBELET_SYSTEM_PODS_ARGS (or corresponding) environment variable, ensure this flag is present and not 0, for example:

    --streaming-connection-idle-timeout=5m

    Save and exit.

  4. Reload systemd and restart kubelet
    Run on every worker node:

    sudo systemctl daemon-reload
    sudo systemctl restart kubelet.service
  5. Verify kubelet restarted cleanly
    Run on every worker node:

    sudo systemctl status kubelet.service --no-pager
  6. Verify non‑zero streaming idle timeout is in effect
    Run on every worker node:

    ps -fC kubelet

    Confirm that --streaming-connection-idle-timeout is either absent (and handled via config file) or present with a value other than 0, and if using /var/lib/kubelet/config.yaml, confirm:

    grep -E '^streamingConnectionIdleTimeout:' /var/lib/kubelet/config.yaml

    shows a non‑zero value (for example 5m).

Using kubectl

kubectl cannot modify kubelet process flags or its config file, so this finding cannot be fixed via the Kubernetes API. The streamingConnectionIdleTimeout setting must be changed directly on each worker node’s host configuration (for example /var/lib/kubelet/config.yaml or the kubelet systemd unit); see the Manual Steps section for how to do that.

Automation
#!/usr/bin/env bash
#
# Remediates CIS AKS 3.2.5 on every worker node:
# - Ensures kubelet streamingConnectionIdleTimeout is not 0
# - Sets it to 5m in /var/lib/kubelet/config.yaml when present
# - Falls back to adding --streaming-connection-idle-timeout=5m to
# /etc/systemd/system/kubelet.service.d/10-kubeadm.conf when needed
#
# Run on: every worker node (as root)
# Safe to re-run: yes

set -euo pipefail

KUBELET_CONFIG="/var/lib/kubelet/config.yaml"
KUBELET_UNIT_DROPIN="/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
DESIRED_TIMEOUT="5m"

echo "=== CIS AKS 3.2.5 remediation: kubelet streamingConnectionIdleTimeout !== 0 ==="

backup_file() {
local file="$1"
if [ -f "$file" ]; then
local ts
ts="$(date +%Y%m%d%H%M%S)"
cp -a "$file" "${file}.cis_backup_${ts}"
echo "Backed up $file to ${file}.cis_backup_${ts}"
fi
}

restart_kubelet() {
echo "Reloading systemd and restarting kubelet..."
systemctl daemon-reload
systemctl restart kubelet.service
}

needs_restart=0

############################################
# 1. Prefer config file: /var/lib/kubelet/config.yaml
############################################
if [ -f "$KUBELET_CONFIG" ]; then
echo "Detected kubelet config file: $KUBELET_CONFIG"

backup_file "$KUBELET_CONFIG"

# If key exists, update it; else append it at the end.
if grep -qE '^[[:space:]]*streamingConnectionIdleTimeout[[:space:]]*:' "$KUBELET_CONFIG"; then
# If already desired and not 0, no change
if grep -qE "^[[:space:]]*streamingConnectionIdleTimeout[[:space:]]*:[[:space:]]*\"?$DESIRED_TIMEOUT\"?[[:space:]]*$" "$KUBELET_CONFIG"; then
echo "streamingConnectionIdleTimeout already set to $DESIRED_TIMEOUT in $KUBELET_CONFIG"
else
echo "Updating streamingConnectionIdleTimeout to $DESIRED_TIMEOUT in $KUBELET_CONFIG"
# Normalize to quoted value for safety
sed -i \
-E "s|^[[:space:]]*streamingConnectionIdleTimeout[[:space:]]*:.*$|streamingConnectionIdleTimeout: \"$DESIRED_TIMEOUT\"|" \
"$KUBELET_CONFIG"
needs_restart=1
fi
else
echo "Adding streamingConnectionIdleTimeout: \"$DESIRED_TIMEOUT\" to $KUBELET_CONFIG"
printf '\nstreamingConnectionIdleTimeout: "%s"\n' "$DESIRED_TIMEOUT" >> "$KUBELET_CONFIG"
needs_restart=1
fi
else
echo "No kubelet config file at $KUBELET_CONFIG; falling back to systemd drop-in."

############################################
# 2. Fallback to command-line flag in 10-kubeadm.conf
############################################
if [ ! -f "$KUBELET_UNIT_DROPIN" ]; then
echo "ERROR: $KUBELET_UNIT_DROPIN not found. Cannot configure kubelet flags automatically."
echo "Please create or edit $KUBELET_UNIT_DROPIN to include:"
echo " --streaming-connection-idle-timeout=$DESIRED_TIMEOUT"
exit 1
fi

backup_file "$KUBELET_UNIT_DROPIN"

# Remove any existing flag occurrences to avoid duplicates/conflicts
if grep -q -- '--streaming-connection-idle-timeout' "$KUBELET_UNIT_DROPIN"; then
echo "Removing existing --streaming-connection-idle-timeout flags from $KUBELET_UNIT_DROPIN"
sed -i 's/--streaming-connection-idle-timeout=[^[:space:]]*//g' "$KUBELET_UNIT_DROPIN"
fi

# Ensure flag is present in the KUBELET_SYSTEM_PODS_ARGS line if it exists,
# otherwise append it to all ExecStart lines as a last resort.
if grep -q '^Environment=.*KUBELET_SYSTEM_PODS_ARGS' "$KUBELET_UNIT_DROPIN"; then
echo "Adding --streaming-connection-idle-timeout=$DESIRED_TIMEOUT to KUBELET_SYSTEM_PODS_ARGS"
sed -i \
"s|^\(Environment=.*KUBELET_SYSTEM_PODS_ARGS=\"[^\"]*\)\"|\1 --streaming-connection-idle-timeout=$DESIRED_TIMEOUT\"|" \
"$KUBELET_UNIT_DROPIN"
elif grep -q '^ExecStart=' "$KUBELET_UNIT_DROPIN"; then
echo "Adding --streaming-connection-idle-timeout=$DESIRED_TIMEOUT to ExecStart command(s)"
sed -i \
"s|^ExecStart=\(.*kubelet.*\)|ExecStart=\1 --streaming-connection-idle-timeout=$DESIRED_TIMEOUT|" \
"$KUBELET_UNIT_DROPIN"
else
echo "ERROR: Could not locate KUBELET_SYSTEM_PODS_ARGS or ExecStart in $KUBELET_UNIT_DROPIN."
echo "Please manually add:"
echo " --streaming-connection-idle-timeout=$DESIRED_TIMEOUT"
exit 1
fi

needs_restart=1
fi

if [ "$needs_restart" -eq 1 ]; then
restart_kubelet
else
echo "No kubelet restart needed; configuration already compliant."
fi

############################################
# 3. Verification (CIS audit adaptation)
############################################
echo "Verifying kubelet process flags and config..."
/bin/ps -fC kubelet || {
echo "ERROR: kubelet process not found."
exit 1
}

echo
echo "Current kubelet process line:"
/bin/ps -fC kubelet

echo
if [ -f "$KUBELET_CONFIG" ]; then
echo "Current streamingConnectionIdleTimeout in $KUBELET_CONFIG:"
grep -E '^[[:space:]]*streamingConnectionIdleTimeout[[:space:]]*:' "$KUBELET_CONFIG" || \
echo "streamingConnectionIdleTimeout not found in $KUBELET_CONFIG"
fi

echo
echo "Checking that streamingConnectionIdleTimeout is not 0 and/or flag is present..."
if [ -f "$KUBELET_CONFIG" ]; then
if grep -Eq '^[[:space:]]*streamingConnectionIdleTimeout[[:space:]]*:[[:space:]]*"?0"?[[:space:]]*$' "$KUBELET_CONFIG"; then
echo "NON-COMPLIANT: streamingConnectionIdleTimeout is still 0 in $KUBELET_CONFIG"
exit 1
fi
fi

if /bin/ps -fC kubelet | grep -q -- '--streaming-connection-idle-timeout=0'; then
echo "NON-COMPLIANT: kubelet still running with --streaming-connection-idle-timeout=0"
exit 1
fi

echo "Verification passed: kubelet streamingConnectionIdleTimeout is not 0 (CIS AKS 3.2.5)."