Skip to main content

Avoid Use Of System:masters Group

More Info:​

The system:masters group is hardcoded into the API server and bypasses RBAC entirely. Credentials granting membership in this group should be removed.

Risk Level​

High

Address​

Security

Compliance Standards​

  • CIS GKE

Triage and Remediation​

Remediation​

Manual Steps
  1. Identify certificates that request membership in system:masters

    • On any machine with kubectl access:
      kubectl get csr
    • Note any CSRs you see; the verification in step 6 will confirm if any are using O = system:masters.
  2. Inspect each suspicious CSR and confirm O = system:masters

    • Replace <csr-name> with each CSR from step 1:
      kubectl get csr <csr-name> -o jsonpath='{.spec.request}' | base64 -d | openssl req -noout -text | grep 'Subject:'
    • If the Subject contains O = system:masters, treat that credential as over‑privileged.
  3. Deny or delete any pending/approved CSRs using system:masters

    • For a CSR you want to explicitly deny:
      kubectl certificate deny <csr-name>
    • Or to delete it entirely:
      kubectl delete csr <csr-name>
  4. Revoke or remove any existing credentials that include O = system:masters outside of CSRs

    • Locate the client certificate/key file(s) used by that user or component (kubeconfig entries, TLS files on disk) and remove or replace them with new credentials that do not include O = system:masters.
    • This step is environment‑specific and may involve updating kubeconfig files and rotating certificates; ensure the replacement credentials are bound only to appropriate RBAC groups/ServiceAccounts.
  5. Ensure no ClusterRoleBindings are granting cluster‑admin via system:masters-style certs

    • On any machine with kubectl access:
      kubectl get clusterrolebindings -o yaml
    • For any binding that is intended for those removed certificates, remove or adjust the binding to use explicit users/groups/ServiceAccounts instead of relying on O = system:masters in certificates:
      kubectl edit clusterrolebinding <binding-name>
    • In the editor, remove or correct the subject that referenced that certificate identity.
  6. Verification: confirm no CSRs request system:masters

    • On any machine with kubectl access:
      found=0
      for csr in $(kubectl get csr -o name 2>/dev/null | sed 's|^.*/||'); do
      req=$(kubectl get csr "$csr" -o jsonpath='{.spec.request}' 2>/dev/null)
      [ -z "$req" ] && continue
      if echo "$req" | base64 -d 2>/dev/null | openssl req -noout -text 2>/dev/null | grep -q 'O = system:masters'; then
      conds=$(kubectl get csr "$csr" -o json | jq -r '[.status.conditions[]?.type] | join(",")')
      echo "FOUND_SYSTEM_MASTERS_CSR:${csr}:${conds:-NONE}"
      found=1
      fi
      done
      if [ "$found" -eq 0 ]; then
      echo "NO_SYSTEM_MASTERS_CREDENTIALS_FOUND"
      fi
    • The control is satisfied when only NO_SYSTEM_MASTERS_CREDENTIALS_FOUND is printed.
Using kubectl

On any machine with kubectl access:

  1. Identify CSRs requesting system:masters
kubectl get csr

For each CSR name you see, inspect it:

kubectl get csr <CSR_NAME> -o jsonpath='{.spec.request}' \
| base64 -d \
| openssl req -noout -text | grep 'O ='

If the output shows:

O = system:masters

that CSR is requesting system:masters and its resulting credential must not be used.

  1. Deny or delete offending CSRs

For CSRs that have not yet been approved and should not be used:

kubectl certificate deny <CSR_NAME>

Optionally also delete them:

kubectl delete csr <CSR_NAME>
  1. Revoke already-approved system:masters credentials

For CSRs already approved (look for Approved in .status.conditions):

kubectl get csr <CSR_NAME> -o json | jq -r '.status.certificate' \
| base64 -d > /tmp/<CSR_NAME>.crt

Use your PKI/CA tooling to identify and revoke the corresponding certificate at the certificate authority. Then ensure any kubeconfig or secret that uses this certificate is removed or rotated. This step cannot be completed with kubectl alone; follow your PKI’s revocation process.

  1. Prevent future issuance with O=system:masters

Update your certificate issuance process (outside of kubectl) so that CSRs for Kubernetes client certs do not set O=system:masters. For any remaining automation that still creates such CSRs, disable or fix it, then deny/delete those CSRs as in step 2.

  1. Verification

Run the benchmark audit logic with kubectl (on any machine with kubectl access):

found=0
for csr in $(kubectl get csr -o name 2>/dev/null | sed 's|^.*/||'); do
req=$(kubectl get csr "$csr" -o jsonpath='{.spec.request}' 2>/dev/null)
[ -z "$req" ] && continue
if echo "$req" | base64 -d 2>/dev/null | openssl req -noout -text 2>/dev/null | grep -q 'O = system:masters'; then
conds=$(kubectl get csr "$csr" -o json | jq -r '[.status.conditions[]?.type] | join(",")')
echo "FOUND_SYSTEM_MASTERS_CSR:${csr}:${conds:-NONE}"
found=1
fi
done
if [ "$found" -eq 0 ]; then
echo "NO_SYSTEM_MASTERS_CREDENTIALS_FOUND"
fi
Automation
#!/usr/bin/env bash
set -euo pipefail

# Automation to detect and deny use of system:masters in CSRs.
# Runs on: any machine with kubectl access to the cluster.
#
# Behavior:
# - Finds all CSRs whose certificate request has O = system:masters
# - For each:
# * If already Denied or Approved: leaves as-is (for audit only)
# * If Pending: Denies the CSR to prevent issuing new system:masters creds
# - Prints a summary and re-runs the audit logic at the end.
#
# Safe to re-run: previously denied CSRs stay denied; no other objects are touched.

# Ensure required tools are available
for bin in kubectl base64 openssl jq; do
if ! command -v "$bin" >/dev/null 2>&1; then
echo "ERROR: required binary '$bin' is not installed or not in PATH" >&2
exit 1
fi
done

echo "Scanning for CSRs that request membership in system:masters..."

found=0
denied_new=0

# Get all CSR names (short names, e.g. 'csr-abc123')
mapfile -t CSRS < <(kubectl get csr -o name 2>/dev/null | sed 's|^.*/||')

for csr in "${CSRS[@]}"; do
# Fetch the base64-encoded CSR request
req=$(kubectl get csr "$csr" -o jsonpath='{.spec.request}' 2>/dev/null || echo "")
[ -z "$req" ] && continue

# Decode and inspect for O = system:masters
if echo "$req" | base64 -d 2>/dev/null \
| openssl req -noout -text 2>/dev/null \
| grep -q 'O = system:masters'; then

found=1

# Determine current conditions on the CSR
conds=$(kubectl get csr "$csr" -o json 2>/dev/null \
| jq -r '[.status.conditions[]?.type] | join(",")' 2>/dev/null)
conds=${conds:-NONE}

echo "FOUND_SYSTEM_MASTERS_CSR:${csr}:${conds}"

# If already Approved or Denied, do not change it; only operate on Pending CSRs
if [[ "$conds" == *"Approved"* || "$conds" == *"Denied"* ]]; then
continue
fi

echo "Denying pending CSR $csr that requests system:masters..."
# Deny the CSR (kubernetes >= 1.19 supports `kubectl certificate deny`)
if kubectl certificate deny "$csr" >/dev/null 2>&1; then
denied_new=$((denied_new + 1))
echo "Denied CSR $csr"
else
echo "WARNING: failed to deny CSR $csr; please review manually." >&2
fi
fi
done

if [ "$found" -eq 0 ]; then
echo "NO_SYSTEM_MASTERS_CREDENTIALS_FOUND"
else
echo "Total CSRs with system:masters requested: $found"
echo "Newly denied CSRs in this run: $denied_new"
fi

echo
echo "Verification: re-running audit to confirm no pending system:masters CSRs remain..."

# Verification uses the benchmark audit logic, but also reports status
verify_found=0
for csr in "${CSRS[@]}"; do
req=$(kubectl get csr "$csr" -o jsonpath='{.spec.request}' 2>/dev/null || echo "")
[ -z "$req" ] && continue

if echo "$req" | base64 -d 2>/dev/null \
| openssl req -noout -text 2>/dev/null \
| grep -q 'O = system:masters'; then

conds=$(kubectl get csr "$csr" -o json 2>/dev/null \
| jq -r '[.status.conditions[]?.type] | join(",")' 2>/dev/null)
conds=${conds:-NONE}

echo "VERIFY_SYSTEM_MASTERS_CSR:${csr}:${conds}"
verify_found=1
fi
done

if [ "$verify_found" -eq 0 ]; then
echo "VERIFICATION: NO_SYSTEM_MASTERS_CREDENTIALS_FOUND"
else
echo "VERIFICATION: system:masters CSRs still exist; ensure none remain Pending."
fi