Tenant Namespaces Should Have A ResourceQuota

More Info:

Advisory: create a ResourceQuota per tenant namespace to bound aggregate CPU, memory and object counts, preventing one tenant from starving others.

Risk Level

Low

Address

Security

Compliance Standards

Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Manual Steps

Identify tenant namespaces (run on any machine with kubectl access):

kubectl get ns \
  --no-headers \
  | grep -Ev '^(kube-system|kube-public|kube-node-lease)\s' \
  | awk '{print $1}'

For each tenant namespace (replace TENANT_NAMESPACE with the real name), create a ResourceQuota limiting aggregate CPU, memory, and object counts (run on any machine with kubectl access):

cat << 'EOF' | kubectl apply -f -
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: TENANT_NAMESPACE
spec:
  hard:
    requests.cpu: "4"
    requests.memory: "8Gi"
    limits.cpu: "8"
    limits.memory: "16Gi"
    pods: "50"
    services: "20"
    configmaps: "50"
    persistentvolumeclaims: "20"
    secrets: "100"
    services.loadbalancers: "5"
    services.nodeports: "5"
EOF

Adjust the hard values to match your tenant’s expected usage and SLOs before running.

If different tenants need different limits, create separate manifests per namespace with tuned values (run on any machine with kubectl access):

kubectl -n TENANT_NAMESPACE get resourcequota tenant-quota -o yaml > tenant-quota-TENANT_NAMESPACE.yaml
# edit tenant-quota-TENANT_NAMESPACE.yaml to adjust .spec.hard values
kubectl apply -f tenant-quota-TENANT_NAMESPACE.yaml

(Optional) Confirm per-namespace enforcement details for a specific tenant (run on any machine with kubectl access):
```
kubectl -n TENANT_NAMESPACE describe resourcequota tenant-quota
```

Verification: confirm that every tenant namespace now has at least one ResourceQuota (run on any machine with kubectl access):

{ kubectl get resourcequotas --all-namespaces -o json
  kubectl get namespaces -o json
} | jq -rs '
  .[0] as $quotas | .[1] |
  [ .items[]
  | select(.metadata.name as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | ([ $quotas.items[] | select(.metadata.namespace == $m.name) ] | length) as $count
  | "name=\($m.name) resourceQuotas=\($count) is_compliant=\(if $count > 0 then "true" else "false" end)"
  ][]'

Ensure all tenant namespaces show is_compliant=true.

Using kubectl

# 1) Create a ResourceQuota manifest per tenant namespace.
# Replace <TENANT-NAMESPACE> with the actual namespace name.
# Run on: any machine with kubectl access

cat << 'EOF' > tenant-resourcequota.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-default-quota
  namespace: <TENANT-NAMESPACE>
spec:
  hard:
    requests.cpu: "2"
    limits.cpu: "4"
    requests.memory: "4Gi"
    limits.memory: "8Gi"
    pods: "50"
    services: "20"
    configmaps: "50"
    persistentvolumeclaims: "20"
    secrets: "100"
    replicationcontrollers: "50"
    services.loadbalancers: "5"
    services.nodeports: "10"
EOF

# 2) Apply the ResourceQuota to the target tenant namespace.
kubectl apply -f tenant-resourcequota.yaml

# (Optional) If you have multiple tenant namespaces, you can loop:
# for ns in tenant-a tenant-b tenant-c; do
#   sed "s/<TENANT-NAMESPACE>/$ns/" tenant-resourcequota.yaml | kubectl apply -f -
# done

# 3) Verification: confirm each non-system namespace has at least one ResourceQuota.
{ kubectl get resourcequotas --all-namespaces -o json \
  kubectl get namespaces -o json; } | jq -rs '
  .[0] as $quotas | .[1] |
  [ .items[]
  | select(.metadata.name as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ $quotas.items[] | select(.metadata.namespace == $m.name) ] | length) as $count
  | "kind=Namespace name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $labels == "" then "" else " labels=\($labels)" end)
    + " resourceQuotas=\($count)"
    + " is_compliant=\(if $count > 0 then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Automation

#!/usr/bin/env bash
#
# Ensure each tenant namespace in an EKS cluster has at least one ResourceQuota.
# - Skips: kube-system, kube-public, kube-node-lease
# - Creates: a default ResourceQuota if none exist in the namespace
# - Idempotent: safe to re-run
#
# Run on: any machine with kubectl access and jq installed.

set -euo pipefail

# Configuration for the default ResourceQuota to create where missing.
# Adjust values as appropriate for your cluster before running.
RQ_NAME="tenant-default-quota"
RQ_CPU_REQUESTS="4"
RQ_CPU_LIMITS="8"
RQ_MEM_REQUESTS="8Gi"
RQ_MEM_LIMITS="16Gi"
RQ_PODS="200"
RQ_SERVICES="50"
RQ_CONFIGMAPS="100"
RQ_SECRETS="100"
RQ_PVCS="50"

# Temporary file for manifest
TMP_MANIFEST="$(mktemp)"
trap 'rm -f "${TMP_MANIFEST}"' EXIT

# 1. Enumerate all tenant namespaces (excluding system namespaces)
TENANT_NAMESPACES=$(kubectl get namespaces -o json | jq -r '
  .items[]
  | select(.metadata.name as $n
           | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata.name
')

if [ -z "${TENANT_NAMESPACES}" ]; then
  echo "No tenant namespaces found (only system namespaces exist). Nothing to do."
else
  echo "Processing tenant namespaces:"
  echo "${TENANT_NAMESPACES}" | sed 's/^/  - /'
fi

# 2. For each tenant namespace, ensure at least one ResourceQuota exists
for NS in ${TENANT_NAMESPACES}; do
  echo "Checking ResourceQuotas in namespace: ${NS}"

  RQ_COUNT=$(kubectl get resourcequota -n "${NS}" -o json 2>/dev/null | jq '.items | length')

  if [ "${RQ_COUNT}" -gt 0 ]; then
    echo "  Namespace ${NS} already has ${RQ_COUNT} ResourceQuota object(s). Skipping."
    continue
  fi

  echo "  Namespace ${NS} has no ResourceQuota. Creating ${RQ_NAME}."

  cat > "${TMP_MANIFEST}" <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:
  name: ${RQ_NAME}
  namespace: ${NS}
spec:
  hard:
    requests.cpu: "${RQ_CPU_REQUESTS}"
    limits.cpu: "${RQ_CPU_LIMITS}"
    requests.memory: "${RQ_MEM_REQUESTS}"
    limits.memory: "${RQ_MEM_LIMITS}"
    pods: "${RQ_PODS}"
    services: "${RQ_SERVICES}"
    configmaps: "${RQ_CONFIGMAPS}"
    secrets: "${RQ_SECRETS}"
    persistentvolumeclaims: "${RQ_PVCS}"
EOF

  kubectl apply -f "${TMP_MANIFEST}"
done

echo
echo "Verification: recomputing compliance status for namespaces."

# 3. Verification (re-run the benchmark audit logic)
{ kubectl get resourcequotas --all-namespaces -o json
  kubectl get namespaces -o json
} | jq -rs '
  .[0] as $quotas | .[1] |
  [ .items[]
  | select(.metadata.name as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
  | .metadata as $m
  | (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
  | ([ $quotas.items[] | select(.metadata.namespace == $m.name) ] | length) as $count
  | "kind=Namespace name=\($m.name) uid=\($m.uid) apiVersion=v1"
    + (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
    + (if $labels == "" then "" else " labels=\($labels)" end)
    + " resourceQuotas=\($count)"
    + " is_compliant=\(if $count > 0 then "true" else "false" end)"
  ] as $rows
  | if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

More Info:​

Risk Level​

Address​

Compliance Standards​

Triage and Remediation​

Remediation​