Skip to main content

Pods Should Be Managed By A Controller

More Info:​

Verifies pods are owned by a controller (Deployment, StatefulSet, DaemonSet, Job). A naked pod is not rescheduled if its node dies.

Risk Level​

Low

Address​

Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Manual Steps
  1. List naked pods (no owner controller) excluding system namespaces (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
    [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | select($own == null)
    | "\($m.namespace) \($m.name)"
    ][]'
  2. For each naked pod, export its full spec to a manifest file (run on any machine with kubectl access):

    NAMESPACE="example-namespace"
    POD_NAME="example-pod"
    kubectl get pod "$POD_NAME" -n "$NAMESPACE" -o yaml > "${POD_NAME}.pod.yaml"
  3. Create a Deployment manifest from the pod spec (typical case; run on any machine with kubectl access):

    • Open the exported file in an editor and transform it into a Deployment, saving as example-pod-deploy.yaml:
      apiVersion: apps/v1
      kind: Deployment
      metadata:
      name: example-pod
      namespace: example-namespace
      labels:
      app: example-pod
      spec:
      replicas: 1
      selector:
      matchLabels:
      app: example-pod
      template:
      metadata:
      labels:
      app: example-pod
      spec:
      containers:
      # Copy containers[] from the original pod spec here
      # Also copy any needed fields from spec: volumes, serviceAccountName, nodeSelector, tolerations, etc.
    • Adjust fields (e.g., replicas, labels, scheduling settings) to match the intent of the original pod.
    • If the workload is better suited for a different controller (e.g., DaemonSet for one pod per node, StatefulSet for stable identities, Job for run-to-completion), construct that controller instead of a Deployment but still based on the original pod’s spec.template.
  4. Apply the new controller manifest (run on any machine with kubectl access):

    kubectl apply -f example-pod-deploy.yaml
  5. After confirming the new controller-created pod is Running and functioning, delete the original naked pod (run on any machine with kubectl access):

    kubectl get pods -n "$NAMESPACE" -o wide
    kubectl delete pod "$POD_NAME" -n "$NAMESPACE"
  6. Verify no remaining naked pods (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
    [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | "kind=Pod ns=\($m.namespace) name=\($m.name) is_compliant=\(if $own == null then "false" else "true" end)"
    ] as $rows
    | if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
    then "is_compliant=true"
    else ($rows[] | select(. | contains("is_compliant=false")))
    end'
Using kubectl

On any machine with kubectl access:

  1. Identify naked pods (excluding GKE system namespaces):
kubectl get pods --all-namespaces -o json | jq -r '
[ .items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| .metadata as $m
| ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
| select($own == null)
| "\($m.namespace) \($m.name)"
][]'

For each <namespace> <pod-name> pair below, create a controller.

  1. Export the naked pod spec (example for one pod):
kubectl get pod POD_NAME -n NAMESPACE -o yaml > /tmp/pod-POD_NAME.yaml
  1. Create a Deployment manifest from the pod (most common case):
cat << 'EOF' > /tmp/deploy-POD_NAME.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: POD_NAME
namespace: NAMESPACE
spec:
replicas: 1
selector:
matchLabels:
app: POD_NAME
template:
metadata:
labels:
app: POD_NAME
spec:
containers:
- name: CONTAINER_NAME
image: IMAGE
# copy over ports, env, resources, volumeMounts, etc. from the original pod
# copy over volumes, serviceAccountName, nodeSelector, tolerations, etc.
EOF

Fill in POD_NAME, NAMESPACE, CONTAINER_NAME, IMAGE, and any other fields from /tmp/pod-POD_NAME.yaml.

Apply the Deployment:

kubectl apply -f /tmp/deploy-POD_NAME.yaml

Wait for the new pod to be ready:

kubectl rollout status deployment POD_NAME -n NAMESPACE

Delete the original naked pod:

kubectl delete pod POD_NAME -n NAMESPACE
  1. If the workload needs a different controller type, create that instead:
  • Long-running replicated: kind: Deployment
  • One-per-node: kind: DaemonSet
  • Ordered or with stable network/storage IDs: kind: StatefulSet
  • Finite / batch: kind: Job or kind: CronJob

Use the same pattern: build a manifest whose .spec.template matches the original pod’s .spec, then kubectl apply -f and delete the naked pod.

  1. Verification (same as audit command):
kubectl get pods --all-namespaces -o json | jq -r '
[ .items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| .metadata as $m
| ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
| "kind=Pod ns=\($m.namespace) name=\($m.name) is_compliant=\(if $own == null then "false" else "true" end)"
] as $rows
| if ($rows | map(select(. | contains("is_compliant=false"))) | length) == 0
then "is_compliant=true"
else $rows[]
end'
Automation
#!/usr/bin/env bash
#
# automate-controllerization-of-naked-pods.sh
#
# Idempotently ensures that non-system, non-controlled Pods are
# recreated via a Deployment named "dp-<podname>" in the same namespace.
#
# REQUIREMENTS:
# - Run on any machine with kubectl, jq, and bash
# - kubeconfig/context pointing at the target GKE cluster
#
# SAFETY:
# - Skips kube-system, kube-public, kube-node-lease
# - Skips Pods that already have an owning controller
# - Skips Pods that are part of a Job/CronJob by label convention
# (job-name / cronjob-name) to avoid breaking Jobs
# - Deletes original naked Pods only after corresponding Deployment
# is created
#
# NOTE:
# - This is a generic automation. You may want to refine label/selector
# logic per application before broad use in production.

set -euo pipefail

# Fail fast if dependencies are missing
for bin in kubectl jq; do
if ! command -v "$bin" >/dev/null 2>&1; then
echo "ERROR: $bin not found in PATH" >&2
exit 1
fi
done

# Optional: namespace allow/deny lists (space-separated). Empty means "all non-system namespaces".
NAMESPACE_INCLUDE="${NAMESPACE_INCLUDE:-}"
NAMESPACE_EXCLUDE="${NAMESPACE_EXCLUDE:-}"

# Label keys we treat as indicating a Job/CronJob pod (common patterns)
JOB_LABEL_KEYS=("job-name" "controller-uid" "batch.kubernetes.io/job-name" "cronjob-name")

# Return 0 (true) if array contains value
array_contains() {
local needle="$1"; shift || true
local x
for x in "$@"; do
[[ "$x" == "$needle" ]] && return 0
done
return 1
}

# Decide whether namespace should be processed
namespace_allowed() {
local ns="$1"

# Skip core system namespaces unconditionally
case "$ns" in
kube-system|kube-public|kube-node-lease)
return 1
;;
esac

# Exclude list
if [[ -n "$NAMESPACE_EXCLUDE" ]]; then
for x in $NAMESPACE_EXCLUDE; do
[[ "$ns" == "$x" ]] && return 1
done
fi

# Include list
if [[ -n "$NAMESPACE_INCLUDE" ]]; then
for x in $NAMESPACE_INCLUDE; do
[[ "$ns" == "$x" ]] && return 0
done
return 1
fi

return 0
}

# Check if pod is controlled (has ownerReferences.controller = true)
is_pod_controlled() {
local ns="$1" name="$2"
local count
count="$(kubectl get pod "$name" -n "$ns" -o json \
| jq '[.metadata.ownerReferences[]? | select(.controller==true)] | length')"
[[ "$count" -gt 0 ]]
}

# Heuristic: check for Job/CronJob related labels
is_job_like_pod() {
local ns="$1" name="$2"
local labels_json
labels_json="$(kubectl get pod "$name" -n "$ns" -o jsonpath='{.metadata.labels}' 2>/dev/null || echo '{}')"
for key in "${JOB_LABEL_KEYS[@]}"; do
if echo "$labels_json" | jq -e --arg k "$key" 'has($k)' >/dev/null 2>&1; then
return 0
fi
done
return 1
}

# Generate a simple Deployment spec from an existing Pod
generate_deployment_from_pod() {
local ns="$1" name="$2" dp_name="$3"

# Fetch pod JSON once
local pod_json
pod_json="$(kubectl get pod "$name" -n "$ns" -o json)"

# Extract container list, labels, and annotations (filtering some well-known pod-only annotations)
local containers labels annotations
containers="$(echo "$pod_json" | jq '.spec.containers')"
labels="$(echo "$pod_json" | jq '.metadata.labels // {}')"
annotations="$(echo "$pod_json" | jq '
.metadata.annotations // {} |
del(
."kubectl.kubernetes.io/last-applied-configuration",
."cni.projectcalico.org/podIP",
."cni.projectcalico.org/podIPs",
."kubernetes.io/config.seen",
."kubernetes.io/config.source"
)')"

# Construct a basic app label if none exist
if [[ "$(echo "$labels" | jq 'length')" -eq 0 ]]; then
labels="$(jq -n --arg app "$dp_name" '{app: $app}')"
fi

# For selector, use same label set (or refine to {app: dp_name})
local selector_labels="$labels"

# Build Deployment manifest
jq -n \
--arg ns "$ns" \
--arg name "$dp_name" \
--argjson containers "$containers" \
--argjson labels "$labels" \
--argjson annotations "$annotations" \
--argjson selector "$selector_labels" '
{
apiVersion: "apps/v1",
kind: "Deployment",
metadata: {
name: $name,
namespace: $ns,
labels: $labels,
annotations: $annotations
},
spec: {
replicas: 1,
selector: {
matchLabels: $selector
},
template: {
metadata: {
labels: $labels,
annotations: $annotations
},
spec: {
containers: $containers
}
}
}
}'
}

main() {
echo "=== Discovering naked Pods (no controller owners) in non-system namespaces ==="

# Get all pods except in system namespaces
mapfile -t pods < <(kubectl get pods --all-namespaces -o json \
| jq -r '
.items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| select(([.metadata.ownerReferences[]? | select(.controller==true)] | length) == 0)
| "\(.metadata.namespace) \(.metadata.name)"
')

if [[ ${#pods[@]} -eq 0 ]]; then
echo "No naked Pods found."
fi

for line in "${pods[@]}"; do
ns="$(awk '{print $1}' <<<"$line")"
pod="$(awk '{print $2}' <<<"$line")"

if ! namespace_allowed "$ns"; then
echo "Skipping Pod $ns/$pod (namespace excluded or system)."
continue
fi

# Re-check control in case state changed since discovery
if is_pod_controlled "$ns" "$pod"; then
echo "Skipping Pod $ns/$pod (now has controller owner)."
continue
fi

if is_job_like_pod "$ns" "$pod"; then
echo "Skipping Pod $ns/$pod (looks like Job/CronJob-managed by labels)."
continue
fi

dp_name="dp-${pod}"

# If Deployment already exists, skip creating another and just delete naked Pod
if kubectl get deploy "$dp_name" -n "$ns" >/dev/null 2>&1; then
echo "Deployment $ns/$dp_name already exists; ensuring it manages Pod template and deleting naked Pod $ns/$pod."
kubectl delete pod "$pod" -n "$ns" --wait=false
continue
fi

echo "Creating Deployment $ns/$dp_name from naked Pod $ns/$pod ..."

# Generate manifest to a temp file
tmpfile="$(mktemp)"
if ! generate_deployment_from_pod "$ns" "$pod" "$dp_name" >"$tmpfile"; then
echo "ERROR: Failed to generate Deployment from Pod $ns/$pod; skipping." >&2
rm -f "$tmpfile"
continue
fi

# Apply Deployment
if ! kubectl apply -f "$tmpfile"; then
echo "ERROR: Failed to apply Deployment for Pod $ns/$pod; leaving Pod untouched." >&2
rm -f "$tmpfile"
continue
fi

rm -f "$tmpfile"

# Delete original naked Pod; Deployment will create replacement
kubectl delete pod "$pod" -n "$ns" --wait=false || true
done

echo
echo "=== Verification: re-running compliance audit ==="
kubectl get pods --all-namespaces -o json | jq -r '
[ .items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| .metadata as $m
| (.spec.nodeName // "") as $node
| (($m.labels // {}) | to_entries | map("\(.key):\(.value)") | join(",")) as $labels
| ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
| "kind=Pod ns=\($m.namespace) name=\($m.name) uid=\($m.uid) apiVersion=v1"
+ (if ($m.creationTimestamp // "") == "" then "" else " created=\($m.creationTimestamp)" end)
+ (if $node == "" then "" else " node=\($node)" end)
+ (if $labels == "" then "" else " labels=\($labels)" end)
+ (if $own == null then "" else " owner=\($own.kind)/\($m.namespace)/\($own.name)/\($own.uid)" end)
+ " is_compliant=\(if $own == null then "false" else "true" end)"
] as $rows
| if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'
}

main "$@"