Skip to main content

Pods Should Be Managed By A Controller

More Info:

Verifies pods are owned by a controller (Deployment, StatefulSet, DaemonSet, Job). A naked pod is not rescheduled if its node dies.

Risk Level

Low

Address

Security

Compliance Standards

  • Cloudanix Best Practice

Triage and Remediation

Remediation

Manual Steps
  1. Identify naked pods (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
    [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | select($own == null)
    | "\($m.namespace) \($m.name)"
    ][]'
  2. For each naked pod, export its manifest (run on any machine with kubectl access):

    NAMESPACE=<namespace-of-pod>
    POD_NAME=<pod-name>

    kubectl get pod "${POD_NAME}" -n "${NAMESPACE}" -o yaml > "${POD_NAME}-pod.yaml"
  3. Create a controller manifest from the pod spec (run on any machine with kubectl access; edit file with your editor of choice):

    • Open the exported file:
      nano "${POD_NAME}-pod.yaml"
    • Remove metadata.uid, resourceVersion, creationTimestamp, status, and any ownerReferences.
    • Wrap the spec under a controller. For a typical Deployment, change the top-level keys to something like:
      apiVersion: apps/v1
      kind: Deployment
      metadata:
      name: <deployment-name>
      namespace: <namespace>
      labels:
      app: <some-label>
      spec:
      replicas: 1
      selector:
      matchLabels:
      app: <some-label>
      template:
      metadata:
      labels:
      app: <some-label>
      spec:
      # paste the original pod.spec content here
    • Adjust to StatefulSet, DaemonSet, or Job if more appropriate for the workload.
  4. Apply the new controller (run on any machine with kubectl access):

    kubectl apply -f "${POD_NAME}-pod.yaml"
  5. Delete the original naked pod after confirming the controller-created pod is running (run on any machine with kubectl access):

    kubectl get pods -n "${NAMESPACE}" -l app=<some-label>
    kubectl delete pod "${POD_NAME}" -n "${NAMESPACE}"
  6. Verify no remaining naked pods (run on any machine with kubectl access):

    kubectl get pods --all-namespaces -o json | jq -r '
    [ .items[]
    | select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
    | .metadata as $m
    | ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
    | select($own == null)
    ] | if (length) == 0 then "is_compliant=true" else .[] | .namespace + " " + .name end'
Using kubectl

On any machine with kubectl access:

  1. Identify naked pods (non-system namespaces)
kubectl get pods --all-namespaces -o wide \
--no-headers | awk '$1 != "kube-system" && $1 != "kube-public" && $1 != "kube-node-lease" {print $1, $2}'
  1. For each naked pod, export its spec as a template (example for namespace app-ns, pod my-app-pod):
kubectl get pod my-app-pod -n app-ns -o yaml > my-app-pod.yaml

Edit my-app-pod.yaml locally:

  • Remove fields that must not be in a controller spec:
    • metadata.uid
    • metadata.resourceVersion
    • metadata.creationTimestamp
    • metadata.ownerReferences
    • metadata.managedFields
    • metadata.selfLink, metadata.generation (if present)
    • spec.nodeName (unless you intentionally pin to a node; usually remove)
    • status section (remove entirely)
  • Decide the right controller type:
    • Deployment for stateless apps
    • StatefulSet for stateful apps needing stable identity
    • DaemonSet to run one pod per node
    • Job for finite work/batch
  1. Example: convert to a Deployment (stateless workload)

Transform my-app-pod.yaml into my-app-deployment.yaml like:

apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
namespace: app-ns
spec:
replicas: 1
selector:
matchLabels:
app: my-app # must match template.metadata.labels
template:
metadata:
labels:
app: my-app # copy labels from the original pod (plus any others)
spec:
containers:
- name: my-app # copy container spec from original pod
image: myregistry.example.com/my-app:1.0.0
ports:
- containerPort: 8080
env:
# copy env from original pod
resources:
# copy resources from original pod
imagePullSecrets:
# copy from original pod if present
nodeSelector:
# copy if used instead of spec.nodeName
tolerations:
# copy from original pod if present
affinity:
# copy from original pod if present

Apply it:

kubectl apply -f my-app-deployment.yaml
  1. Scale down and delete the original naked pod

If you need to avoid downtime:

  • Label the old pod so you can distinguish it (optional):
kubectl label pod my-app-pod -n app-ns migrate=from-naked
  • Once the Deployment pod is Ready, delete the naked pod:
kubectl delete pod my-app-pod -n app-ns
  1. Example: convert to a Job (for one-off/batch pods)

Create my-batch-job.yaml:

apiVersion: batch/v1
kind: Job
metadata:
name: my-batch-job
namespace: app-ns
spec:
template:
metadata:
labels:
job: my-batch-job # any appropriate labels
spec:
restartPolicy: OnFailure
containers:
- name: my-batch
image: myregistry.example.com/my-batch:1.0.0
# copy command/args/env from original pod

Apply and remove the naked pod:

kubectl apply -f my-batch-job.yaml
kubectl delete pod my-batch-pod -n app-ns
  1. Verification (same logic as the audit)
kubectl get pods --all-namespaces -o json | jq -r '
[ .items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| .metadata as $m
| ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
| "kind=Pod ns=\($m.namespace) name=\($m.name) is_compliant=\(if $own == null then "false" else "true" end)"
] as $rows
| if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'
Automation
#!/usr/bin/env bash
#
# Fix naked Pods by exporting them as manifests and recreating them
# under controllers (Deployments or Jobs) in AKS.
#
# WARNING:
# - This script is intentionally conservative and NON‑destructive:
# it only prints controller manifests and guidance; it does NOT
# delete or modify existing Pods.
# - A human must review and apply the generated manifests.
#
# Requirements (run on any machine with kubectl + jq + yq access):
# - kubectl configured to talk to the AKS cluster (cluster‑admin suggested)
# - jq
# - yq (https://github.com/mikefarah/yq) v4+
#
# Idempotency:
# - Safe to re‑run. Existing naked Pods will be re‑evaluated and
# manifests re‑generated in the output directory.

set -euo pipefail

OUT_DIR="./naked-pod-controllers"
mkdir -p "${OUT_DIR}"

echo "Discovering naked Pods (excluding kube-system, kube-public, kube-node-lease)..."

# Capture list of naked pods: namespace name
mapfile -t NAKED_PODS < <(
kubectl get pods --all-namespaces -o json | jq -r '
.items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| select((.metadata.ownerReferences // []) | length == 0)
| "\(.metadata.namespace) \(.metadata.name)"
'
)

if [ "${#NAKED_PODS[@]}" -eq 0 ]; then
echo "No naked Pods found. Cluster is compliant."
exit 0
fi

echo "Found ${#NAKED_PODS[@]} naked Pod(s). Generating controller manifests in ${OUT_DIR}/"
echo

for line in "${NAKED_PODS[@]}"; do
NS=$(awk '{print $1}' <<< "${line}")
NAME=$(awk '{print $2}' <<< "${line}")

echo "Processing naked Pod: ${NS}/${NAME}"

POD_YAML="${OUT_DIR}/${NS}-${NAME}-pod.yaml"
CTRL_YAML="${OUT_DIR}/${NS}-${NAME}-controller.yaml"

# Export current Pod manifest without cluster‑managed fields
kubectl get pod "${NAME}" -n "${NS}" -o yaml > "${POD_YAML}"

# Build a controller manifest template around the Pod spec
# Heuristics:
# - If restartPolicy == Never or OnFailure -> Job
# - Else -> Deployment (replicas: 1)
RESTART_POLICY=$(yq '.spec.restartPolicy // "Always"' "${POD_YAML}")

if [[ "${RESTART_POLICY}" == "Never" || "${RESTART_POLICY}" == "OnFailure" ]]; then
CTRL_KIND="Job"
CTRL_API="batch/v1"
else
CTRL_KIND="Deployment"
CTRL_API="apps/v1"
fi

CTRL_NAME="${NAME}"

# Strip fields that should not be copied from Pod to controller
CLEAN_SPEC=$(yq '
.spec
| del(.nodeName)
| del(.serviceAccount) # deprecated field
| del(.hostname)
| del(.subdomain)
| del(.tolerations[]?.key == "node.kubernetes.io/not-ready")
| del(.tolerations[]?.key == "node.kubernetes.io/unreachable")
' "${POD_YAML}")

# Build controller YAML
{
echo "# GENERATED TEMPLATE for ${NS}/${NAME}"
echo "# KIND: ${CTRL_KIND}"
echo "# REVIEW CAREFULLY BEFORE APPLYING."
echo "# Original Pod saved in: ${POD_YAML}"
echo "---"
echo "apiVersion: ${CTRL_API}"
echo "kind: ${CTRL_KIND}"
echo "metadata:"
echo " name: ${CTRL_NAME}"
echo " namespace: ${NS}"
echo " labels:"
echo " app: ${CTRL_NAME}"
echo "spec:"
} > "${CTRL_YAML}"

if [[ "${CTRL_KIND}" == "Deployment" ]]; then
{
echo " replicas: 1"
echo " selector:"
echo " matchLabels:"
echo " app: ${CTRL_NAME}"
echo " template:"
echo " metadata:"
echo " labels:"
echo " app: ${CTRL_NAME}"
echo " spec:"
} >> "${CTRL_YAML}"

# Append cleaned Pod spec fields under template.spec
# We drop restartPolicy because Deployment manages it implicitly
yq 'del(.restartPolicy)' - <<< "${CLEAN_SPEC}" | sed 's/^/ /' >> "${CTRL_YAML}"
else
# Job
{
echo " backoffLimit: 3"
echo " template:"
echo " metadata:"
echo " labels:"
echo " app: ${CTRL_NAME}"
echo " spec:"
} >> "${CTRL_YAML}"

# Append cleaned Pod spec fields under template.spec
yq '.' - <<< "${CLEAN_SPEC}" | sed 's/^/ /' >> "${CTRL_YAML}"
fi

echo " -> Generated controller manifest: ${CTRL_YAML}"
echo
done

cat <<'EOF'

NEXT STEPS (manual, per application owner):

1. REVIEW each generated controller manifest in ./naked-pod-controllers/*-controller.yaml
- Confirm KIND choice (Deployment vs Job).
- Adjust labels, resource requests/limits, probes, and other settings.
- Ensure any configMaps, secrets, PVCs, and ServiceAccount references are correct.

2. PLAN migration for each naked Pod (example for a Deployment):
# Optional: backup original Pod spec (already saved as *-pod.yaml)
# Delete naked Pod (will disrupt that workload once):
# kubectl delete pod <pod-name> -n <namespace>
#
# Create controller:
# kubectl apply -f <ns-pod-controller>.yaml

3. VERIFY compliance after applying controllers:
kubectl get pods --all-namespaces -o json | jq -r '
[ .items[]
| select(.metadata.namespace as $n | ["kube-system","kube-public","kube-node-lease"] | index($n) | not)
| .metadata as $m
| ([ ($m.ownerReferences // [])[] | select(.controller) ] | first) as $own
| select($own == null)
| "kind=Pod ns=\($m.namespace) name=\($m.name) is_compliant=false"
] as $rows
| if ($rows | length) == 0 then "is_compliant=true" else $rows[] end'

Cluster is compliant when the final line prints: is_compliant=true

EOF