What is Kubernetes?
The standard platform for running containerized workloads at scale
Kubernetes (often abbreviated K8s) is an open-source container orchestration platform originally developed by Google and now maintained by the Cloud Native Computing Foundation (CNCF). It automates the deployment, scaling, and management of containerized applications across clusters of machines.
As of August 2026, Kubernetes is at version 1.36 (with 1.34, 1.35, and 1.36 actively maintained). The project has evolved far beyond basic container scheduling — it now provides a declarative platform for networking, security policy, secrets management, storage orchestration, and multi-cluster workloads that powers everything from startups to the world’s largest cloud providers.
Why Kubernetes Matters for Cloud Teams
- Declarative infrastructure: Define desired state in YAML; Kubernetes continuously reconciles actual state to match.
- Self-healing: Automatically restarts failed containers, replaces unhealthy pods, and reschedules workloads when nodes go down.
- Horizontal autoscaling: Scales workloads based on CPU, memory, or custom metrics, optimizing cost and performance.
- Portability: Runs consistently across AWS, Azure, GCP, on-premises, and edge environments.
- Extensibility: Custom Resource Definitions (CRDs) and operators let you extend the platform for any workload type — databases, ML pipelines, message queues.
- Ecosystem maturity: A vast CNCF ecosystem of graduated projects (Prometheus, Envoy, Cilium, ArgoCD, Flux, and more) integrates natively.
Core Architecture
Control Plane
The control plane manages the overall cluster state:
- kube-apiserver: The front door to the cluster. All interactions (kubectl, controllers, operators) go through the API server.
- etcd: A distributed key-value store that holds all cluster state and configuration.
- kube-scheduler: Assigns pods to nodes based on resource requirements, affinity rules, and constraints.
- kube-controller-manager: Runs controllers (Deployment, ReplicaSet, Job, etc.) that reconcile desired state with actual state.
- cloud-controller-manager: Integrates with cloud provider APIs for load balancers, storage, and node lifecycle.
Worker Nodes
Each node runs the workloads:
- kubelet: The agent on every node that ensures containers are running as specified by the pod spec.
- kube-proxy (or eBPF replacement): Handles service networking and load balancing at the node level.
- Container runtime: containerd or CRI-O runs the actual containers.
Key Objects
| Object | Purpose |
|---|---|
| Pod | Smallest deployable unit; one or more containers sharing network and storage |
| Deployment | Manages stateless workloads with rolling updates and rollbacks |
| StatefulSet | Manages stateful workloads with stable network identity and persistent storage |
| DaemonSet | Ensures a pod runs on every (or selected) node |
| Job / CronJob | Runs batch or scheduled tasks to completion |
| Service | Stable networking endpoint that load-balances traffic to pods |
| ConfigMap / Secret | Externalized configuration and sensitive data |
| Namespace | Logical isolation boundary within a cluster |
Modern Kubernetes Networking
Gateway API: The Successor to Ingress
The Kubernetes Gateway API has replaced the legacy Ingress resource as the standard for managing external traffic into clusters. Ingress NGINX, the most widely used ingress controller, reached end-of-maintenance in March 2026, making migration to Gateway API essential.
Gateway API provides:
- Role-oriented design: Separates concerns between infrastructure providers (GatewayClass), cluster operators (Gateway), and application developers (HTTPRoute, GRPCRoute, TLSRoute).
- Expressive routing: Header-based routing, traffic splitting, URL rewrites, and request mirroring — all without vendor-specific annotations.
- Extensibility: A standard extension model replaces the annotation sprawl of Ingress.
- Multi-protocol support: HTTP, gRPC, TLS, and TCP/UDP routing in a unified API.
All major cloud providers (AWS, Azure, GCP) and service mesh projects now ship Gateway API implementations.
eBPF-Based Networking with Cilium
Cilium has become the dominant Container Network Interface (CNI) plugin in production Kubernetes, used in over 60% of surveyed deployments. Built on eBPF (extended Berkeley Packet Filter), it replaces traditional iptables-based networking with programs that run directly in the Linux kernel.
Key advantages:
- High performance: Bypasses iptables entirely, eliminating per-packet overhead at scale.
- Advanced network policies: L3/L4 and L7-aware policies (HTTP, gRPC, Kafka, DNS) that go far beyond native Kubernetes NetworkPolicy.
- Transparent encryption: WireGuard or IPsec encryption between pods without application changes.
- Observability: Hubble provides real-time flow visibility and service maps.
- Service mesh integration: Cilium can provide mTLS and L7 load balancing without sidecar proxies.
Azure CNI (powered by Cilium) and GKE Dataplane V2 both use Cilium under the hood.
Service Mesh: Istio Ambient Mode
The service mesh landscape has matured significantly. Istio Ambient Mode reached GA in v1.24, offering a sidecar-less architecture that provides mTLS, observability, and traffic management through node-level ztunnel proxies and optional waypoint proxies — dramatically reducing resource overhead compared to traditional sidecar injection.
Kubernetes Security
Security in Kubernetes operates at multiple layers: cluster infrastructure, workload configuration, network, supply chain, and runtime.
Pod Security Standards (PSS)
PodSecurityPolicy was removed in Kubernetes 1.25 and replaced by Pod Security Standards enforced through the built-in Pod Security Admission controller. PSS defines three security levels:
- Privileged: Unrestricted (for system-level workloads like CNI plugins).
- Baseline: Prevents known privilege escalations (blocks hostNetwork, hostPID, privileged containers).
- Restricted: Heavily hardened (requires non-root, read-only root filesystem, drops all capabilities, enforces seccomp profiles).
Enforcement is configured via namespace labels with three modes: enforce, audit, and warn.
RBAC Best Practices
Role-Based Access Control is the primary authorization mechanism:
- Follow least-privilege: grant only the permissions each workload or user actually needs.
- Prefer namespaced
Role/RoleBindingover cluster-wideClusterRole/ClusterRoleBinding. - Audit
system:mastersgroup membership and avoid binding it to service accounts. - Use short-lived tokens (bound service account tokens) rather than long-lived secrets.
- Regularly audit RBAC with tools like
kubectl auth can-iand policy reports.
Network Policies
Network Policies enforce micro-segmentation at the pod level:
- Default-deny all ingress and egress per namespace, then explicitly allow required traffic.
- Use Cilium or Calico for L7-aware policies (filter by HTTP path, DNS name, or Kafka topic).
- Combine with Gateway API for defense-in-depth at the ingress layer.
Admission Controllers and Policy Engines
Beyond Pod Security Standards, production clusters use dedicated policy engines for business-specific guardrails:
- ValidatingAdmissionPolicy (GA in Kubernetes 1.30): Built-in policy enforcement using CEL (Common Expression Language) expressions — no external webhook needed. Ideal for simple, high-performance validation rules.
- Kyverno (CNCF Graduated, March 2026): Kubernetes-native policies written in YAML. Supports validation, mutation, generation, and image verification. The easiest on-ramp for teams already comfortable with Kubernetes manifests.
- OPA Gatekeeper (CNCF Graduated): Uses the Rego policy language for complex, multi-resource validation. Better suited when you need a single policy language across Kubernetes, Terraform, CI/CD, and microservice APIs.
Common policies enforced:
- Require trusted image registries only.
- Mandate resource requests and limits on all containers.
- Enforce mandatory labels (team, cost-center, environment).
- Block
latestimage tags in production. - Require NetworkPolicies in every namespace.
Secrets Management
Native Kubernetes Secrets are base64-encoded (not encrypted at rest by default). Production best practices:
- Enable etcd encryption at rest for the secrets resource.
- External Secrets Operator (ESO): Syncs secrets from external providers (AWS Secrets Manager, Azure Key Vault, Google Secret Manager, HashiCorp Vault) into Kubernetes Secrets automatically.
- Workload Identity Federation: Eliminates long-lived credentials entirely. Pods authenticate with short-lived tokens tied to their Kubernetes ServiceAccount, which cloud IAM exchanges for access tokens. Supported by EKS (IAM Roles for Service Accounts), GKE (Workload Identity Federation), and AKS (Entra Workload Identity).
- Sealed Secrets or SOPS: Encrypt secrets in Git for GitOps workflows.
Runtime Security
- Use Falco or Tetragon (eBPF-based) for real-time detection of anomalous behavior: unexpected process execution, file access, and network connections.
- Enforce seccomp and AppArmor/SELinux profiles to restrict syscall access.
- Scan running workloads for CVEs with continuous vulnerability scanners.
GitOps and Continuous Delivery
GitOps treats Git as the single source of truth for cluster state. Two CNCF-graduated projects dominate the space:
ArgoCD
- Application-centric with a polished web UI and topology visualization.
- Strong multi-cluster management through ApplicationSets.
- Integrated RBAC and SSO.
- Runs in approximately 60% of Kubernetes clusters using GitOps.
Flux
- Composable, CLI-first toolkit of Kubernetes controllers.
- Native image automation (auto-detects new container images and commits updates).
- Deep Kustomize and Helm integration.
- Decentralized architecture — no central server.
Both support Helm, Kustomize, encrypted secrets (SOPS), drift detection, and automated reconciliation.
Multi-Cluster Management
As organizations scale, managing multiple clusters becomes necessary for regional availability, team isolation, or regulatory compliance. Key approaches:
- Cluster API (CAPI): Declarative lifecycle management for clusters themselves — provision, upgrade, and decommission clusters as Kubernetes resources.
- GitOps fleet management: ArgoCD ApplicationSets or Flux’s multi-cluster support to deploy workloads consistently across clusters.
- Service mesh federation: Istio or Cilium Cluster Mesh for cross-cluster service discovery and mTLS.
- Policy federation: Kyverno or Gatekeeper policies synced across clusters from a central Git repo.
Native Sidecar Containers
Introduced in Kubernetes 1.28 and reaching GA in 1.33, native sidecar containers formalize the sidecar pattern as a first-class concept. Defined as init containers with restartPolicy: Always, they:
- Start before the main application container and remain running for the pod’s lifetime.
- Can restart independently without affecting the main container.
- Are terminated gracefully after the main containers exit (critical for Jobs and batch workloads).
- Solve long-standing issues with log collectors, service mesh proxies, and secret agents that previously ran as regular containers with no lifecycle guarantees.
Managed Kubernetes Services
Amazon Elastic Kubernetes Service (EKS)
AWS’s managed Kubernetes offering handles the control plane, etcd, and node lifecycle. Key features:
- EKS Auto Mode: Automated node provisioning and right-sizing (extends Karpenter natively).
- IAM Roles for Service Accounts (IRSA) and EKS Pod Identity: Workload identity without long-lived credentials.
- EKS Add-ons: Managed deployment of VPC CNI, CoreDNS, kube-proxy, and third-party add-ons.
- Fargate profiles: Serverless pods with no node management.
- Supports Kubernetes versions up to 1.34 with extended support available.
Google Kubernetes Engine (GKE)
Google Cloud’s managed Kubernetes with deep integration into the GCP ecosystem:
- GKE Autopilot: Fully managed node infrastructure — you only define pods.
- GKE Dataplane V2 (Cilium-powered): eBPF networking with built-in network policy enforcement.
- Workload Identity Federation: Keyless authentication to GCP services.
- Release channels (Rapid, Regular, Stable): Automated version management.
- Binary Authorization: Enforce that only signed, trusted container images are deployed.
Azure Kubernetes Service (AKS)
Microsoft’s managed Kubernetes offering tightly integrated with Azure services:
- Azure CNI powered by Cilium: eBPF networking with advanced network policies.
- Entra Workload Identity (formerly Azure AD Workload Identity): Federated identity for pods accessing Azure resources.
- AKS Automatic: Simplified cluster management with best-practice defaults.
- Azure Policy for AKS: Built-in Gatekeeper integration for compliance guardrails.
- Defender for Containers: Runtime threat protection and vulnerability assessment.
- KEDA integration: Event-driven autoscaling native to AKS.
Kubernetes Observability
Effective cluster operations require visibility into three pillars:
- Metrics: Prometheus (CNCF Graduated) for cluster and application metrics, paired with Grafana for visualization and alerting.
- Logging: Structured logs collected by Fluent Bit or the OpenTelemetry Collector, forwarded to your log backend.
- Tracing: Distributed tracing with OpenTelemetry for request-level visibility across microservices.
- Network observability: Hubble (part of Cilium) for real-time network flow visibility and service dependency maps.
Best Practices for Production Clusters
Workload Configuration
- Always set resource requests and limits.
- Run containers as non-root with read-only root filesystems.
- Use liveness, readiness, and startup probes appropriately.
- Define Pod Disruption Budgets (PDBs) for high-availability workloads.
- Use topology spread constraints for fault-domain distribution.
Cluster Operations
- Automate upgrades through release channels or Cluster API.
- Enable audit logging and monitor API server access patterns.
- Use namespaces and ResourceQuotas for multi-tenant isolation.
- Implement backup and disaster recovery (Velero) for cluster state and persistent volumes.
Supply Chain Security
- Scan images in CI/CD with Trivy, Grype, or Snyk.
- Sign and verify images with Sigstore/Cosign.
- Enforce image provenance with admission controllers (Kyverno image verification or Binary Authorization).
- Pin image digests instead of mutable tags in production manifests.
- Use distroless or minimal base images to reduce attack surface.
Secure Your Kubernetes Workloads With Cloudanix
Cloudanix provides comprehensive security coverage across your Kubernetes infrastructure on AWS, Azure, and GCP:
- Kubernetes Security Posture Management: Continuous assessment against CIS Kubernetes Benchmarks and custom policies.
- Runtime Threat Detection: Real-time alerting on anomalous container behavior, privilege escalation, and lateral movement.
- Misconfiguration Detection: Automated scanning of Deployments, RBAC, NetworkPolicies, and admission control for security gaps.
- IAM and Identity Security: Visibility into workload identity bindings, over-permissioned service accounts, and credential exposure.
- Container Image Security: Vulnerability scanning and compliance checks across your container registry.
- Cloud Workload Protection (CWPP): End-to-end protection from build to runtime.
