Customer Snapshot
| Attribute | Details |
|---|---|
| Industry | Technology / AI SaaS |
| Cloud Environment | AWS (4 accounts), EKS clusters across environments |
| Team Size | ~150 users (~100 Engineering, ~10 DevOps) |
| K8s Access Pattern | 75% of daily access requests are EKS RBAC-related |
| Identity Provider | Google Workspace → AWS IAM Identity Center |
| Access Model Before | Standing kubeconfigs, shared cluster-admin, Jira-based requests |
| Communication | Slack |
| Primary Interest | Kubernetes JIT with pre-defined Roles / Permission Sets |
| Cloudanix Scope | Kubernetes JIT, Cloud Console JIT |
The Situation: EKS Is the Primary Workload, and Access Is the Primary Problem
For this AI SaaS company, Kubernetes wasn’t a secondary compute layer — it was where 75% of daily engineering work happened. EKS clusters across development, staging, and production accounts hosted the core platform. Engineers needed access to pods, logs, deployments, and services multiple times a day. The access request volume reflected this: three out of every four access requests the DevOps team received were for EKS cluster access.
The problem wasn’t that engineers needed Kubernetes access. It was how that access was managed:
- Standing kubeconfigs on developer laptops with long-lived tokens.
- Shared cluster-admin bindings that gave every engineer god-mode access to clusters they touched once a month.
- Jira tickets for access changes that took hours to process.
- K8s audit logs that all read
kubernetes-admin— making forensics impossible when something went wrong.
The DevOps team had defined roles and permission sets for how access should work (ClusterRoleBindings scoped to specific namespaces, read-only vs edit vs admin). But the enforcement mechanism was manual RBAC binding creation, manual revocation, and the perpetual temptation to just grant cluster-admin because it’s faster.
The Core Challenge
75% of access requests hitting the DevOps team were for EKS clusters. Each request required manual RBAC configuration, and the path of least resistance (cluster-admin for everyone) had become the de facto standard. The team needed Kubernetes access to be self-service, time-bound, and scoped — without abandoning the RBAC model they’d already designed.
Where the Gaps Were
Standing Kubeconfigs on Every Laptop
Every engineer who had ever needed cluster access had a ~/.kube/config with credentials for one or more EKS clusters. These credentials didn’t expire. They weren’t scoped to specific namespaces. And they persisted long after the original task was complete.
The security implications are straightforward:
- One stolen laptop with a valid kubeconfig is a direct path to any cluster that config references.
- No rotation because rotating kubeconfig tokens breaks every developer’s local setup. So the team never does it.
- Credentials accumulate. An engineer who worked on a production issue six months ago still has production cluster credentials on their machine.
- No visibility into which engineers have active credentials to which clusters. The DevOps team has no inventory of outstanding kubeconfigs.
Shared Cluster-Admin: The Path of Least Resistance
The team had designed proper RBAC roles: namespace-scoped view roles for debugging, edit roles for deployment updates, and admin roles for cluster-level operations. But in practice, engineers were often granted cluster-admin because:
- Creating namespace-scoped bindings takes time. The DevOps lead has to determine the right namespace, the right role, create the binding, test it, and communicate back. For urgent requests, cluster-admin is a one-command solution.
- Debugging doesn’t always know its scope in advance. An engineer investigating a pod crashloop may need to look at multiple namespaces, check network policies, and inspect cluster-level resources. Starting with view access to one namespace means they’ll be back with another request in 10 minutes.
- Bindings don’t self-destruct. A cluster-admin binding created for a 30-minute debugging session persists until someone manually removes it. Which happens approximately never.
The result: multiple engineers with standing cluster-admin access to production clusters, each with audit logs that say the same federated role name made the change. When a misconfigured deployment causes an outage, the investigation starts with “which of our 15 cluster-admins did this?” and often ends with “we can’t tell.”
Audit Logs That Audit Nothing Useful
Kubernetes audit logs in EKS record API server activity. But when every engineer authenticates through the same assumed IAM role (which gets mapped to a Kubernetes identity), the audit logs all attribute actions to the same user:
user: "kubernetes-admin"
verb: "delete"
resource: "deployments"
namespace: "payments"Which human was kubernetes-admin at that moment? The K8s audit log doesn’t know. CloudTrail shows who assumed the IAM role, but correlating CloudTrail AssumeRole events with specific kubectl commands inside a cluster requires cross-referencing timestamps across two different log systems — a forensics exercise nobody does proactively.
Private Clusters vs. Productive Engineers
The team had done the right thing: EKS API servers were configured as private endpoints, not publicly accessible. But this created a practical problem: engineers couldn’t kubectl from their laptops without:
- A VPN with cluster-admin-level network access baked in, or
- A bastion host that everyone SSH’d through with shared credentials, or
- Complex port-forwarding setups that broke regularly.
The security decision (private cluster) and the productivity requirement (engineers need kubectl) were in constant tension. The workarounds that bridged this gap all reintroduced the security problems the private endpoint was meant to solve.
The Cloudanix Solution: Kubernetes JIT with Pre-Defined Roles
Cloudanix Kubernetes JIT replaced standing kubeconfigs, shared cluster-admin bindings, and manual RBAC management with ephemeral, scoped, and audited cluster access — covering the 75% of daily requests that hit EKS.
How It Works for EKS Clusters
-
Engineer requests access via Slack or the Cloudanix Console — selecting the cluster, namespace, role (view / edit / admin / cluster-admin), and duration. An incident ID or reason is attached.
-
Approval routes by sensitivity. View access on a development namespace auto-approves. Edit access on production routing requires human approval. Cluster-admin on any production cluster escalates to the DevOps lead with full context.
-
Engineer runs
cdx k8s connect. The Cloudanix CLI opens an outbound tunnel to the cluster’s VPC (solving the private-cluster problem), mints a short-lived kubeconfig scoped to the granted role and namespace, and sets the kubectl context. -
Engineer uses their normal tools.
kubectl,k9s, Lens, Helm — anything that reads a kubeconfig works. The session is scoped to what was granted. An engineer withviewonpaymentsnamespace can’t exec into pods or access other namespaces. -
Every kubectl command is stamped to the real human identity — not the shared assumed role. When the audit log shows a
delete deploymentin thepaymentsnamespace, it says exactly which engineer ran it. -
Access auto-expires. When the time window closes, the kubeconfig is invalidated, the tunnel closes, and the RBAC binding is removed. No standing access survives the session.
Pre-Defined Roles Mapped to Teams
The team had already done the RBAC design work — they’d defined what roles engineers should have. Cloudanix makes those roles requestable and enforceable:
- Engineering group can request
viewandediton development and staging namespaces (auto-approved), andviewon production namespaces (one approval). - DevOps group can request up to
adminon any namespace (auto-approved for non-prod), andcluster-adminon production (requires escalated approval). - Custom policies for non-admin privileges. The team configured pre-defined policies for admin-level access and custom, scoped policies for standard engineering work — matching their existing RBAC design exactly.
The RBAC model the team designed years ago is now the RBAC model that’s actually enforced — not the aspirational one that’s overridden by cluster-admin grants every time urgency wins.
Solving the Private Cluster Problem
For fully-private EKS control planes, cdx k8s connect dials outbound to a lightweight proxy in the customer’s own VPC. The connection path:
- Engineer’s laptop → outbound HTTPS → Cloudanix relay → customer VPC proxy → private EKS API server.
- No VPN required. No bastion host. No inbound network changes to the cluster.
- The API server stays private. No new ingress rules. No security group modifications.
Engineers get kubectl access to private clusters from anywhere, without the VPN-with-cluster-admin workaround that defeats the purpose of making the cluster private in the first place.
Identity-Stamped Audit: Who Actually Ran That kubectl
Every JIT session produces an audit trail that resolves the “which human was kubernetes-admin?” problem:
session: jit-4a2c91f8
user: engineer@company.com
cluster: eks-prod-us-west-2
role: edit
namespace: payments
duration: 30m
commands:
[00:14] kubectl get pods -n payments
[00:32] kubectl describe pod payments-7d6c9-abc1 -n payments
[01:07] kubectl logs payments-7d6c9-abc1 --tail=50
[02:21] kubectl rollout restart deploy/payments -n payments
[04:11] exit
status: auto-revoked at expiryThe session recording, the identity attribution, and the K8s API audit all link to the same human. For compliance (SOC 2, ISO 27001) and for incident investigation, this is the difference between “the cluster-admin role did it” and “this specific engineer ran this specific command at this specific time.”
Session Recording for Compliance and Forensics
Every Kubernetes JIT session is captured as a replayable recording — every kubectl command, every output, every Helm install. Recordings are:
- Stored in the customer’s own S3 bucket (never in Cloudanix infrastructure).
- Linked to the JIT request, the approval, and the identity.
- Shareable with auditors via signed URLs that expire.
- Replayable in the Cloudanix console for incident review.
For a team running SOC 2 audits, the evidence package for “who has access to production Kubernetes clusters” goes from “we have RBAC configured” to “here are the recorded sessions of every production cluster access in the last quarter, each stamped to a named human, each time-bounded and auto-revoked.”
Platform Impact
| Metric | Before | After |
|---|---|---|
| Standing cluster-admin bindings | 15+ engineers | Zero |
| Time to cluster access | 15 min – 4 hours (Jira ticket) | Under 60 seconds |
| Audit attribution | “kubernetes-admin” | Real human identity |
| Private cluster access | VPN/bastion with broad access | Scoped outbound tunnel |
| Kubeconfig rotation | Never (breaks setups) | Not needed — ephemeral per session |
| RBAC enforcement | Aspirational (overridden in practice) | Enforced per request |
| Session forensics | Cross-reference CloudTrail + K8s audit | Single recorded timeline |
Why Standing K8s Access Is Particularly Dangerous
Kubernetes access is different from cloud console access in one important way: the blast radius of a single kubectl command can be higher than any IAM action. kubectl delete ns production removes every pod, service, secret, and configmap in that namespace instantly. kubectl exec into a pod with a mounted service account token gives access to whatever that service account can reach.
Standing cluster-admin on a production EKS cluster isn’t “elevated access.” It’s the highest-risk standing privilege most engineering teams have — more dangerous than AWS console access because the damage is more immediate and harder to roll back.
That’s why 75% of the access request volume being EKS-related made this the highest-priority JIT surface for this team. The biggest risk required the tightest control — and the control needed to be fast enough that engineers didn’t work around it.
Running EKS as Your Primary Compute Layer?
If Kubernetes is where most of your engineering work happens, and your access model is still based on standing kubeconfigs, shared cluster-admin bindings, and manual RBAC changes — Cloudanix Kubernetes JIT gives your engineers the speed they need (sub-60-second access) with the controls your security team requires (scoped, time-bound, identity-stamped, auto-revoked).
Book a Free Assessment to see Kubernetes JIT working with your EKS clusters — including private control planes — in a single session.
Related Resources
- Kubernetes Security Checklist 2026: Hardening EKS, AKS & GKE
- Container Security and Real-Time Visibility for Modern Cloud
- Implementing JIT Access in AWS, Azure & GCP: Complete Guide
- PAM for Cloud Infrastructure: Why Traditional PAM Falls Short
- The End of Permanent Access: Just-In-Time Granular Database Security
