Why 4 Weeks Is the Right Pilot Duration
Shorter pilots (1–2 weeks) don’t give enough time to encounter edge cases, test with enough users, or build confidence that the system handles real-world patterns. Longer pilots (8+ weeks) lose momentum — teams move on to other priorities and the evaluation stalls without a decision.
Four weeks provides enough time to:
- Configure and validate the core workflow.
- Test with a small group and resolve initial issues.
- Expand to a broader group and validate at scale.
- Make a go/no-go decision with data, not just demos.
This playbook is based on patterns from real JIT deployments — the sequence that consistently leads to confident production adoption.
Before the Pilot: Define Success Criteria
Before week 1 starts, agree on what “success” means. Success criteria should be specific and verifiable:
Mandatory Criteria (Must Pass)
- Access Request Creation: Engineers can request access via Slack/Teams for configured roles and accounts.
- Granting Permissions: Access is provisioned within 60 seconds of approval. The permission set appears in the SSO portal.
- Automatic Revocation: Access is revoked automatically at session expiry without manual intervention.
- Audit Trail: Every request, approval, grant, and revocation is captured in a single timeline per event.
- Zero Service Disruption: Cloudanix operations don’t impact existing AWS services, Identity Center configuration, or developer workflows.
Nice-to-Have Criteria (Inform Decision, Not Block)
- Database JIT working for primary database type.
- Kubernetes JIT working for production clusters.
- Auto-approval functioning for configured low-risk patterns.
- Integration with communication platform (Slack/Teams buttons work).
Document and Share
Write the success criteria in a shared document. Have both the evaluating team and the Cloudanix team agree. This prevents scope creep (“but can it also…?”) and provides a clear framework for the go/no-go decision at week 4.
Week 1: Configuration and Initial Testing
Day 1–2: Onboarding
Actions:
- Onboard AWS accounts into Cloudanix (run CloudFormation stack in each account).
- Connect the Organization and verify account visibility.
- Trigger Identity Center sync — users, groups, and permission sets appear in Cloudanix.
Verify:
- All target accounts visible in Cloudanix Console.
- User and group data from Identity Center synced correctly.
- Permission sets from Identity Center available for JIT policy configuration.
Day 3–4: Policy Configuration
Actions:
- Map groups to accounts with permission sets (who can request what on which account).
- Configure approval routing (who approves requests for each group/role/account combination).
- Set auto-approval rules for low-risk access patterns.
- Configure duration caps per role.
- Disable JIT for roles that aren’t part of the pilot scope.
Verify:
- Policy configuration matches intended access model.
- Groups see only the roles they should be able to request.
- Auto-approval triggers for intended patterns.
Day 5: Initial Testing (Core Team Only)
Actions:
- DevOps team (2–3 people) tests the full lifecycle:
- Request access via Slack.
- Approve the request.
- Verify permission set appears in SSO portal.
- Use the access (assume role, perform AWS actions).
- Verify revocation at session end (role disappears from portal).
- Check audit trail completeness.
Verify:
- End-to-end flow works without errors.
- Permissions are correctly applied post-elevation.
- Permissions are correctly removed post-revocation.
- Audit timeline captures all lifecycle events.
Common Issues in Week 1:
- Permission sets not syncing (refresh needed).
- Approval routing misconfigured (wrong approver receives notification).
- SSO portal cache showing stale roles (browser cache, not a platform issue).
Week 2: Broader Testing Group
Expanding the Test Group
Actions:
- Invite 15–20 engineers to use JIT for their daily access needs.
- Share internal documentation (how to request, what to expect, where to ask questions).
- Set up a support channel (Slack channel or Teams channel) for questions.
- Monitor for errors, edge cases, and unexpected behavior.
What to Watch:
- Request volume: Are engineers actually using JIT, or falling back to old methods?
- Error rate: How many requests fail? What’s the failure pattern?
- Response time: How quickly are approvers responding?
- User feedback: What’s confusing? What’s slow? What’s unexpected?
Typical Patterns in Week 2:
- 4–6 out of 18 engineers actively use the system in the first week (early adopters). Others wait to see if it works.
- 1–2 error scenarios surface (specific role/account combinations that weren’t tested in Week 1).
- Errors are resolved quickly (configuration fixes, sync refreshes).
- Engineers who successfully use it report the experience is fast and intuitive.
- No issues with documentation or guides (if the system works, the guide is simple: “request in Slack, get approved, access appears”).
Resolving Issues in Real Time
Issues discovered during Week 2 should be resolved same-day:
- Configuration errors: fix in the Cloudanix Console.
- Sync issues: trigger manual refresh.
- Permission boundary issues: adjust JIT policies.
- Documentation gaps: update the internal guide.
The goal isn’t zero issues — it’s fast resolution and confidence that issues are operational (configuration), not architectural (platform limitations).
Week 3: Validation and Broader Demo
Demo to the Broader Team
Actions:
- Conduct a walkthrough session for the broader DevOps/Engineering team (10–15+ people).
- Show the complete JIT flow: request, approval, elevation, usage, revocation.
- Demonstrate the audit trail.
- Answer questions about edge cases, scaling, and integration.
Common Questions from Broader Teams:
- “Can approvers be auto-populated?” (Yes, per-role configuration.)
- “Do Slack buttons hide after approval?” (Yes, the message updates.)
- “Can I select multiple approvers — do all need to approve?” (First approval clears it at single-level.)
- “What if someone approves and then I click Reject?” (Shows it’s already processed.)
- “Can we require support ticket numbers in the request?” (Reason field captures context; enforcement policies can be configured.)
Validate Success Criteria
By end of Week 3, verify each success criterion:
| Criterion | Status | Evidence |
|---|---|---|
| Access Request Creation | ✓ / ✗ | X successful requests from Y users |
| Permission Granting | ✓ / ✗ | Average grant time: Z seconds |
| Automatic Revocation | ✓ / ✗ | 100% of sessions auto-revoked on time |
| Audit Trail | ✓ / ✗ | Sample audit records show complete lifecycle |
| Zero Service Disruption | ✓ / ✗ | No incidents attributed to Cloudanix |
Expand Usage (If Week 2 Was Clean)
If Week 2 showed no significant issues:
- Open JIT to all engineers who want to try it.
- Let organic adoption happen alongside the formal pilot.
- Track adoption metrics: what percentage of access requests go through JIT vs. old methods?
Week 4: Decision
Go/No-Go Meeting
Agenda:
- Review success criteria results.
- Review usage metrics (requests processed, users active, error rate).
- Review user feedback (from the pilot group).
- Discuss any remaining concerns or blockers.
- Make decision: proceed to production rollout, extend pilot, or discontinue.
Decision Framework:
| Outcome | Criteria |
|---|---|
| Go to production | All mandatory criteria pass. No unresolved blocking issues. User feedback positive. |
| Extend pilot (2 more weeks) | Minor issues remain that need more time to validate. Success criteria mostly pass. |
| Discontinue | Fundamental mismatch between platform capability and requirements. Success criteria fail. |
In practice, teams that reach Week 4 with a clean pilot almost always proceed to production. The 4-week structure provides enough evidence for a confident decision.
Production Rollout Planning
If the decision is Go:
- Timeline: How quickly to expand from pilot group to full team.
- Communication: Announce to all engineers when JIT becomes the standard access method.
- Old workflow: When to deprecate the previous access method (Jira tickets, manual grants).
- Training: Usually minimal — “request in Slack, same as the pilot group has been doing.”
- Additional surfaces: Plan for Database JIT, Kubernetes JIT expansion beyond the initial Cloud JIT pilot.
Post-Pilot: The First Month of Production
Week 5–6: Full Team Onboarding
- All engineers invited to use JIT.
- Old access methods deprecated (Jira tickets no longer accepted for access requests).
- Standing permission set assignments removed (access is JIT-only).
- Weekly sync with the Cloudanix team for any issues.
Week 7–8: Optimization
- Review auto-approval policies (are they covering enough? too much?).
- Adjust duration caps based on actual usage patterns.
- Add Database JIT or Kubernetes JIT as additional surfaces.
- Begin SOC 2 evidence production using JIT audit trail.
Key Principles for a Successful Pilot
Start Small, Prove Fast
2–3 accounts. 15–20 users. 2 permission sets. Prove the core workflow works before expanding scope. Complexity is the enemy of a clean evaluation.
Resolve Issues, Don’t Accumulate Them
Every issue found in Week 1 should be resolved before Week 2 starts. Issues that compound across weeks erode confidence. A clean Week 2 builds on a resolved Week 1.
User Experience Drives Adoption
The pilot succeeds not when the platform is configured correctly, but when engineers voluntarily use it because it’s faster than the alternative. If pilot users aren’t choosing JIT over old methods, investigate why — it’s usually a friction point in the request flow.
Document for the Decision Maker
The person making the go/no-go decision may not have used the platform themselves. Document concrete results: time-to-access metrics, error rates, user quotes, audit trail samples. Data drives decisions better than enthusiasm.
Ready to Start Your JIT Pilot?
If you’re evaluating JIT access for your organization, a structured 4-week pilot gives you enough time to configure, test, validate, and decide — without the risk of committing before you’ve seen the platform work with your actual infrastructure, users, and access patterns.
Book a Free Assessment to start your pilot planning with infrastructure review and success criteria definition.
