There is a sudden shift in requirements, where organizations are increasingly adopting Just-in-Time (JIT) access to minimize standing privileges and reduce the attack surface. While this approach significantly enhances security, it introduces a critical challenge: what happens when a system is down, a security breach is in progress, and the regular JIT process is too slow or unavailable?
The pressure to act quickly can lead to insecure, ad-hoc methods like sharing static privileged passwords or creating backdoors. These actions, born out of necessity, can lead to significant security gaps, audit failures, and potential policy violations. This article introduces the concept of a “break glass” procedure: a pre-defined, controlled, and auditable process designed to grant secure, temporary, elevated access during a true emergency. We’ll provide a blueprint for establishing this critical procedure, transforming a chaotic, insecure response into a structured, secure, and fully accountable one.
What is a “Break Glass” Procedure?
In the world of information security, a “break glass” procedure is a critical safeguard, designed to be used only in extreme circumstances. Its name is a metaphor for a physical “break glass in case of emergency” fire alarm. It is a pre-planned, highly secure, and temporary method for granting elevated, often administrative, access to a small, authorized group of individuals during a major incident. This could be a production outage, a suspected breach, or a critical security patch that requires immediate application.
The core principle behind this process is to balance security with operational necessity. While robust access control models, such as Just-in-Time (JIT) and Privileged Access Management (PAM), are designed to prevent unauthorized access, they can sometimes create friction during a crisis. A well-defined “break glass” procedure is not a security bypass, but a security control in itself—it replaces a chaotic, ad-hoc response with a structured, auditable one.
Key characteristics of an effective “break glass” process include:
- Exceptional Use: It is not for everyday tasks. Its use must be rare and justified by a genuine emergency.
- Temporary: Access is granted for a minimal, time-bound duration (e.g., 30-60 minutes) and is automatically revoked.
- Pre-Approved: The entire process, from trigger to audit, is defined and approved by security and leadership before an incident occurs.
- Auditable: Every step is meticulously logged and monitored for post-incident review and accountability.
Core Components of an Effective “Break Glass” Process
Establishing a robust “break glass” process requires a multi-faceted approach that covers triggering mechanisms, access granting, a streamlined workflow, and a rigorous post-incident audit.
Triggering the Process
The first step is to clearly define what constitutes an emergency and who is authorized to initiate the procedure. This prevents misuse and ensures that “break glass” access is not used as a shortcut for routine tasks.
Who can initiate?
This should be limited to specific, pre-designated roles such as an Incident Commander, a senior Site Reliability Engineer (SRE), or a designated Security Operations Center (SOC) team member. These individuals are trusted to make a judgment call under pressure.
What constitutes an emergency?
The triggers must be well-documented in the organization’s Incident Response (IR) plan. Examples include:
- A production outage is impacting core business services.
- An active ransomware attack or data exfiltration event.
- A critical security vulnerability requires an immediate, unscheduled patch to all systems.
- Failure of the primary identity provider (IdP) or privileged access management (PAM) system, which prevents normal administrative access.
The Access Granting Mechanism
The technology used to grant “break glass” access is arguably the most critical component. It must be reliable, fast, and secure, even if primary systems are unavailable.
- Dedicated Credentials: Do not use shared accounts. The process should rely on isolated, one-time-use credentials. These “break glass” accounts (e.g., breakglass-prod-root, emergency-db-admin) should be distinct from regular admin accounts and not used for any other purpose. As a best practice, these credentials should not be static; they should be ephemeral and created on-demand.
- Secure Vaulting: These master credentials must be stored in a highly secure, offline, and segregated secrets management vault. In an emergency, these secrets can be retrieved by an authorized party, often requiring a manual, multi-person approval process in a physical or digital “lockbox.” Research from institutions like the AWS Well-Architected Framework and Microsoft Entra ID guidance often recommends storing these “break glass” account credentials using strong passwordless authentication methods and ensuring they are not dependent on the primary authentication system.
- Automation: Manual processes are prone to error and delay. An automated “break glass” workflow, often orchestrated by a JIT platform, is essential. Once triggered, the system can automatically retrieve the necessary credentials from a secure vault, apply a pre-defined, time-bound policy, and provision access to the user. This automation is key to ensuring a rapid and secure response.
Post-Incident Review and Audit
The audit is what makes the “break glass” procedure a security control, not a security gap. The process isn’t over when the access is revoked; it’s a critical phase of accountability and improvement.
- Comprehensive Logging: The system must capture every action performed with the “break glass” privileges. This includes commands executed, files accessed, and changes made.
- Mandatory Post-Mortem: After the incident is resolved, a mandatory review must take place. This post-mortem should answer key questions:
- Was the “break glass” procedure truly necessary?
- Were the actions taken justified and aligned with the incident response plan?
- What was the duration of the access, and was it sufficient?
- Could the incident have been handled with a regular JIT access request?
- Were there any policy violations or suspicious activities?
- Credential Rotation: The “break glass” credentials must be automatically rotated after each use. This ensures that a compromised or leaked emergency credential becomes instantly invalid, a crucial step noted in frameworks for privileged access and incident response
Practical Implementation with a JIT Access Solution
A modern Just-in-Time (JIT) access solution is the ideal platform for implementing a robust “break glass” procedure. It automates the entire lifecycle, from request to audit, replacing insecure manual methods.
- Dedicated Emergency Access Accounts: The JIT platform can manage a separate pool of emergency accounts that are not tied to any individual user. These accounts exist in a state of “zero standing privilege” until an emergency is declared.
- Role-Based “Break Glass” Policies: You can define a specific JIT role, such as “SRE Incident Responder,” with a pre-configured policy. This policy dictates what resources (e.g., specific servers, databases, or cloud accounts) are accessible and for how long. The policy can be triggered by a single request from an authorized user.
- Secure Vault Integration: JIT platforms integrate with secrets management solutions (like HashiCorp Vault or AWS Secrets Manager) to securely store and retrieve the ephemeral “break glass” credentials. This ensures the credentials are never exposed to the end-user.
- Automated Access and Revocation: When an authorized user requests the “SRE Incident Responder” role, the JIT platform automatically grants temporary, elevated access to the target system. The access is provisioned as an ephemeral credential, and after the predefined time (e.g., 30 minutes) expires, the JIT platform automatically revokes all privileges. This automated process is key to minimizing the risk window.
- Real-Time Auditing: Every request, approval (or implicit approval), and action is logged in a tamper-proof audit trail. The JIT platform’s logs provide the necessary data for the post-mortem review, enabling compliance with standards and facilitating forensic investigations after an incident. This comprehensive logging and monitoring are a cornerstone of modern security frameworks, as noted in numerous studies on privileged access management.
Conclusion
A “break glass” procedure is not a security bypass; it is a fundamental security control. It replaces the chaos and risk of an unstructured emergency response with a controlled, repeatable, and fully auditable process. By establishing a clear plan, pre-defining triggers, and leveraging modern JIT access solutions, organizations can ensure that their teams can act decisively during a crisis without compromising on security or accountability.
This proactive approach ensures that even when the worst happens, the security posture remains intact. Ultimately, having a robust “break glass” plan is a hallmark of a mature security program, demonstrating that the organization has anticipated the unpredictable and is prepared to handle it with precision and security.
