Skip to main content

CloudFormation Stacks Should Not Have A Failed Status

More Info:​

None of your Amazon CloudFormation stacks should be in Failed mode for more than 6 hours. Any failed CloudFormation stacks that are not fixed on time can lead to application downtime, security issues or unexpected costs on your AWS bill.

Risk Level​

Informational

Address​

Operational Maturity

Compliance Standards​

  • APRA CPS 234 (Australia)
  • BSI C5 (Germany)
  • Brazil LGPD
  • CCPA / CPRA (California)
  • CIS Critical Security Controls v8
  • CMMC 2.0
  • CSA Cloud Controls Matrix v4
  • Cloudanix Best Practice
  • DPDPA
  • Digital Operational Resilience Act (EU)
  • Essential 8
  • ISO/IEC 27017
  • ISO/IEC 27018
  • ISO/IEC 27701
  • KSA PDPL
  • MAS Technology Risk Management (Singapore)
  • MITRE ATT&CK (Cloud)
  • NIST SP 800-171
  • NYDFS 23 NYCRR 500
  • SWIFT Customer Security Controls Framework
  • Sarbanes-Oxley IT General Controls
  • StateRAMP
  • UK NCSC Cyber Assessment Framework

Triage and Remediation​

Remediation​

Using Console

To remediate a CloudFormation stack with a failed status in AWS using the AWS console, follow these steps:

  1. Open the AWS Management Console and navigate to the CloudFormation service.
  2. Select the CloudFormation stack with a failed status that you want to remediate.
  3. Click on the "Events" tab to view the events associated with the stack.
  4. Review the events to identify the root cause of the failure. The event details will provide information on the resource that failed and the reason for the failure.
  5. Once you have identified the root cause of the failure, take the appropriate action to remediate the issue. This may involve updating the CloudFormation template, modifying the resource configuration, or resolving any dependencies or permissions issues.
  6. After making the necessary changes, update the CloudFormation stack by clicking on the "Update Stack" button.
  7. Follow the prompts to upload the updated template and apply the changes to the stack.
  8. Monitor the stack events to ensure that the update is successful and the stack status changes to "CREATE_COMPLETE" or "UPDATE_COMPLETE".

By following these steps, you can remediate a CloudFormation stack with a failed status in AWS using the AWS console.

Using CLI

To remediate the CloudFormation Stack failed status in AWS using AWS CLI, follow these steps:

  1. Identify the CloudFormation Stack that has a failed status by running the following command in the AWS CLI:
aws cloudformation describe-stacks --stack-name <stack-name>

Replace <stack-name> with the name of the CloudFormation Stack that has a failed status.

  1. Check the events associated with the failed stack by running the following command:
aws cloudformation describe-stack-events --stack-name <stack-name>

This will provide information on the events that led to the failed status of the stack.

  1. Fix any issues that caused the stack to fail. This may involve updating the CloudFormation template or fixing any resource dependencies.

  2. Once the issues are fixed, update the stack by running the following command:

aws cloudformation update-stack --stack-name <stack-name> --template-body file://<path-to-template-file> --parameters file://<path-to-parameters-file>

Replace <stack-name> with the name of the CloudFormation Stack that has a failed status, <path-to-template-file> with the path to the updated CloudFormation template file, and <path-to-parameters-file> with the path to the updated parameters file.

  1. Wait for the stack to update and check its status by running the following command:
aws cloudformation describe-stacks --stack-name <stack-name>

If the stack status is CREATE_COMPLETE or UPDATE_COMPLETE, then the remediation is successful. If not, repeat steps 2 to 4 until the stack status is successful.

Using Python

To remediate the "CloudFormation Stacks Should Not Have A Failed Status" misconfiguration in AWS using Python, you can follow the below steps:

  1. First, you need to identify the CloudFormation stack(s) that have a failed status. You can use the boto3 library in Python to get the list of CloudFormation stacks and their status.
import boto3

# Create a CloudFormation client
cf_client = boto3.client('cloudformation')

# Get the list of stacks
stacks = cf_client.list_stacks(StackStatusFilter=['CREATE_FAILED', 'ROLLBACK_COMPLETE'])

# Print the stack names and their status
for stack in stacks['StackSummaries']:
    print(stack['StackName'], stack['StackStatus'])
  1. Once you have identified the failed stacks, you can delete them using the delete_stack method of the CloudFormation client.
# Delete the failed stacks
for stack in stacks['StackSummaries']:
    cf_client.delete_stack(StackName=stack['StackName'])
  1. You can also update the CloudFormation template and re-create the stack(s) to remediate the misconfiguration.
# Update the CloudFormation template
with open('template.yaml', 'r') as file:
    template_body = file.read()

# Update the stack(s)
for stack in stacks['StackSummaries']:
    cf_client.update_stack(StackName=stack['StackName'], TemplateBody=template_body)

Note: Make sure to test the CloudFormation template before updating the stack(s) to avoid any further misconfigurations.

Using Terraform

Terraform cannot remediate a failed CloudFormation stack state. CloudFormation stack lifecycle (CREATE_FAILED, UPDATE_ROLLBACK_FAILED, etc.) is not exposed as a Terraform-managed argument, so you cannot express delete-stack or continue-update-rollback in HCL.

To fix this finding you must use AWS directly, following the verified CLI remediation:

# 1. Investigate failure cause
aws cloudformation describe-stack-events \
  --stack-name YOUR_STACK_NAME \
  --region YOUR_AWS_REGION \
  --query "StackEvents[?ends_with(ResourceStatus, '_FAILED')].{ResourceType:ResourceType,LogicalId:LogicalResourceId,Status:ResourceStatus,Reason:ResourceStatusReason}" \
  --output table

# 2a. If in CREATE_FAILED or no longer needed: delete the stack (destructive, irreversible)
aws cloudformation delete-stack \
  --stack-name YOUR_STACK_NAME \
  --region YOUR_AWS_REGION

# 2b. If in UPDATE_ROLLBACK_FAILED and you have fixed the root cause: continue rollback
aws cloudformation continue-update-rollback \
  --stack-name YOUR_STACK_NAME \
  --region YOUR_AWS_REGION

In Terraform you can only ensure that new infrastructure is defined as Terraform resources instead of CloudFormation stacks so that Terraform, not CloudFormation, owns creation and updates. There is no aws_cloudformation_stack resource argument that corresponds to these CLI operations, so terraform plan will show no drift related to stack failure state.

Additional Reading:​