Cloudtrail changes alarm remediation

Using Console

Below are the steps to enable a CloudTrail changes alarm in CloudWatch using the AWS Console.

1. Confirm CloudTrail is sending logs to CloudWatch Logs

Go to AWS Console → CloudTrail.
In the left menu, choose Trails.
Select your trail.
Under CloudWatch Logs, confirm:
- CloudWatch Logs log group is set (for example: /aws/cloudtrail/main).
- If not set:
  1. Click Edit.
  2. In CloudWatch Logs, choose or create a Log group.
  3. Choose or create an IAM role (CloudTrail will suggest one).
  4. Save the changes.

You need the log group name for the next steps.

2. Create a Metric Filter for CloudTrail configuration changes

Go to CloudWatch → Logs → Log groups.
Click the log group used by CloudTrail (e.g. /aws/cloudtrail/main).
Go to the Metric filters tab.
Click Create metric filter.

In Filter pattern, paste:

{ ($.eventSource = "cloudtrail.amazonaws.com") && 
  (($.eventName = "CreateTrail") || 
   ($.eventName = "UpdateTrail") || 
   ($.eventName = "DeleteTrail") || 
   ($.eventName = "StartLogging") || 
   ($.eventName = "StopLogging") || 
   ($.eventName = "PutEventSelectors") || 
   ($.eventName = "PutInsightSelectors")) }

Click Next.
Under Assign metric, fill in:
- Filter name: CloudTrailConfigChanges
- Metric namespace: e.g. Security/CloudTrail
- Metric name: e.g. CloudTrailChangesCount
- Metric value: 1
- Leave default for other options (or as required by your org).
Click Next, then Create metric filter.

3. Create a CloudWatch Alarm on that metric

Still in CloudWatch, go to Alarms → All alarms.
Click Create alarm.
Click Select metric.
Navigate to the namespace you used:
- Custom namespaces → Security/CloudTrail → Metrics with no dimensions (or matching your setup).
Select CloudTrailChangesCount.
Click Next.

Configure alarm conditions

Under Statistic, select Sum.
Under Period, choose a period (e.g. 5 minutes).
Under Conditions:
- Threshold type: Static
- Whenever metric is: >=
- Threshold value: 1
Click Next.

Configure notifications

Under Notification, choose an existing SNS topic or:
- Click Create a new topic.
- Give it a name (e.g. cloudtrail-config-change-alerts).
- Add email endpoints (e.g. your security team email).
- Confirm the subscription via the email sent from AWS.
Choose Alarm state trigger: In alarm.
Click Next.

Name and create alarm

Alarm name: CloudTrail-Config-Changes-Alarm
Add a description (optional, e.g. “Alerts on create/update/delete or logging changes to CloudTrail”).
Review all settings and click Create alarm.

Once done, any time someone changes CloudTrail configuration (creates/updates/deletes trail, starts/stops logging, or changes selectors), CloudWatch will detect it via the metric filter and trigger the alarm, which will send a notification.

Using CLI

Below is a minimal, CLI‑only way to set up an alarm that triggers when CloudTrail configuration is changed (CreateTrail/UpdateTrail/DeleteTrail/StartLogging/StopLogging).Assumptions:

You already have a CloudTrail trail sending logs to a CloudWatch Logs log group.
You know the log group name (replace YOUR_LOG_GROUP_NAME below).
You know the SNS topic ARN you want to notify (replace YOUR_SNS_TOPIC_ARN).

1. Create a CloudWatch Logs metric filter for CloudTrail changes

aws logs put-metric-filter \
  --log-group-name "YOUR_LOG_GROUP_NAME" \
  --filter-name "CloudTrail_Changes_Filter" \
  --filter-pattern '{ ($.eventName = CreateTrail) || ($.eventName = UpdateTrail) || ($.eventName = DeleteTrail) || ($.eventName = StartLogging) || ($.eventName = StopLogging) }' \
  --metric-transformations \
      metricName="CloudTrailChanges",metricNamespace="CloudTrailMetrics",metricValue="1"

This creates a metric CloudTrailChanges in namespace CloudTrailMetrics whenever such an event appears in the log stream.

2. Create a CloudWatch alarm on that metric

aws cloudwatch put-metric-alarm \
  --alarm-name "CloudTrail_Changes_Alarm" \
  --alarm-description "Alarm when CloudTrail configuration is changed" \
  --metric-name "CloudTrailChanges" \
  --namespace "CloudTrailMetrics" \
  --statistic Sum \
  --period 300 \
  --evaluation-periods 1 \
  --threshold 1 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions "YOUR_SNS_TOPIC_ARN"

Explanation of key choices (brief):

period 300 = 5 minutes.
Alarm fires if at least 1 such event is detected in 1 evaluation period (threshold 1, evaluation-periods 1).

3. (Optional) Test the alarm

Trigger a CloudTrail change (e.g., update a trail description) and confirm:

The CloudTrailChanges metric increments.
The CloudTrail_Changes_Alarm goes into ALARM state.
SNS notifications are delivered.

This fully enables a CloudTrail changes alarm in CloudWatch using the AWS CLI.

Using Python

Below are step‑by‑step remediation instructions and a minimal Python (boto3) example that:

Creates a CloudWatch Logs metric filter to detect CloudTrail configuration changes.
Creates a CloudWatch alarm on that metric.

Assumptions (adjust names/regions as needed):

You already have at least one CloudTrail writing to a CloudWatch Logs log group.
You know the log group name where CloudTrail is sending logs (e.g., /aws/cloudtrail/organization).
You have permissions for logs:*, cloudwatch:*, and sns:* (if using SNS notifications).

1. What you need to detect

CloudTrail changes usually include events such as:

CreateTrail
UpdateTrail
DeleteTrail
StartLogging
StopLogging

We’ll create a metric filter that matches these in the CloudTrail logs.Filter pattern:

{ ($.eventName = CreateTrail) || ($.eventName = UpdateTrail) || ($.eventName = DeleteTrail) || ($.eventName = StartLogging) || ($.eventName = StopLogging) }

2. High-level remediation steps

Identify the CloudWatch Logs log group that CloudTrail is using.
Create a metric filter in that log group for CloudTrail configuration change events.
Create or use a CloudWatch metric namespace and metric name (e.g., CloudTrailMetrics, CloudTrailConfigChanges).
Create a CloudWatch alarm that:
- Monitors the metric from step 3.
- Triggers when the metric is ≥ 1 within a 5-minute period (or your chosen interval).
(Optional) Attach an SNS topic to the alarm for notifications.

3. Python (boto3) example

Replace the following placeholders before running:

REGION → e.g. "us-east-1"
LOG_GROUP_NAME → CloudTrail log group (e.g. "/aws/cloudtrail/organization")
METRIC_NAMESPACE → e.g. "CloudTrailMonitoring"
METRIC_NAME → e.g. "CloudTrailConfigChanges"
ALARM_NAME → e.g. "CloudTrailChangesAlarm"
SNS_TOPIC_ARN → Your SNS topic ARN for notifications (or remove if not needed).

import boto3

REGION = "us-east-1"
LOG_GROUP_NAME = "/aws/cloudtrail/organization"
METRIC_NAMESPACE = "CloudTrailMonitoring"
METRIC_NAME = "CloudTrailConfigChanges"
METRIC_FILTER_NAME = "CloudTrailConfigChangesFilter"
ALARM_NAME = "CloudTrailChangesAlarm"
SNS_TOPIC_ARN = "arn:aws:sns:us-east-1:123456789012:cloudtrail-changes-topic"

logs_client = boto3.client("logs", region_name=REGION)
cw_client = boto3.client("cloudwatch", region_name=REGION)

def create_metric_filter():
    filter_pattern = (
        '{ ($.eventName = CreateTrail) || '
        '($.eventName = UpdateTrail) || '
        '($.eventName = DeleteTrail) || '
        '($.eventName = StartLogging) || '
        '($.eventName = StopLogging) }'
    )

    logs_client.put_metric_filter(
        logGroupName=LOG_GROUP_NAME,
        filterName=METRIC_FILTER_NAME,
        filterPattern=filter_pattern,
        metricTransformations=[
            {
                "metricName": METRIC_NAME,
                "metricNamespace": METRIC_NAMESPACE,
                "metricValue": "1"
            }
        ]
    )
    print(f"Metric filter '{METRIC_FILTER_NAME}' created/updated.")

def create_alarm():
    cw_client.put_metric_alarm(
        AlarmName=ALARM_NAME,
        AlarmDescription="Alarm when CloudTrail configuration changes occur",
        Namespace=METRIC_NAMESPACE,
        MetricName=METRIC_NAME,
        Statistic="Sum",
        Period=300,                # 5 minutes
        EvaluationPeriods=1,
        Threshold=1.0,
        ComparisonOperator="GreaterThanOrEqualToThreshold",
        TreatMissingData="notBreaching",
        AlarmActions=[SNS_TOPIC_ARN],   # Remove or adjust if not using SNS
        ActionsEnabled=True
    )
    print(f"Alarm '{ALARM_NAME}' created/updated.")

if __name__ == "__main__":
    create_metric_filter()
    create_alarm()

4. Post‑setup verification

Confirm the metric filter exists:
- CloudWatch Console → Logs → Log groups → your log group → Metric filters.
Confirm the alarm:
- CloudWatch Console → Alarms → look for CloudTrailChangesAlarm.
Trigger a test (e.g., update a trail in a non‑prod account) and verify:
- The metric increments.
- The alarm goes into ALARM state and sends SNS notification (if configured).

Using Terraform

# CloudWatch Logs metric filter for CloudTrail configuration changes
resource "aws_cloudwatch_log_metric_filter" "cloudtrail_config_changes" {
  name           = "CloudTrailConfigChanges"
  log_group_name = aws_cloudwatch_log_group.CLOUDTRAIL_LOG_GROUP.name # replace with your CloudTrail log group resource or hard-coded name

  # Matches CreateTrail, UpdateTrail, DeleteTrail, StartLogging, StopLogging
  pattern = "{($.eventName=CreateTrail)||($.eventName=UpdateTrail)||($.eventName=DeleteTrail)||($.eventName=StartLogging)||($.eventName=StopLogging)}"

  metric_transformation {
    name      = "CloudTrailConfigChanges"
    namespace = "CloudTrailMetrics"
    value     = "1"
  }
}

# CloudWatch alarm for CloudTrail configuration changes
resource "aws_cloudwatch_metric_alarm" "cloudtrail_config_changes_alarm" {
  alarm_name          = "CloudTrailConfigChangesAlarm"
  alarm_description   = "Alarm for CloudTrail configuration changes"
  namespace           = "CloudTrailMetrics"
  metric_name         = aws_cloudwatch_log_metric_filter.cloudtrail_config_changes.metric_transformation[0].name
  statistic           = "Sum"
  period              = 300
  evaluation_periods  = 1
  threshold           = 1
  comparison_operator = "GreaterThanOrEqualToThreshold"

  alarm_actions = [
    "arn:aws:sns:REGION:ACCOUNT_ID:TOPIC_NAME", # replace with your existing SNS topic ARN (<SNS_TOPIC_ARN>)
  ]

  depends_on = [aws_cloudwatch_log_metric_filter.cloudtrail_config_changes]
}

# Example CloudWatch Logs group that receives CloudTrail events
# Replace CLOUDTRAIL_LOG_GROUP with your actual resource or use an existing name:
resource "aws_cloudwatch_log_group" "CLOUDTRAIL_LOG_GROUP" {
  name = "/aws/cloudtrail/CLOUDTRAIL_LOG_GROUP_NAME" # replace CLOUDTRAIL_LOG_GROUP_NAME with the actual <LOG_GROUP_NAME>
}

This change does not force replacement of existing CloudTrail or SNS resources; it only creates/updates the metric filter and alarm. After you substitute CLOUDTRAIL_LOG_GROUP_NAME, REGION, ACCOUNT_ID, and TOPIC_NAME with your actual values, terraform plan should show one aws_cloudwatch_log_metric_filter to add and one aws_cloudwatch_metric_alarm to add (or to update if they already exist but differ).

AWS Introduction

AWS Pricing

AWS Threats

AWS Misconfigurations

Cloudtrail changes alarm remediation

Triage and Remediation

Remediation

1. Confirm CloudTrail is sending logs to CloudWatch Logs

2. Create a Metric Filter for CloudTrail configuration changes

3. Create a CloudWatch Alarm on that metric

Configure alarm conditions

Configure notifications

Name and create alarm

1. Create a CloudWatch Logs metric filter for CloudTrail changes

2. Create a CloudWatch alarm on that metric

3. (Optional) Test the alarm

1. What you need to detect

2. High-level remediation steps

3. Python (boto3) example

4. Post‑setup verification

​Triage and Remediation

​Remediation

​1. Confirm CloudTrail is sending logs to CloudWatch Logs

​2. Create a Metric Filter for CloudTrail configuration changes

​3. Create a CloudWatch Alarm on that metric

​Configure alarm conditions

​Configure notifications

​Name and create alarm

​1. Create a CloudWatch Logs metric filter for CloudTrail changes

​2. Create a CloudWatch alarm on that metric

​3. (Optional) Test the alarm

​1. What you need to detect

​2. High-level remediation steps

​3. Python (boto3) example

​4. Post‑setup verification

Triage and Remediation

Remediation

1. Confirm CloudTrail is sending logs to CloudWatch Logs

2. Create a Metric Filter for CloudTrail configuration changes

3. Create a CloudWatch Alarm on that metric

Configure alarm conditions

Configure notifications

Name and create alarm

1. Create a CloudWatch Logs metric filter for CloudTrail changes

2. Create a CloudWatch alarm on that metric

3. (Optional) Test the alarm

1. What you need to detect

2. High-level remediation steps

3. Python (boto3) example

4. Post‑setup verification