Skip to main content

Sagemaker Endpoints Should Be Encrypted With KMS Customer

More Info:​

Sagemaker Endpoints should be encrypted with KMS Customer Managed Keys

Risk Level​

High

Address​

Security

Compliance Standards​

  • APRA CPS 234 (Australia)
  • BSI C5 (Germany)
  • Brazil LGPD
  • CCPA / CPRA (California)
  • CIS Critical Security Controls v8
  • CMMC 2.0
  • CSA Cloud Controls Matrix v4
  • DPDPA
  • Digital Operational Resilience Act (EU)
  • ISO 27001
  • ISO/IEC 27018
  • ISO/IEC 27701
  • MAS Technology Risk Management (Singapore)
  • MITRE ATT&CK (Cloud)
  • NIST SP 800-171
  • NYDFS 23 NYCRR 500
  • SOC2
  • SWIFT Customer Security Controls Framework
  • Sarbanes-Oxley IT General Controls
  • UK NCSC Cyber Assessment Framework

Triage and Remediation​

Remediation​

Using Console

Below are the console steps to ensure SageMaker endpoints use a KMS customer-managed key (CMK) for encryption at rest.

1. Create or identify a KMS customer-managed key​

  1. In the AWS Management Console, go to Key Management Service (KMS).
  2. In the left menu, choose Customer managed keys.
  3. Either:
    • Use an existing key: Note the Key ID or ARN; or
    • Create a new key:
      1. Click Create key.
      2. Key type: Symmetric → Next.
      3. Set an alias (e.g., alias/sagemaker-endpoints).
      4. Configure key administrators and usage permissions.
        • Under Key users, add the SageMaker execution role(s) used by your endpoints (IAM role that your SageMaker models/endpoint use).
      5. Finish the wizard and create the key.

You will need the KMS key ARN.

2. Create a new Endpoint Configuration using the CMK​

You cannot edit an existing endpoint configuration; you must create a new one and then update the endpoint to use it.

  1. Go to Amazon SageMaker console.
  2. In the left navigation pane, choose Endpoint configurations.
  3. Click Create endpoint configuration.
  4. Enter a Name for the configuration.
  5. Under Production variants:
    1. Choose the Model.
    2. Choose Instance type, Initial instance count, etc.
    3. For Encryption key (KMS) or Volume encryption (label may vary):
      • Select Enter a KMS key ARN or select from the dropdown.
      • Provide the KMS CMK ARN you prepared in step 1.
  6. (Optional but recommended) Configure Data capture:
    1. Enable Data capture.
    2. Select or specify the S3 location.
    3. For KMS key for data capture (if shown), select the same CMK or another CMK ARN.
  7. Click Create endpoint configuration.

3. Update the existing Endpoint to use the new configuration​

  1. In the SageMaker console, go to Endpoints.
  2. Click the endpoint that is reported as non-compliant.
  3. Choose Update (or Update endpoint).
  4. Under Endpoint configuration, select the new endpoint configuration you created.
  5. Review settings and click Update endpoint.

SageMaker will transition the endpoint to the new configuration (this may briefly impact availability depending on your deployment settings).

4. Verify encryption with CMK​

  1. After the update finishes, open the Endpoint details.
  2. Confirm it references the new endpoint configuration.
  3. Open the Endpoint configuration details and verify:
    • The KMS key ARN is your customer-managed key for:
      • Production variant volume encryption.
      • Data capture (if enabled).

New SageMaker endpoints should always be created with the desired CMK set in the endpoint configuration so they are compliant by default.

Using CLI

Below are step‑by‑step AWS CLI instructions to ensure SageMaker endpoints use a KMS customer managed key (CMK) for encryption at rest.

Important: Encryption settings for a SageMaker endpoint are defined in its endpoint configuration and cannot be changed in-place. To remediate, you must:

  1. Create/use a KMS CMK.
  2. Create a new endpoint configuration using that CMK.
  3. Update the endpoint to use the new config.

1. Create (or identify) a KMS CMK​

If you don’t already have a CMK for SageMaker:

aws kms create-key \
  --description "CMK for SageMaker endpoint encryption" \
  --key-usage ENCRYPT_DECRYPT \
  --origin AWS_KMS

Note the KeyMetadata.KeyId from the output (e.g. 1234abcd-12ab-34cd-56ef-1234567890ab).

Optionally give it an alias:

aws kms create-alias \
  --alias-name alias/sagemaker-endpoint-kms \
  --target-key-id 1234abcd-12ab-34cd-56ef-1234567890ab

You can use either the alias (alias/sagemaker-endpoint-kms) or the key ID/ARN in the next steps.

Ensure your SageMaker execution role has permissions:

{
  "Effect": "Allow",
  "Action": [
    "kms:Encrypt",
    "kms:Decrypt",
    "kms:GenerateDataKey*",
    "kms:DescribeKey"
  ],
  "Resource": "arn:aws:kms:<region>:<account-id>:key/1234abcd-12ab-34cd-56ef-1234567890ab"
}

Attach this policy to the IAM role used by the SageMaker endpoint (its execution role).

2. Get the current endpoint configuration​

Assume your endpoint is named my-endpoint:

aws sagemaker describe-endpoint \
  --endpoint-name my-endpoint > current-endpoint.json

In the output, note EndpointConfigName, e.g. my-endpoint-config.

Now get the current endpoint config details:

aws sagemaker describe-endpoint-config \
  --endpoint-config-name my-endpoint-config > current-endpoint-config.json

This file contains:

  • ProductionVariants (models, instance types, counts, etc.)
  • DataCaptureConfig (if enabled)
  • Any existing KmsKeyId (likely missing or set to AWS-managed KMS)

3. Create a new endpoint configuration with KMS CMK​

You cannot modify an existing endpoint config; you must create a new one.

Prepare a JSON file (e.g. new-endpoint-config.json). You can base it on current-endpoint-config.json and:

  • Keep ProductionVariants the same.
  • Add/update KmsKeyId with your CMK.
  • Preserve DataCaptureConfig if you use it (you can also add a KmsKeyId inside the data capture config if needed).

Example new-endpoint-config.json:

{
  "EndpointConfigName": "my-endpoint-config-kms",
  "ProductionVariants": [
    {
      "VariantName": "AllTraffic",
      "ModelName": "my-model-name",
      "InitialInstanceCount": 1,
      "InstanceType": "ml.m5.large",
      "InitialVariantWeight": 1.0
    }
  ],
  "KmsKeyId": "alias/sagemaker-endpoint-kms"
}

If using data capture and you want that encrypted with the same CMK:

{
  "EndpointConfigName": "my-endpoint-config-kms",
  "ProductionVariants": [
    {
      "VariantName": "AllTraffic",
      "ModelName": "my-model-name",
      "InitialInstanceCount": 1,
      "InstanceType": "ml.m5.large",
      "InitialVariantWeight": 1.0
    }
  ],
  "KmsKeyId": "alias/sagemaker-endpoint-kms",
  "DataCaptureConfig": {
    "EnableCapture": true,
    "InitialSamplingPercentage": 100,
    "DestinationS3Uri": "s3://my-sagemaker-capture-bucket/prefix/",
    "CaptureOptions": [
      {"CaptureMode": "Input"},
      {"CaptureMode": "Output"}
    ],
    "CaptureContentTypeHeader": {
      "CsvContentTypes": ["text/csv"],
      "JsonContentTypes": ["application/json"]
    },
    "KmsKeyId": "alias/sagemaker-endpoint-kms"
  }
}

Create the new endpoint configuration:

aws sagemaker create-endpoint-config \
  --cli-input-json file://new-endpoint-config.json

4. Update the endpoint to use the new KMS-encrypted config​

Now point the existing endpoint to the new endpoint configuration:

aws sagemaker update-endpoint \
  --endpoint-name my-endpoint \
  --endpoint-config-name my-endpoint-config-kms

You can monitor the update:

aws sagemaker describe-endpoint \
  --endpoint-name my-endpoint \
  --query 'EndpointStatus'

Wait until status is InService.

5. (Optional) Clean up old endpoint configuration​

Once the endpoint is stable and using the KMS CMK, you can delete the old, unencrypted endpoint config (not strictly required but good hygiene):

aws sagemaker delete-endpoint-config \
  --endpoint-config-name my-endpoint-config

This sequence ensures that the SageMaker endpoint’s storage (including model artifacts on the endpoint instances, and optionally data capture) is encrypted with your customer managed KMS key using only AWS CLI.

Using Python

To use a KMS customer-managed key (CMK) for SageMaker endpoints, you must:

  1. Have (or create) a KMS CMK.
  2. Create a new endpoint configuration that references that CMK.
  3. Update the endpoint to use the new configuration.

Existing endpoint configurations cannot be edited in-place.

Below is a minimal, end‑to‑end Python (boto3) example.

1. Prerequisites​

pip install boto3
import boto3
import time

sagemaker = boto3.client("sagemaker", region_name="us-east-1")
kms = boto3.client("kms", region_name="us-east-1")

endpoint_name = "my-endpoint"
model_name = "my-model"  # existing model used by your endpoint
kms_key_alias = "alias/sagemaker-endpoints-cmk"

2. Create or get a KMS CMK​

If you already have a CMK with a known key ID or alias, skip to step 3.

# Create a CMK with an alias (run once)
kms_response = kms.create_key(
    Description="CMK for encrypting SageMaker endpoint volumes and artifacts",
    KeyUsage="ENCRYPT_DECRYPT",
    Origin="AWS_KMS"
)
kms_key_id = kms_response["KeyMetadata"]["KeyId"]

kms.create_alias(
    AliasName=kms_key_alias,
    TargetKeyId=kms_key_id
)

If you already have the CMK:

# Resolve alias -> key ID (optional)
alias_desc = kms.describe_key(KeyId=kms_key_alias)
kms_key_id = alias_desc["KeyMetadata"]["KeyId"]

3. Inspect the current endpoint (to copy its settings)​

# Get current endpoint description
ep_desc = sagemaker.describe_endpoint(EndpointName=endpoint_name)
old_ep_config_name = ep_desc["EndpointConfigName"]

# Get current endpoint configuration
old_ep_config = sagemaker.describe_endpoint_config(
    EndpointConfigName=old_ep_config_name
)
production_variants = old_ep_config["ProductionVariants"]

# Optional configs you may want to preserve if present
data_capture_config = old_ep_config.get("DataCaptureConfig")
async_inference_config = old_ep_config.get("AsyncInferenceConfig")
tags = sagemaker.list_tags(ResourceArn=old_ep_config["EndpointConfigArn"]).get("Tags", [])

4. Create a new endpoint configuration with the CMK​

You must create a new config; you cannot modify the existing one.

Key fields that support KMS encryption:

  • KmsKeyId in ProductionVariants → encrypts the ML storage volume.
  • DataCaptureConfig.KmsKeyId (if using data capture).
  • AsyncInferenceConfig.OutputConfig.KmsKeyId (if using async inference).
# Ensure every production variant has the KMS key
new_production_variants = []
for pv in production_variants:
    pv_new = pv.copy()
    pv_new["KmsKeyId"] = kms_key_id
    new_production_variants.append(pv_new)

# Set KMS for Data Capture if present
if data_capture_config:
    data_capture_config = data_capture_config.copy()
    data_capture_config["KmsKeyId"] = kms_key_id

# Set KMS for Async Inference Output if present
if async_inference_config:
    async_inference_config = async_inference_config.copy()
    out_cfg = async_inference_config.get("OutputConfig", {})
    out_cfg["KmsKeyId"] = kms_key_id
    async_inference_config["OutputConfig"] = out_cfg

new_ep_config_name = old_ep_config_name + "-kms-cmk"

create_args = {
    "EndpointConfigName": new_ep_config_name,
    "ProductionVariants": new_production_variants
}

if data_capture_config:
    create_args["DataCaptureConfig"] = data_capture_config
if async_inference_config:
    create_args["AsyncInferenceConfig"] = async_inference_config

resp = sagemaker.create_endpoint_config(**create_args)

# Re-apply tags to the new endpoint config, if any
if tags:
    sagemaker.add_tags(ResourceArn=resp["EndpointConfigArn"], Tags=tags)

5. Update the endpoint to use the new configuration​

update_resp = sagemaker.update_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=new_ep_config_name
)

# Optionally, wait until the update is complete
while True:
    status = sagemaker.describe_endpoint(EndpointName=endpoint_name)["EndpointStatus"]
    print("Endpoint status:", status)
    if status in ("InService", "Failed"):
        break
    time.sleep(30)

6. (Optional) Clean up old endpoint configuration​

Once the endpoint is InService and stable on the new configuration:

sagemaker.delete_endpoint_config(EndpointConfigName=old_ep_config_name)

Summary​

  • Use a KMS CMK (not the default AWS-managed key).
  • Attach KmsKeyId in:
    • ProductionVariants for the endpoint storage volume.
    • DataCaptureConfig.KmsKeyId and AsyncInferenceConfig.OutputConfig.KmsKeyId if those features are used.
  • You must create a new endpoint configuration and then update the endpoint to use it.
Using Terraform
# KMS key to use for SageMaker endpoint encryption
resource "aws_kms_key" "sagemaker_endpoint_cmk" {
  description             = "CMK for SageMaker endpoint ${var.SAGEMAKER_ENDPOINT_NAME}"
  enable_key_rotation     = true
  deletion_window_in_days = 30

  tags = {
    Name = "sagemaker-endpoint-cmk-${var.SAGEMAKER_ENDPOINT_NAME}"
  }
}

# SageMaker endpoint configuration with CMK encryption enabled
resource "aws_sagemaker_endpoint_configuration" "this" {
  name = var.SAGEMAKER_ENDPOINT_CONFIG_NAME

  production_variants {
    model_name           = var.SAGEMAKER_MODEL_NAME
    variant_name         = "AllTraffic"
    initial_instance_count = 1
    instance_type        = "ml.m5.large"
  }

  # This enables encryption of the endpoint’s attached storage with a CMK
  kms_key_arn = aws_kms_key.sagemaker_endpoint_cmk.arn
}

# SageMaker endpoint that uses the encrypted configuration
resource "aws_sagemaker_endpoint" "this" {
  name                 = var.SAGEMAKER_ENDPOINT_NAME
  endpoint_config_name = aws_sagemaker_endpoint_configuration.this.name
}

Substitute:

  • var.SAGEMAKER_ENDPOINT_NAME with your endpoint name.
  • var.SAGEMAKER_ENDPOINT_CONFIG_NAME with your endpoint configuration name.
  • var.SAGEMAKER_MODEL_NAME with your SageMaker model name.
  • Adjust instance type/count as required.

Changing kms_key_arn on aws_sagemaker_endpoint_configuration forces replacement of that configuration, which in turn forces replacement of the aws_sagemaker_endpoint resource and may cause downtime during recreation.

Verification: terraform plan should show an update to aws_sagemaker_endpoint_configuration.this adding kms_key_arn, and a replace (-/+) for aws_sagemaker_endpoint.this to use the new encrypted configuration.