Skip to main content

Elastic Cache Encrypted At Rest And Transit Remediation

Triage and Remediation​

Remediation​

Using Console

Below are step‑by‑step console instructions to enable encryption at rest and encryption in transit for Amazon OpenSearch Service / Amazon Elasticsearch Service (the managed service often called “AWS Elasticsearch”).

Important: Some encryption options cannot be changed on an existing domain. If the options are grayed out, you must create a new domain with encryption enabled and migrate data.

1. Check Current Encryption Settings​

  1. Sign in to the AWS Management Console.
  2. Go to Amazon OpenSearch Service (or “Amazon Elasticsearch Service” if you’re on an older console).
  3. In the left pane, choose Domains.
  4. Click your domain name.
  5. On the domain’s detail page:
    • Look at Security or Encryption sections:
      • Encryption at rest: check if it’s Enabled.
      • Node-to-node encryption: check if it’s Enabled.
      • Domain endpoint: verify if it’s using HTTPS only.

If any of these are disabled and you cannot edit them, proceed to create a new domain.

2. Enable Encryption (New Domain – Recommended if Current One Isn’t Encrypted)​

2.1 Create a New Domain with Encryption​

  1. In the OpenSearch Service console, click Create domain.
  2. Engine version: choose your required OpenSearch/Elasticsearch version.
  3. Domain name: enter a unique name (e.g., my-secure-domain).
  4. Continue through:
    • Network:
      • Choose VPC access if possible (recommended).
    • Data nodes: choose instance type and count.

2.2 Enable Encryption at Rest​

  1. Scroll to the Data protection or Encryption section.
  2. Check Enable encryption at rest.
  3. Choose a KMS key:
    • Use the AWS managed key or
    • Choose a customer-managed CMK in KMS.
  4. (Optional) Enable Auto-Tune and other performance settings as needed.

2.3 Enable Node-to-Node Encryption (In-Transit Within the Cluster)​

  1. In the same security/encryption section, check Enable node-to-node encryption.
    • This encrypts traffic between cluster nodes.

2.4 Enforce HTTPS for Client Connections (In Transit from Clients)​

  1. In the Domain endpoint or Network/“Security” section:

    • Ensure Require HTTPS is selected (or equivalent option that disallows HTTP).
    • If there’s an Endpoint security or TLS policy field, select a modern TLS policy (e.g., Policy-Min-TLS-1-2-2019-07).

  2. Configure access policy:

    • Use fine-grained access control + IAM, or
    • Restrict by VPC and security groups, or
    • If using a public endpoint, restrict by IAM and/or source IP.

  3. Click Next, review all settings, then click Create.

AWS will now create the secure domain (this can take several minutes).

3. Migrate Data from Old Domain to New Secure Domain​

  1. Once the new domain is Active, note its HTTPS endpoint.

  2. From an EC2 instance, container, or your workstation (with network access to both domains), use:

    • reindex from remote (if supported by your version), or
    • A tool such as Logstash, Elasticsearch/OpenSearch reindex APIs, or a custom script:
      • Read from http(s)://old-domain-endpoint
      • Write to https://new-domain-endpoint

  3. Update:

    • Application configuration
    • Lambda functions
    • Log shippers (Filebeat, Fluentd, CloudWatch Logs subscription filters, etc.) to use the new domain’s HTTPS endpoint.

  4. Validate:

    • Index and document counts between old and new domains.
    • Application functionality and performance.

  5. When fully migrated and tested, delete the old (unencrypted) domain from the console.

4. If You Must Modify an Existing Domain​

  1. In the OpenSearch Service console, open your domain.
  2. Click Edit (or Edit domain).
  3. Look for:
    • Encryption at rest: Try to select Enable encryption at rest and choose a KMS key.
    • Node-to-node encryption: Try to select Enable.
    • Require HTTPS: Ensure it is turned on.
  4. If any are grayed out or locked, AWS does not allow changing them in-place for that domain → you must use the new domain + migration approach above.
  5. Save/Apply changes and wait for the domain to finish updating.

5. Verify Encryption​

  1. On the domain’s details page:
    • Encryption at rest: shows Enabled with the KMS key.
    • Node-to-node encryption: shows Enabled.
    • Endpoint: begins with https:// and Only HTTPS allowed.
  2. From a client:
    • Ensure all connections use the HTTPS endpoint.
    • Confirm TLS negotiation (e.g., using curl -v https://your-domain-endpoint).

If you tell me whether you’re on the OpenSearch or legacy Elasticsearch console and whether the domain is in a VPC or public, I can tailor the exact console menu/field names for your environment.

Using CLI

For Amazon Elasticsearch / OpenSearch, encryption at rest and in-transit cannot be turned on for an existing domain. You must create a new domain with encryption enabled, migrate data, then cut over.

Below are concise, CLI-focused steps.

1. Capture current domain config (for reference)​

aws es describe-elasticsearch-domain \
  --domain-name OLD_DOMAIN_NAME \
  > old-domain-config.json

Use this JSON to copy any important settings (instance type, count, EBS size, access policy, etc.) into the new domain command.

2. Create a new domain with encryption enabled​

Key options:

  • --encryption-at-rest-options Enabled=true
  • --node-to-node-encryption-options Enabled=true
  • --domain-endpoint-options EnforceHTTPS=true,TLSecurityPolicy=Policy-Min-TLS-1-2-2019-07

Example (adjust values as needed):

aws es create-elasticsearch-domain \
  --domain-name NEW_DOMAIN_NAME \
  --elasticsearch-version 7.10 \
  --elasticsearch-cluster-config InstanceType=m5.large.elasticsearch,InstanceCount=2,ZoneAwarenessEnabled=true \
  --ebs-options EBSEnabled=true,VolumeType=gp3,VolumeSize=200 \
  --encryption-at-rest-options Enabled=true \
  --node-to-node-encryption-options Enabled=true \
  --domain-endpoint-options EnforceHTTPS=true,TLSecurityPolicy=Policy-Min-TLS-1-2-2019-07 \
  --access-policies file://access-policy.json

To use a specific KMS key for encryption at rest:

--encryption-at-rest-options Enabled=true,KmsKeyId=YOUR_KMS_KEY_ARN

Wait for the domain to be active:

aws es describe-elasticsearch-domain \
  --domain-name NEW_DOMAIN_NAME \
  --query "DomainStatus.Processing"

Repeat until it returns false.

3. Verify encryption settings​

aws es describe-elasticsearch-domain \
  --domain-name NEW_DOMAIN_NAME \
  --query "DomainStatus.{AtRest:EncryptionAtRestOptions,NodeToNode:NodeToNodeEncryptionOptions,EndpointOptions:DomainEndpointOptions}"

Confirm:

  • EncryptionAtRestOptions.Enabled = true
  • NodeToNodeEncryptionOptions.Enabled = true
  • DomainEndpointOptions.EnforceHTTPS = true

4. Migrate data from old to new domain​

Use reindex-from-remote via the Elasticsearch/OpenSearch API (not CLI) from the new domain, for each index:

curl -X POST "https://NEW_DOMAIN_ENDPOINT/_reindex?wait_for_completion=false" \
  -H 'Content-Type: application/json' \
  -d '{
    "source": {
      "remote": {
        "host": "https://OLD_DOMAIN_ENDPOINT"
      },
      "index": "old-index-name"
    },
    "dest": {
      "index": "new-index-name"
    }
  }'

Repeat for all indices (you can keep same index names if desired).

5. Update clients to use HTTPS endpoint of new domain​

From CLI, get the endpoint:

aws es describe-elasticsearch-domain \
  --domain-name NEW_DOMAIN_NAME \
  --query "DomainStatus.Endpoints"

Update all applications, Lambdas, etc. to use:

  • The HTTPS endpoint of NEW_DOMAIN_NAME
  • TLS 1.2–compatible clients

6. Decommission old domain​

Once you have:

  • Verified data in the new domain
  • Switched all traffic to the new domain
  • Observed stable operations

Delete old domain:

aws es delete-elasticsearch-domain \
  --domain-name OLD_DOMAIN_NAME

If instead you actually meant ElastiCache, say so and I’ll give the corresponding aws elasticache CLI steps.

Using Python

You’re mixing two different services in the question, so I’ll answer for what you explicitly asked: AWS Elasticsearch / OpenSearch Service (not ElastiCache).

For Amazon Elasticsearch / OpenSearch, “encrypted at rest and in transit” means:

  • At rest: EncryptAtRestOptions.Enabled = True
  • Node-to-node: NodeToNodeEncryptionOptions.Enabled = True
  • Client-to-domain (in transit): HTTPS-only endpoint (DomainEndpointOptions.EnforceHTTPS = True)

You cannot turn on at-rest encryption on an existing domain where it was disabled at creation; you must create a new domain with encryption and migrate your data.

Below are step‑by‑step instructions and Python (boto3) examples.

1. Prerequisites​

  1. Install boto3 and configure AWS credentials:
pip install boto3
aws configure  # or use env vars/instance role
  1. Use Python 3 and import boto3:
import boto3

es = boto3.client("es")  # For OpenSearch service; still named "es" in boto3

2. Check current domain encryption settings​

domain_name = "your-domain-name"

response = es.describe_elasticsearch_domain(
    DomainName=domain_name
)

domain_status = response["DomainStatus"]

encrypt_at_rest = domain_status.get("EncryptionAtRestOptions", {}).get("Enabled")
node_to_node = domain_status.get("NodeToNodeEncryptionOptions", {}).get("Enabled")
endpoint_opts = domain_status.get("DomainEndpointOptions", {})
enforce_https = endpoint_opts.get("EnforceHTTPS")

print("Encrypt at rest:", encrypt_at_rest)
print("Node-to-node:", node_to_node)
print("Enforce HTTPS:", enforce_https)

If Encrypt at rest or Node-to-node is False or missing, you must create a new domain.

3.1. Decide on new domain name and region​

SOURCE_DOMAIN = "your-domain-name"
TARGET_DOMAIN = "your-new-secure-domain-name"

es = boto3.client("es", region_name="us-east-1")  # choose your region

3.2. Read configuration from existing domain​

src = es.describe_elasticsearch_domain(DomainName=SOURCE_DOMAIN)["DomainStatus"]

ebs_opts = src.get("EBSOptions", {})
cluster_cfg = src.get("ElasticsearchClusterConfig", {})
vpc_options = src.get("VPCOptions", {})
access_policies = src.get("AccessPolicies", "{}")  # JSON string
snapshot_opts = src.get("SnapshotOptions", {})
engine_version = src.get("ElasticsearchVersion", "OpenSearch_1.3")  # or whatever you use

3.3. Create new domain with encryption enabled​

resp = es.create_elasticsearch_domain(
    DomainName=TARGET_DOMAIN,
    ElasticsearchVersion=engine_version,
    ElasticsearchClusterConfig=cluster_cfg,
    EBSOptions=ebs_opts,
    VPCOptions=vpc_options,  # omit if public domain
    SnapshotOptions=snapshot_opts,
    AccessPolicies=access_policies,
    EncryptionAtRestOptions={
        "Enabled": True,
        # Optionally specify a custom KMS key:
        # "KmsKeyId": "arn:aws:kms:region:account-id:key/key-id"
    },
    NodeToNodeEncryptionOptions={
        "Enabled": True
    },
    DomainEndpointOptions={
        "EnforceHTTPS": True,
        "TLSSecurityPolicy": "Policy-Min-TLS-1-2-2019-07"  # recommended
    }
)
print("Creating domain:", resp["DomainStatus"]["DomainName"])

Wait for the domain to become active:

import time

def wait_for_domain(domain):
    while True:
        status = es.describe_elasticsearch_domain(DomainName=domain)["DomainStatus"]
        if status["Processing"] is False:
            print("Domain ready:", domain)
            break
        print("Waiting for domain to be ready...")
        time.sleep(60)

wait_for_domain(TARGET_DOMAIN)

4. Migrate data from old (unencrypted) to new (encrypted) domain​

Use a migration method like:

  • reindex from remote (if supported by your engine version), or
  • Logstash, or
  • a custom Python script using elasticsearch/opensearch-py client.

Example using reindex from remote (conceptual, done via HTTP, not boto3):

  1. Get endpoints:
src_endpoint = src["Endpoints"]["vpc"] if "vpc" in src["Endpoints"] else src["Endpoint"]
tgt_status = es.describe_elasticsearch_domain(DomainName=TARGET_DOMAIN)["DomainStatus"]
tgt_endpoint = tgt_status["Endpoints"].get("vpc", tgt_status.get("Endpoint"))
  1. Use an HTTP client (e.g., requests) or opensearch-py to call _reindex from target domain:
from opensearchpy import OpenSearch

src_client = OpenSearch(
    hosts=[{"host": src_endpoint, "port": 443}],
    http_auth=("user", "password"),  # or IAM auth plugin, etc.
    use_ssl=True,
    verify_certs=True,
)

tgt_client = OpenSearch(
    hosts=[{"host": tgt_endpoint, "port": 443}],
    http_auth=("user", "password"),
    use_ssl=True,
    verify_certs=True,
)

# Example: reindex one index
body = {
  "source": {
    "remote": {
      "host": f"https://{src_endpoint}"
    },
    "index": "my-index"
  },
  "dest": {
    "index": "my-index"
  }
}

tgt_client.transport.perform_request("POST", "/_reindex", body=body, params={"wait_for_completion": "true"})

Repeat for all indices. Validate data, aliases, index settings, etc.

5. Switch traffic and delete old domain​

  1. Update any clients, apps, or DNS records to use the new domain endpoint (https://<new-endpoint>).
  2. Test thoroughly.
  3. Delete the old domain once you’re sure:
es.delete_elasticsearch_domain(DomainName=SOURCE_DOMAIN)

6. If encryption is already enabled but HTTPS is not​

You can update the endpoint options in-place:

domain_name = "your-domain-name"

es.update_elasticsearch_domain_config(
    DomainName=domain_name,
    DomainEndpointOptions={
        "EnforceHTTPS": True,
        "TLSSecurityPolicy": "Policy-Min-TLS-1-2-2019-07"
    }
)
wait_for_domain(domain_name)

This enforces in-transit encryption for clients; node-to-node and at-rest must have been enabled at creation time.

Summary​

  • Use create_elasticsearch_domain with:
    • EncryptionAtRestOptions.Enabled = True
    • NodeToNodeEncryptionOptions.Enabled = True
    • DomainEndpointOptions.EnforceHTTPS = True
  • If encryption wasn’t enabled originally, create a new domain and migrate data.
  • Manage all of this via boto3 as shown above.
Using Terraform
resource "aws_opensearch_domain" "THIS_DOMAIN" {
  domain_name = "YOUR_DOMAIN_NAME" # replace with your domain name

  engine_version = "OpenSearch_2.11" # replace with your required engine version

  cluster_config {
    instance_type = "m6g.large.search" # replace with your desired instance type
    # other cluster config as needed
  }

  # Enable encryption at rest (forces replacement if changed from false -> true)
  encrypt_at_rest {
    enabled    = true
    kms_key_id = "YOUR_KMS_KEY_ARN_OR_ID" # optional; omit to use AWS managed key
  }

  # Enable node-to-node encryption (in-transit inside the cluster; forces replacement if changed)
  node_to_node_encryption {
    enabled = true
  }

  # Enforce HTTPS for client connections (encryption in transit)
  domain_endpoint_options {
    enforce_https       = true
    tls_security_policy = "Policy-Min-TLS-1-2-2019-07"
  }

  # add/keep any other settings you already manage here (e.g., access_policies, vpc_options, etc.)
}

Changing encrypt_at_rest.enabled or node_to_node_encryption.enabled from false to true forces replacement of the OpenSearch/Elasticsearch domain, which can cause downtime unless you manage cutover (e.g., with blue/green or DNS).

To verify, terraform plan should show encrypt_at_rest.enabled = true, node_to_node_encryption.enabled = true, and domain_endpoint_options.enforce_https = true, with a resource replacement if you are enabling these on an existing unencrypted domain.