> ## Documentation Index > Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Elastic cache encrypted at rest and transit remediation ### Triage and Remediation ### Remediation Below are step‑by‑step console instructions to enable **encryption at rest** and **encryption in transit** for **Amazon OpenSearch Service / Amazon Elasticsearch Service** (the managed service often called “AWS Elasticsearch”). > Important: Some encryption options **cannot be changed on an existing domain**. If the options are grayed out, you must **create a new domain** with encryption enabled and migrate data. *** ## 1. Check Current Encryption Settings 1. Sign in to the **AWS Management Console**. 2. Go to **Amazon OpenSearch Service** (or “Amazon Elasticsearch Service” if you’re on an older console). 3. In the left pane, choose **Domains**. 4. Click your domain name. 5. On the domain’s detail page: * Look at **Security** or **Encryption** sections: * **Encryption at rest**: check if it’s **Enabled**. * **Node-to-node encryption**: check if it’s **Enabled**. * **Domain endpoint**: verify if it’s using **HTTPS** only. If any of these are disabled and you cannot edit them, proceed to create a new domain. *** ## 2. Enable Encryption (New Domain – Recommended if Current One Isn’t Encrypted) ### 2.1 Create a New Domain with Encryption 1. In the **OpenSearch Service** console, click **Create domain**. 2. **Engine version**: choose your required OpenSearch/Elasticsearch version. 3. **Domain name**: enter a unique name (e.g., `my-secure-domain`). 4. Continue through: * **Network**: * Choose **VPC access** if possible (recommended). * **Data nodes**: choose instance type and count. ### 2.2 Enable Encryption at Rest 1. Scroll to the **Data protection** or **Encryption** section. 2. Check **Enable encryption at rest**. 3. Choose a **KMS key**: * Use the **AWS managed key** or * Choose a **customer-managed CMK** in KMS. 4. (Optional) Enable **Auto-Tune** and other performance settings as needed. ### 2.3 Enable Node-to-Node Encryption (In-Transit Within the Cluster) 1. In the same security/encryption section, check **Enable node-to-node encryption**. * This encrypts traffic between cluster nodes. ### 2.4 Enforce HTTPS for Client Connections (In Transit from Clients) 1. In the **Domain endpoint** or **Network**/“Security” section: * Ensure **Require HTTPS** is selected (or equivalent option that disallows HTTP). * If there’s an **Endpoint security** or **TLS policy** field, select a modern TLS policy (e.g., `Policy-Min-TLS-1-2-2019-07`). 2. Configure **access policy**: * Use **fine-grained access control** + IAM, or * Restrict by **VPC** and security groups, or * If using a public endpoint, restrict by IAM and/or source IP. 3. Click **Next**, review all settings, then click **Create**. AWS will now create the secure domain (this can take several minutes). *** ## 3. Migrate Data from Old Domain to New Secure Domain 1. Once the **new domain is Active**, note its **HTTPS endpoint**. 2. From an EC2 instance, container, or your workstation (with network access to both domains), use: * **reindex from remote** (if supported by your version), or * A tool such as **Logstash**, **Elasticsearch/OpenSearch reindex APIs**, or a custom script: * Read from `http(s)://old-domain-endpoint` * Write to `https://new-domain-endpoint` 3. Update: * Application configuration * Lambda functions * Log shippers (Filebeat, Fluentd, CloudWatch Logs subscription filters, etc.) to use the **new domain’s HTTPS endpoint**. 4. Validate: * Index and document counts between old and new domains. * Application functionality and performance. 5. When fully migrated and tested, **delete** the old (unencrypted) domain from the console. *** ## 4. If You Must Modify an Existing Domain 1. In the **OpenSearch Service** console, open your domain. 2. Click **Edit** (or **Edit domain**). 3. Look for: * **Encryption at rest**: Try to select **Enable encryption at rest** and choose a KMS key. * **Node-to-node encryption**: Try to select **Enable**. * **Require HTTPS**: Ensure it is turned on. 4. If any are **grayed out or locked**, AWS does not allow changing them in-place for that domain → you must use the **new domain + migration** approach above. 5. Save/Apply changes and wait for the domain to finish **updating**. *** ## 5. Verify Encryption 1. On the domain’s details page: * **Encryption at rest**: shows **Enabled** with the KMS key. * **Node-to-node encryption**: shows **Enabled**. * **Endpoint**: begins with `https://` and **Only HTTPS allowed**. 2. From a client: * Ensure all connections use the **HTTPS endpoint**. * Confirm TLS negotiation (e.g., using `curl -v https://your-domain-endpoint`). *** If you tell me whether you’re on the OpenSearch or legacy Elasticsearch console and whether the domain is in a VPC or public, I can tailor the exact console menu/field names for your environment. For Amazon Elasticsearch / OpenSearch, encryption at rest and in-transit **cannot be turned on for an existing domain**. You must create a new domain with encryption enabled, migrate data, then cut over. Below are concise, CLI-focused steps. *** ### 1. Capture current domain config (for reference) ```bash theme={null} aws es describe-elasticsearch-domain \ --domain-name OLD_DOMAIN_NAME \ > old-domain-config.json ``` Use this JSON to copy any important settings (instance type, count, EBS size, access policy, etc.) into the new domain command. *** ### 2. Create a new domain with encryption enabled Key options: * `--encryption-at-rest-options Enabled=true` * `--node-to-node-encryption-options Enabled=true` * `--domain-endpoint-options EnforceHTTPS=true,TLSecurityPolicy=Policy-Min-TLS-1-2-2019-07` Example (adjust values as needed): ```bash theme={null} aws es create-elasticsearch-domain \ --domain-name NEW_DOMAIN_NAME \ --elasticsearch-version 7.10 \ --elasticsearch-cluster-config InstanceType=m5.large.elasticsearch,InstanceCount=2,ZoneAwarenessEnabled=true \ --ebs-options EBSEnabled=true,VolumeType=gp3,VolumeSize=200 \ --encryption-at-rest-options Enabled=true \ --node-to-node-encryption-options Enabled=true \ --domain-endpoint-options EnforceHTTPS=true,TLSecurityPolicy=Policy-Min-TLS-1-2-2019-07 \ --access-policies file://access-policy.json ``` To use a specific KMS key for encryption at rest: ```bash theme={null} --encryption-at-rest-options Enabled=true,KmsKeyId=YOUR_KMS_KEY_ARN ``` Wait for the domain to be active: ```bash theme={null} aws es describe-elasticsearch-domain \ --domain-name NEW_DOMAIN_NAME \ --query "DomainStatus.Processing" ``` Repeat until it returns `false`. *** ### 3. Verify encryption settings ```bash theme={null} aws es describe-elasticsearch-domain \ --domain-name NEW_DOMAIN_NAME \ --query "DomainStatus.{AtRest:EncryptionAtRestOptions,NodeToNode:NodeToNodeEncryptionOptions,EndpointOptions:DomainEndpointOptions}" ``` Confirm: * `EncryptionAtRestOptions.Enabled = true` * `NodeToNodeEncryptionOptions.Enabled = true` * `DomainEndpointOptions.EnforceHTTPS = true` *** ### 4. Migrate data from old to new domain Use reindex-from-remote via the Elasticsearch/OpenSearch API (not CLI) from the **new** domain, for each index: ```bash theme={null} curl -X POST "https://NEW_DOMAIN_ENDPOINT/_reindex?wait_for_completion=false" \ -H 'Content-Type: application/json' \ -d '{ "source": { "remote": { "host": "https://OLD_DOMAIN_ENDPOINT" }, "index": "old-index-name" }, "dest": { "index": "new-index-name" } }' ``` Repeat for all indices (you can keep same index names if desired). *** ### 5. Update clients to use HTTPS endpoint of new domain From CLI, get the endpoint: ```bash theme={null} aws es describe-elasticsearch-domain \ --domain-name NEW_DOMAIN_NAME \ --query "DomainStatus.Endpoints" ``` Update all applications, Lambdas, etc. to use: * The **HTTPS** endpoint of `NEW_DOMAIN_NAME` * TLS 1.2–compatible clients *** ### 6. Decommission old domain Once you have: * Verified data in the new domain * Switched all traffic to the new domain * Observed stable operations Delete old domain: ```bash theme={null} aws es delete-elasticsearch-domain \ --domain-name OLD_DOMAIN_NAME ``` *** If instead you actually meant **ElastiCache**, say so and I’ll give the corresponding `aws elasticache` CLI steps. You’re mixing two different services in the question, so I’ll answer for what you explicitly asked: **AWS Elasticsearch / OpenSearch Service** (not ElastiCache). For Amazon Elasticsearch / OpenSearch, “encrypted at rest and in transit” means: * **At rest**: `EncryptAtRestOptions.Enabled = True` * **Node-to-node**: `NodeToNodeEncryptionOptions.Enabled = True` * **Client-to-domain (in transit)**: HTTPS-only endpoint (`DomainEndpointOptions.EnforceHTTPS = True`) You cannot turn on at-rest encryption on an existing domain where it was disabled at creation; you must create a **new domain** with encryption and migrate your data. Below are step‑by‑step instructions and Python (boto3) examples. *** ## 1. Prerequisites 1. Install boto3 and configure AWS credentials: ```bash theme={null} pip install boto3 aws configure # or use env vars/instance role ``` 2. Use Python 3 and import boto3: ```python theme={null} import boto3 es = boto3.client("es") # For OpenSearch service; still named "es" in boto3 ``` *** ## 2. Check current domain encryption settings ```python theme={null} domain_name = "your-domain-name" response = es.describe_elasticsearch_domain( DomainName=domain_name ) domain_status = response["DomainStatus"] encrypt_at_rest = domain_status.get("EncryptionAtRestOptions", {}).get("Enabled") node_to_node = domain_status.get("NodeToNodeEncryptionOptions", {}).get("Enabled") endpoint_opts = domain_status.get("DomainEndpointOptions", {}) enforce_https = endpoint_opts.get("EnforceHTTPS") print("Encrypt at rest:", encrypt_at_rest) print("Node-to-node:", node_to_node) print("Enforce HTTPS:", enforce_https) ``` If `Encrypt at rest` or `Node-to-node` is `False` or missing, you must create a new domain. *** ## 3. Create a new encrypted domain (recommended path) ### 3.1. Decide on new domain name and region ```python theme={null} SOURCE_DOMAIN = "your-domain-name" TARGET_DOMAIN = "your-new-secure-domain-name" es = boto3.client("es", region_name="us-east-1") # choose your region ``` ### 3.2. Read configuration from existing domain ```python theme={null} src = es.describe_elasticsearch_domain(DomainName=SOURCE_DOMAIN)["DomainStatus"] ebs_opts = src.get("EBSOptions", {}) cluster_cfg = src.get("ElasticsearchClusterConfig", {}) vpc_options = src.get("VPCOptions", {}) access_policies = src.get("AccessPolicies", "{}") # JSON string snapshot_opts = src.get("SnapshotOptions", {}) engine_version = src.get("ElasticsearchVersion", "OpenSearch_1.3") # or whatever you use ``` ### 3.3. Create new domain with encryption enabled ```python theme={null} resp = es.create_elasticsearch_domain( DomainName=TARGET_DOMAIN, ElasticsearchVersion=engine_version, ElasticsearchClusterConfig=cluster_cfg, EBSOptions=ebs_opts, VPCOptions=vpc_options, # omit if public domain SnapshotOptions=snapshot_opts, AccessPolicies=access_policies, EncryptionAtRestOptions={ "Enabled": True, # Optionally specify a custom KMS key: # "KmsKeyId": "arn:aws:kms:region:account-id:key/key-id" }, NodeToNodeEncryptionOptions={ "Enabled": True }, DomainEndpointOptions={ "EnforceHTTPS": True, "TLSSecurityPolicy": "Policy-Min-TLS-1-2-2019-07" # recommended } ) print("Creating domain:", resp["DomainStatus"]["DomainName"]) ``` Wait for the domain to become active: ```python theme={null} import time def wait_for_domain(domain): while True: status = es.describe_elasticsearch_domain(DomainName=domain)["DomainStatus"] if status["Processing"] is False: print("Domain ready:", domain) break print("Waiting for domain to be ready...") time.sleep(60) wait_for_domain(TARGET_DOMAIN) ``` *** ## 4. Migrate data from old (unencrypted) to new (encrypted) domain Use a migration method like: * **reindex from remote** (if supported by your engine version), or * **Logstash**, or * a custom Python script using `elasticsearch`/`opensearch-py` client. Example using reindex from remote (conceptual, done via HTTP, not boto3): 1. Get endpoints: ```python theme={null} src_endpoint = src["Endpoints"]["vpc"] if "vpc" in src["Endpoints"] else src["Endpoint"] tgt_status = es.describe_elasticsearch_domain(DomainName=TARGET_DOMAIN)["DomainStatus"] tgt_endpoint = tgt_status["Endpoints"].get("vpc", tgt_status.get("Endpoint")) ``` 2. Use an HTTP client (e.g., `requests`) or `opensearch-py` to call `_reindex` from target domain: ```python theme={null} from opensearchpy import OpenSearch src_client = OpenSearch( hosts=[{"host": src_endpoint, "port": 443}], http_auth=("user", "password"), # or IAM auth plugin, etc. use_ssl=True, verify_certs=True, ) tgt_client = OpenSearch( hosts=[{"host": tgt_endpoint, "port": 443}], http_auth=("user", "password"), use_ssl=True, verify_certs=True, ) # Example: reindex one index body = { "source": { "remote": { "host": f"https://{src_endpoint}" }, "index": "my-index" }, "dest": { "index": "my-index" } } tgt_client.transport.perform_request("POST", "/_reindex", body=body, params={"wait_for_completion": "true"}) ``` Repeat for all indices. Validate data, aliases, index settings, etc. *** ## 5. Switch traffic and delete old domain 1. Update any clients, apps, or DNS records to use the **new domain endpoint** (`https://`). 2. Test thoroughly. 3. Delete the old domain once you’re sure: ```python theme={null} es.delete_elasticsearch_domain(DomainName=SOURCE_DOMAIN) ``` *** ## 6. If encryption is already enabled but HTTPS is not You can **update** the endpoint options in-place: ```python theme={null} domain_name = "your-domain-name" es.update_elasticsearch_domain_config( DomainName=domain_name, DomainEndpointOptions={ "EnforceHTTPS": True, "TLSSecurityPolicy": "Policy-Min-TLS-1-2-2019-07" } ) wait_for_domain(domain_name) ``` This enforces **in-transit encryption for clients**; node-to-node and at-rest must have been enabled at creation time. *** ### Summary * Use `create_elasticsearch_domain` with: * `EncryptionAtRestOptions.Enabled = True` * `NodeToNodeEncryptionOptions.Enabled = True` * `DomainEndpointOptions.EnforceHTTPS = True` * If encryption wasn’t enabled originally, create a new domain and migrate data. * Manage all of this via boto3 as shown above. ```hcl theme={null} resource "aws_opensearch_domain" "THIS_DOMAIN" { domain_name = "YOUR_DOMAIN_NAME" # replace with your domain name engine_version = "OpenSearch_2.11" # replace with your required engine version cluster_config { instance_type = "m6g.large.search" # replace with your desired instance type # other cluster config as needed } # Enable encryption at rest (forces replacement if changed from false -> true) encrypt_at_rest { enabled = true kms_key_id = "YOUR_KMS_KEY_ARN_OR_ID" # optional; omit to use AWS managed key } # Enable node-to-node encryption (in-transit inside the cluster; forces replacement if changed) node_to_node_encryption { enabled = true } # Enforce HTTPS for client connections (encryption in transit) domain_endpoint_options { enforce_https = true tls_security_policy = "Policy-Min-TLS-1-2-2019-07" } # add/keep any other settings you already manage here (e.g., access_policies, vpc_options, etc.) } ``` Changing `encrypt_at_rest.enabled` or `node_to_node_encryption.enabled` from `false` to `true` forces replacement of the OpenSearch/Elasticsearch domain, which can cause downtime unless you manage cutover (e.g., with blue/green or DNS). To verify, `terraform plan` should show `encrypt_at_rest.enabled = true`, `node_to_node_encryption.enabled = true`, and `domain_endpoint_options.enforce_https = true`, with a resource replacement if you are enabling these on an existing unencrypted domain.